Next Article in Journal
Effect of Whole-Body Vibration on Serum Levels of Brain Derived Neurotrophic Factor and Cortisol in Young, Healthy Women
Previous Article in Journal
Job Dissatisfaction and Its Predictors among Healthcare Workers of ‘Type 2 Health Clinics’ in North-Eastern Malaysia
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Encouraging Healthier Food and Beverage Purchasing and Consumption: A Review of Interventions within Grocery Retail Settings

1
Center for Research in Education and Social Policy, University of Delaware, Newark, DE 19716, USA
2
Temple University, Philadelphia, PA 19122, USA
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(23), 16107; https://doi.org/10.3390/ijerph192316107
Submission received: 4 November 2022 / Revised: 23 November 2022 / Accepted: 24 November 2022 / Published: 1 December 2022

Abstract

:
This review identifies the most promising intervention strategies for promoting the purchase and consumption of healthier items within U.S. grocery retail settings, with a particular focus on those strategies that may be most effective when implemented within SNAP-authorized retail settings. Searches of nine electronic databases, as well as forward and backward searches, yielded 1942 studies. After being screened, 73 peer-reviewed academic articles were identified for inclusion. Of these, 33 analyzed single-component interventions, while 40 assessed multi-component interventions. The following unique intervention types were considered as evaluated in these studies for their ability to increase healthy item purchasing and consumption: (1) nutrition scoring, (2) nutritional messaging, (3) non-nutritional messaging, (4) endcaps and secondary placement, (5) point-of-sale interventions, (6) increased stocking, (7) food tasting and demonstrations, (8) nutrition education, and (9) placement on shelf interventions. Nutritional scoring and nutritional messaging emerged as the most rigorously tested and effective intervention strategies. Other strategies warrant more research attention. Simple intervention strategies, as opposed to complex ones, yield the most successful results and minimize shopper burden. Therefore, these strategies should be reviewed for policy implementation within SNAP-authorized grocery retailers.

1. Introduction

As concerns for global dietary quality persist, public health experts continue to evaluate the role that individuals’ environments play in shaping their health behaviors [1,2,3]. Internationally, concerns for malnutrition in all its forms, has brought attention to the important connections between obesity and undernutrition on health. Preventing poor dietary quality is a leading priority for a wide variety of chronic diseases, and a critical disease mitigation strategy globally. Food retail is an important nexus between food policy and the consumer and as such has remained a critical facet of food environment intervention [4,5].
Grocery retail stores are one aspect of individuals’ environments that have the potential to directly impact their dietary quality [6,7]. Research demonstrates that these stores vary widely in their availability of, and strategies used to promote, healthy options [8,9,10]. Findings from individual studies demonstrate that manipulating certain aspects of these environments (e.g., placing healthy items at eye level, installing freezers to increase a store’s stock of fresh fruits and vegetables) can promote healthier purchasing behaviors and consumption patterns among shoppers [11]. Multiple studies have been conducted across a range of countries which examine how supermarket environments influence promotion and purchasing of less healthy food, however fewer examine the promotion of healthier items [12,13].
In the United States, there is potential to impact the retail environment at the federal level through the Supplemental Nutrition Assistance Program (SNAP), in which over 250,000 grocery retailers participate across the United States. SNAP-authorized grocery retailers profit substantially from the program, with $61 billion in SNAP sales representing approximately 8 percent of grocery sales industry-wide [14]. Despite evidence demonstrating that interventions within grocery retail settings can improve the healthfulness of shoppers’ purchases and consumption, the U.S. Department of Agriculture (USDA), which has the authority to authorize SNAP grocery retailers and establish eligibility criteria, requires little from grocery retailers to participate beyond stocking minimal staple foods.
At the same time, disparities in dietary quality by income persist. For example, research demonstrates that disparities in the price of nutrient-dense foods versus less nutrient-dense foods are rising [15]. Specifically, studies show that ultra-processed foods tend to be less expensive, though more energy dense, than unprocessed foods [16]. These cost barriers contribute to socioeconomic inequities in dietary quality, leading individuals with lower incomes to have less choice in food purchasing, and worse dietary outcomes [15,17].
The purpose of the present study is to provide an updated review of healthy food marketing strategies in retail stores, with a focus on research conducted in stores accepting SNAP benefits [18]. Further, the study goes beyond prior reviews, which broadly characterize the effectiveness of interventions according to the 4 P’s of marketing, to examine specific strategies and their effect on consumer purchasing, consumption or sales. Such information is intended to guide future research and interventions in retail settings.

2. Methods

This review used the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.

2.1. Search Strategy

The authors used several methods to ensure a thorough and comprehensive review of the literature on in-store marketing interventions for healthy food and beverage promotion. First, a list of inclusion criteria was created to identify papers to be included in the review sample. Second, a list of key terms was created to search for studies. Third, appropriate databases were identified for the search based on the database topics. Finally, a database search was conducted to identify inclusion articles, using both forward and backward searches for each inclusion article. Below are the processes used to identify studies for this review.

2.2. Inclusion Criteria

The studies included are original empirical research published between 1 January 2010 and 6 June 2022, in English, and from the United States. The authors chose to focus on the United States for this review because SNAP-authorized retail contexts are unique to this country. Studies were researcher- or retailor-initiated, conducted inside the physical grocery retail environment, and manipulated the grocery retail environment. Evaluations could be quantitative or use mixed methods. All interventions had to include at least one of the following outcomes: (1) purchase-related (i.e., objective store sales data, objective food purchasing data, customer receipts, and survey self-reported purchases or expenditures, store sales, or intent to purchase), and/or (2) consumption-related (i.e., food frequency questionnaires (FFQ), 24 h dietary recalls, food diary, Veggie MeterTM or other biometrics, or other self-reported diet/consumption or intent to eat surveys).

2.3. Exclusion Criteria

Interventions were excluded if they were implemented by an entity other than a researcher or grocery retailer (e.g., price intervention at the wholesale level or front-of-pack labels initiated by a food company), if they did not occur inside the physical grocery retail environment (e.g., restaurants, schools, mobile food trucks, online, and laboratory), or if they did not manipulate the grocery retail environment (e.g., grocery store tours). Additionally, articles were excluded that solely utilized price-based interventions, namely those implementing a coupon or discount-based behavioral nudges.

2.4. Search Terms and Databases

Nine databases (i.e., Academic OneFile, Business Source Premier, CAB Abstracts, Communication and Mass Media Complete, Family and Society Studies Worldwide, PsycINFO, PubMed, Sociological Abstracts, and Web of Science) from a variety of sectors (i.e., agriculture, business, communication, health, and psychology) were searched. Key terms were constructed based on three concepts: (1) healthier food, (2) study design, and (3) setting. A variety of search terms were used to ensure articles would be included with nuanced differences in terms (e.g., healthy food vs. better-for-you) across sectors. The following key terms were used in all database searches:
Healthier food“health* food*” OR “healthy eating” OR “fruit*” OR “vegetable*” OR “low* fat” OR “low* sodium” OR “low* sugar” OR “low-fat” OR “low-sodium” OR “low-sugar” OR “better for you” OR “nutritio*”
Study design“intervention” OR “pilot” OR “experiment*”
Setting“supermarket*” OR “grocery store*” OR “corner store*” OR “bodega*” OR “retail environment”

2.5. Procedure of Article Search

The RefWorks database was used to organize all articles [19,20]. The searches were conducted by two authors and yielded 1942 studies (Figure 1). After excluding 865 duplicate articles, five coders reviewed each full-text article to determine eligibility and excluded 1026 studies. This review yielded 46 articles that met all inclusion criteria. Then, citation and bibliography searches were conducted with all 46 articles identifying an additional 27 articles for a final total of 73 articles. Of all the article articles reviewed, 38 journals were represented. The journals represented most frequently were Journal of Nutrition education and Behavior, Public Health Nutrition, and Preventive Medicine Reports.
After removing duplicates, five reviewers independently screened the title, abstract, and full text of the remaining 1077 articles. Reviewers discussed any differences and consulted a sixth reviewer, when necessary, and a consensus was reached. Five reviewers conducted forward and backward searches of the included articles. Titles and then full texts were reviewed to assess eligibility. Articles were abstracted and coded independently with five coders; discrepancies were discussed until a consensus was reached. Article abstractions included participants, study design, intervention description, intervention type, intervention setting, duration of intervention, data collection methods, outcome variables, and key findings.
Researchers reviewed the studies and categorized them by intervention type. The following unique intervention types were evaluated for their ability to increase healthy item purchasing and consumption: (1) nutrition scoring, (2) nutritional messaging, (3) non-nutritional messaging, (4) endcaps and secondary placement, (5) point-of-sale interventions (which includes healthy checkout interventions), (6) increased stocking, (7) food tasting and demonstrations, (8) nutrition education, and (9) placement on shelf interventions (Table 1). Articles were then coded either as single-component, meaning that they evaluated the effects of one intervention strategy in isolation, or multi-component, meaning that they evaluated the effects of two or more intervention strategies alongside each other.
Articles were also coded to identify whether the intervention was conducted within SNAP-authorized retail settings. SNAP acceptance was coded based on the following categorizations: (1) the study explicitly stated that the retailer was SNAP-authorized, (2) the study was highly likely to have been conducted in a SNAP-authorized retail setting (e.g., low-income neighborhoods, corner stores, bodegas, major supermarkets), and (3) the study was not likely to have been conducted in a SNAP-authorized retail setting (e.g., high-income neighborhood) or lacked sufficient information to make a determination.
Additionally, articles were coded to identify whether they were experimental, quasi-experimental, or pre-experimental. Articles were coded as experimental if participants were randomized to conditions, quasi-experimental if participants were not randomized to conditions but there was a comparison group, or pre-experimental if participants were not randomized to conditions and there was no comparison group.

3. Results

There were 73 peer-reviewed academic articles identified for inclusion in this review, each of which evaluated the healthy food and/or beverage purchasing or consumption-related effects of retail interventions. These studies employed nine unique intervention types (Table 1). In total, 33 studies analyzed single-component interventions, while 40 studies assessed multi-component interventions.

3.1. Single-Component Interventions

3.1.1. Nutrition Scoring

Eleven studies evaluated nutrition scoring (Table 2) [21,22,23,24,25,26,27,28,29,30,31]. The results of nutrition scoring interventions are largely positive. In 9 of 11 studies, at least one finding showed a significant positive effect on healthy purchasing or consumption. Among available research, findings suggest that simpler scoring systems are more effective than those with multiple nutrition facts. For example, the sole study of nutrition scoring using an experimental approach, conducted by Kiesel and Villas-Boas, found that no trans-fat labels significantly increased sales of treated products by 23 percent. Yet, labels with multiple measures of healthfulness were found to have a null effect on healthy purchasing, likely overwhelming shoppers with high information costs [26]. Five quasi-experimental studies echoed the experimental study’s increased effectiveness in simpler nutritional scoring systems, including studies of the NuVal and Guiding Stars program which both assign a single value to the healthfulness of a product [21,22,23,28,29]. For example, Rahkovsky et al. found that the Guiding Stars Program decreased sales of unstarred (least healthy) products at intervention stores by 2.58 percent, while also increasing sales of 1-star, 2-star, and 3-star cereals by 1.15 percent, 0.89 percent, and 0.54 percent. Five studies of nutrition scoring were pre-experimental [22,24,25,27,30]. Additionally, 10 studies collected objective data [21,22,24,25,26,27,28,29,30,31]. Of the eleven interventions that utilized a nutrition scoring component, ten were conducted in a retail setting that explicitly or likely accepts SNAP benefits [21,22,23,24,26,27,28,29,30,31].

3.1.2. Increased Stocking

Six studies evaluated single-component interventions in which stores increased their stock of healthy foods and beverages (Table 2) [32,33,34,35,36,37]. None of the studies evaluating increased stocking interventions were experimental. Less rigorous studies using quasi-experimental and pre-experimental designs demonstrate that, although increased stocking interventions tend to result in an increased supply of healthy foods and beverages offered in stores, the interventions generally do not have a significant impact on participants’ healthy food and beverage purchasing or consumption [35,36,37]. For example, the North Carolina Healthy Food Small Retailer Program (HFSRP) is an intervention in which small food stores were provided with up to $25,000 to spend on refrigeration units to increase their stocking of healthy foods and beverages (e.g., fresh fruits and vegetables, lean meats, whole grains, etc.) [32,33,35,36]. Results of quasi-experimental studies evaluating this intervention demonstrate that after one year of implementation, Healthy Food Supply (HFS) scores, a measure of availability of healthy items within a store, increased by 3.13 points between baseline and follow-up among HFSRP stores, whereas HFS scores decreased by 0.44 points in control stores. However, participants’ purchases, skin carotenoids, and self-reported consumption did not significantly differ between HFSRP stores and control stores. Additionally, 4 of 6 articles assessing increased stocking interventions collected objective purchasing data. Of the studies evaluating increased stocking interventions, four were conducted in stores that explicitly served SNAP customers [32,33,36,37], and one was located in a low-income community that likely served SNAP customers [34].

3.1.3. Nutritional Messaging

Five studies evaluated the effect of nutritional messaging techniques (Table 2) [38,39,40,41,42]. Two studies employed experimental designs [40,42]. Two studies were quasi-experimental [38,41], and one study was pre-experimental [39]. Four nutritional messaging studies measured outcomes using objective data [38,40,41,42]. Of the two experimental studies examining purchasing and sales data, both found mixed effects including a significant increase in healthy item purchasing despite no change in healthy item consumption [40,42]. Further, results were largely dependent on the food group under study [40]. Results from quasi- and pre-experimental studies, however, were predominantly positive. For example, Finnell et al. conducted a marketing campaign, 1% Low-Fat Milk Has Perks!, that included dairy case clings, souvenir buttons, and a handout with nutrition information entitled “Lactaid Factoids.” They found that the campaign significantly increased lower fat milk sales at intervention stores 4.8 percent over baseline [38]. Moreover, three of the five studies examining nutritional messaging were explicitly [42] or highly likely [41] to have been conducted in a retail setting that accepts SNAP benefits [40].

3.1.4. Non-Nutritional Messaging

Four studies evaluated the effect of non-nutritional messaging techniques yielding mixed results (Table 2) [43,44,45,46]. Of the two studies that employed experimental designs, one found positive effects on healthy food purchasing while the other yielded mixed effects [44]. Results of two quasi-experimental studies found positive and negative results [45,46]. In the most successful non-nutritional messaging intervention, Payne et al. applied behavioral economics principles and created placards for grocery carts that listed both the average number of produce items purchased at the store as well as the top ten fruits and vegetables purchased in that store to “give shoppers a specific idea of not only the appropriate or normal amount of fruits and vegetables to purchase, but also the most common types of fruits and vegetables purchased” [46]. Findings reported an increase in shopper spending on produce between 7.5 and 16 percent. However, Chapman et al. found non-nutritional messaging labels and floor stickers unsuccessful at increasing healthy purchasing [45]. Specifically, Chapman et al. found that both scarcity labeling of healthy items as well as floor labeling guiding shoppers to healthier store sections led to no significant increase in healthy food sales. One additional non-nutritional messaging approach, which included the piping of the smell of cookies through a grocery store generated counter-intuitive results, whereby consumers smelling the cookies purchased a significantly higher proportion of healthier items [44]. Of the four studies examining the effects of non-nutritional messaging, all four were explicitly [45,46] or highly likely [43,44] to have been conducted in a retail setting that accepts SNAP benefits.

3.1.5. Food Tasting and Demonstrations

Two studies evaluated interventions that were solely composed of food tasting and demonstrations (Table 2) [47,48]. Of these, one study was experimental [47] and one was pre-experimental [48]. Both studies reported positive effects on healthy food purchasing or consumption. The most rigorous study, which examined the effects of food tasting using an experimental design and objective data, posed the question “Can healthy samples given at a grocery store prompt healthier choices?” To answer this question, participants were divided into three groups: those that received an apple sample, those that received a cookie sample, and those that received no sample. It was found that participants who received an apple sample purchased a greater amount of fruits and vegetables (M = 2.78, SD = 2.15) than did participants who received a cookie sample (M = 2.17, SD = 2.26) or no sample (M = 2.22, SD = 2.15), providing support for the hypothesis that sampling a product that is considered healthy will increase the healthiness of purchases. Of the two articles that analyzed food tasting and demonstrations in the single-component intervention category, one article was explicitly conducted in a retail setting that accepts SNAP benefits [48]

3.1.6. Nutrition Education

Two studies examined the effectiveness of single-component nutrition education intervention strategies, both of which were pre-experimental (Table 2) [49,50]. Findings were both positive and null, with both studies examining the effects of nutrition education delivered via podcasts. As a result of this intervention, there was a “clear increase” in the purchase of foods emphasized in the podcasts, which 59 percent of participants (n = 102) purchased more of. Furthermore, 38 percent of participants (n = 66) had purchased none of the emphasized foods during the 6 months prior to the intervention but did so afterward. Notably, there was no significant relationship found between intervention-day responses regarding intent to purchase the emphasized foods and actual purchases [49]. A smaller pilot study conducted by the same authors corroborated the conclusion that nutrition education was effective at influencing actual and planned purchasing behavior [50]. Note that while the larger study reported objective store sales data, the smaller pilot study only included subjective self-report data. In conclusion, while findings are largely positive and tested in retail settings accepting SNAP benefits, the limited number of studies demonstrates that there is limited evidence of the overall applicability of educational interventions to improve healthy purchasing and consumption.

3.1.7. Endcaps and Secondary Placement

Two studies examined the effectiveness of endcaps and secondary placement on increasing healthier purchasing and consumption in retail environments (Table 2) [51,52]. One of these studies was experimental [51] and one was quasi-experimental [52]; both yielded positive outcomes. Liu et al. measured the influence of endcaps on purchasing of indulgent, healthy and neutral foods in 12 different convenience stores for which SNAP redemption is unspecified [51]. Featuring healthy products alone on the endcap increased healthy product sales by one-third. In the other study which evaluated a similar strategy, Payne & Niculescu found that SNAP participants purchased significantly more healthy items when they were displayed on endcaps. However, total produce spending in both the general population and SNAP recipients did not significantly increase [52].

3.1.8. Point-of-Sale Interventions

One study implemented a point-of-sale intervention, which occurred directly in a checkout line (Table 2) [53]. Adjoian et al., using objective purchasing data, found that healthy checkout lanes are an effective method for increasing healthy purchases. For example, SNAP participants bought more than twice the amount of healthy food items (e.g., nuts, fresh produce, granola bars, etc.) in the intervention checkout lane as opposed to the control checkout lane [53].

3.1.9. Placement on Shelf

There were no single-component studies examining placement on shelf interventions (Table 2). However, 13 studies evaluated placement on shelf interventions in tandem with other intervention strategies (e.g., endcaps and secondary placement, increased stocking, etc.), and are described below in the multi-component section.

3.2. Multi-Component Intervention

A total of 40 studies analyzed the effects of multi-component interventions on healthy food item purchasing and consumption (Table 3) [32,35,39,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88]. Of these, 16 studies were experimental [55,59,60,61,62,63,64,72,73,74,82,83,86,88,89,90], 11 were quasi-experimental [35,56,57,58,65,66,67,75,76,77,84], and 13 were pre-experimental [32,39,54,68,69,70,71,78,79,80,81,85,87]. A total of 17 multi-component studies collected objective outcomes data [32,35,39,55,56,61,62,67,68,69,72,73,75,78,82,84,89]. Additionally, of the studies evaluating multi-component interventions, 14 were conducted in stores that explicitly served SNAP customers [32,54,58,60,64,67,68,72,73,78,82,85,86,89], 15 were located in a low-income community that likely served SNAP customers [39,55,56,57,62,63,69,74,75,76,77,80,83,87,90], 5 were conducted in a corner store or bodega without any neighborhood demographics or income levels disclosed [35,61,66,71,88] and 3 were conducted in major supermarkets [59,79,81]. Results of multi-component intervention studies are presented below, with the most rigorous studies (those with experimental and objective data) presented first (Table A1).
Analysis of the 7 experimental studies of multi-component interventions using objective data generally showed consistency in outcomes when compared to the single component study results. For example, multi-component studies which utilized nutrition scoring and nutrition education were also successful in a multi-component intervention. For example, Milliron et al. conducted a randomized control trial evaluating a multi-component intervention using objective sales data and found that pairing a brief front-of-store nutrition education effort with the EatSmart nutritional scoring system, which marked healthy products across the store, positively influenced the number of servings of fruit and vegetables purchased by shoppers. Specifically, this multi-component intervention resulted in shoppers purchasing 9.5 more servings of whole fruit and 4.8 more servings of dark green/bright yellow vegetables than the control group [55].
In contrast to Milliron et al., who evaluated a multi-component intervention including just two strategies, Foster et al. conducted a randomized controlled trial evaluating a multi-component intervention comprising five intervention strategies: placement on shelf, endcaps and secondary placement, food tasting and demonstrations, nutritional marketing, and increased stocking. Placement on shelf was the dominant strategy within this intervention, whereas other strategies (e.g., food tasting and demonstrations) were only applied to one of the five targeted food and beverage categories. Store sales data demonstrate that sales of skim and 1% milk, water (in aisle and at checkout), and two out of three types of frozen meals significantly increased compared to control store sales. Null results were found for the other categories (i.e., cereal, whole or 2% milk, beverages, or diet beverages). The authors concluded that simple, straightforward placement and product availability strategies were effective at significantly influencing the purchase of healthier items in several food and beverage categories [82].
In line with findings from Foster et al., several other studies show that simple placement strategies proved effective over more complex strategies. In an experimental study with objective outcome data, Williams et al. employed a wide variety of intervention techniques to encourage healthy purchasing, of which the most promising results stemmed from a low-cost placement technique. Williams et al. analyzed a multi-component intervention that combined nutritional marketing, increased stocking, and endcaps and secondary placement and found that shoppers on average doubled their weekly spending on the fruit and vegetable basket from $2.00–$3.00 to $5.00–$6.00 as a result [61].
While most multi-component interventions manipulate just one level of the food system (e.g., corner stores), researchers in Baltimore, MD manipulated several levels of the food system (e.g., wholesalers, corner stores, carry-outs, recreation centers, households) simultaneously in a community-wide effort to reduce obesity and generally found positive impacts on product availability but no effect on purchasing. Within this multi-level, multi-component intervention, called B’more Healthy Communities for Kids (BHCK), wholesalers, corner stores, and carry-outs were all asked to increase their stock of healthier products, which included low-fat, low-sugar snacks and beverages, fruits and vegetables, and whole grain products. Additionally, nutritional messaging techniques (e.g., posters and signage) were then used within corner stores and carry-outs to promote these items. Corner stores also implemented nutrition education and food tasting and demonstration strategies, which were led by BHCK interventionists and allowed customers to sample healthy food items. Intervention efforts at the wholesale level had a positive effect on sales of healthier items for resale. Intervention stores had larger increases in their stock of healthy items, as measured by the Healthy Food Availability Index, than did control scores (e.g., 5.65-point increase vs. 1.67-point increase). The intervention had no effect on adult caregivers’ healthy food purchasing [62].
Although increased stocking did not lead to increases in healthy purchasing when paired with nutritional messaging, nutrition education, and food tasting and demonstrations in the BHCK intervention [62], findings from Thorndike et al. show that efforts to improve available stock of healthy items can be successful when paired with other intervention strategies. In this randomized controlled study, three stores were randomly assigned to receive the multi-component intervention consisting of increased stocking, non-nutritional marketing, and placement on shelf, and three were randomly assigned to the control group. All stores were located in low-income communities and accepted SNAP and WIC. Each intervention store had a combination of interventions: A consultant that helped advise fresh fruit and vegetable maintenance, different supplies and advice such as display tips and items that would make fruits and vegetables more attractive to customers, and a service job provided to the stores such as refrigeration installation, or having the walls repainted. Findings showed an increase in fruit and vegetable sales by $40.00 per month by participants using WIC fruit/vegetable cash-value vouchers, whereas fruit and vegetable sales declined in the control group by $23.00 per month [73].

4. Discussion

Interventions to improve the healthfulness of individuals’ food and beverage purchasing and consumption within retail environments are varied and heterogeneous. Results of this review demonstrate that nutritional scoring and nutritional messaging are the most widely-researched and effective single-component healthy retail intervention strategies [24,29]. Furthermore, nutritional messaging interventions, including simple signage at the front of the store, corresponding with shelf tags for reinforcement, are generally effective [41]. Notably, both of these intervention strategies were commonly implemented within SNAP-authorized retail settings.
Two additional intervention strategies, food tasting and demonstrations and nutrition education, also demonstrate positive effects on healthy food and beverage purchasing and consumption, however, the literature in this area is sparse. Non-nutritional marketing is another intervention strategy commonly implemented within grocery retail settings, however, results from studies examining this strategy are mixed. Collectively there is emerging evidence for the use of interventions focused on endcaps and secondary placement to increase healthy purchasing and consumption, although only a small number of articles (n = 2) analyzing single-component interventions have assessed this approach alone. Finally, point-of-sale and placement on shelf interventions’ impact on healthy purchasing and consumption are difficult to discern because both strategies are incorporated predominantly as part of multi-component interventions. Additional research examining the unique contribution of placement and point-of-sale interventions is needed.
Findings from studies evaluating increased stocking interventions demonstrated that on its own, this strategy does not significantly influence shoppers’ purchasing and consumption of healthier items [35,36,37]. While increased stocking did not yield significant effects on shoppers’ purchasing and consumption of healthy foods and beverages on its own, research demonstrates that this intervention strategy may be effective when combined with other intervention types in multi-component interventions. Thus, more research regarding the effectiveness of increased stocking both on its own and with other interventions is needed.
Two additional factors to consider when evaluating the success of interventions within grocery retail environments are their potential scalability and sustainability [91]. Across both single- and multi-component studies reviewed, results demonstrate that simple interventions with low information costs and minimal barriers to implementation are more effective than complex strategies at increasing healthy purchasing and consumption behaviors among shoppers. Simple intervention strategies are easily understandable and require little effort from customers, whereas more complex strategies can be difficult, and laborious, for customers to interpret. Additionally, simple intervention strategies not only benefit customers but also retailers, as they are both feasible to implement and able to be retained over time, potentially increasing intervention sustainability.
Approximately 90 percent of studies (n = 66) analyzed in this review were likely to have been conducted in retail environments that accepted SNAP benefits indicating a high degree of intervention scalability for increasing SNAP-eligible populations purchasing and consumption of healthier items.

5. Limitations and Future Directions

Although most studies reviewed were likely to have been conducted in SNAP-authorized retail settings, outcomes of SNAP participants can often not be identified separately from non-SNAP participants. To identify whether these intervention strategies are effective with SNAP populations specifically, future studies can attempt to directly measure interventions’ impact on SNAP shoppers’ purchasing and consumption rates.
Furthermore, of the 73 studies reviewed, 34 percent of studies (n = 25) were conducted without a control or comparison group (Table 2), indicating a need for more rigorous methodological designs in grocery retail environment research. Furthermore, 55 percent of studies reviewed were multi-component interventions (n = 40), and the complexity of these studies made it difficult to parse out the effects of individual intervention strategies on shoppers’ purchasing and consumption of healthy items.
Of the 33 single-component studies reviewed, only three of nine intervention types (e.g., nutrition scoring, increased stocking, nutritional messaging) were assessed in five or more studies. All other intervention types (e.g., non-nutritional messaging, food tasting and demos, nutrition education, endcaps and secondary placement, point-of-sale interventions, and placement on shelf) were evaluated in fewer than five single-component studies, highlighting a need for further evidence to determine their effectiveness. Thus, the frequent use of multi-component interventions, and the lack of studies evaluating single-component interventions, complicates the generalizability of findings within grocery retail environment literature.
Another factor not analyzed in this paper that likely contributes to the scalability and sustainability of healthy retail interventions includes intervention cost. Costly, time-consuming, interventions are unlikely to be maintained long-term and could prohibit broader adoption of these efforts. Future research evaluating the effect of intervention cost on SNAP shoppers’ purchasing and consumption outcomes is needed.
Currently, SNAP-authorized retailers must only meet minimum stocking standards. The USDA should test additional intervention strategies and grocery retailer eligibility requirements to evaluate whether they improve the health of grocery retail environments and whether they have an impact on grocery retailer participation. Additional requirements may be reasonable given substantive grocery retailer benefits from SNAP and would have the potential to positively impact the grocery retail environment for all, given widespread grocery retailer participation in SNAP.

6. Conclusions

This review identifies intervention strategies implemented within U.S. grocery retail stores that are effective in improving the healthfulness of shoppers’ food and beverage purchasing and consumption. Of the intervention strategies reviewed, nutritional scoring and nutritional messaging were the most rigorously tested and effective. Therefore, these strategies should be reviewed for policy implementation within SNAP-authorized grocery retailers. Additionally, research demonstrates that simple interventions yield the most successful results and minimize shopper burden. Other strategies, such as nutrition education and food tastings and demonstrations, were also effective, though more research is needed to corroborate existing findings. On its own, increased stocking did not significantly change participants’ purchasing or consumption behaviors; however, this strategy shows promise when implemented in conjunction with some other strategies in multi-component interventions. More research is needed to understand whether endcaps and secondary placement, point-of-sale, and placement on shelf are successful.

Author Contributions

H.W. contributed to research design, analysis, and manuscript writing. M.M.H. contributed to data analysis, writing, and editing. N.K. contributed to data analysis, writing, and editing. I.G. contributed to data collection and manuscript preparation. A.K. oversaw project, conceptualized study, and contributed to paper writing and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by Healthy Eating Research, a national program of the Robert Wood Johnson Foundation.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Sales, Purchasing, and Intake/Consumption Results of Multi-Component Intervention Studies.
Table A1. Sales, Purchasing, and Intake/Consumption Results of Multi-Component Intervention Studies.
Study NameSales OutcomesPurchasing Outcomes Intake/Consumption Outcomes
Milliron, 2012 (+) Fresh fruit (+/null) Fresh vegetables
Gittelsohn et al., 2017(+/null) Healthy beverages, dairy, fruits and vegetables (fresh and dried), grains, nuts and seeds, protein, salty snacks
Williams et al., 2021(+/−) Fruits and vegetables (fresh, canned, or frozen) (+/null) Salads
Thorndike et al., 2017 (+) Fruits and vegetables(−) Fruits and vegetables
Wensel et al., 2019 (+/−/null) Healthy juice, dairy, fruits and vegetables (fresh), grains, infant foods and formula, protein
Foster et al., 2014 (+/null) Dairy (−) Cereal (+/−) Healthy frozen meals
Ayala et al., 2022 (+) Fruit (fresh)(+/null) Fruit (fresh)
Martinez-Donate et al., 2015 (+/null) Fruits and vegetables
Lent et al., 2014 (null) Healthy grains, beverages, snacks, protein
Trude et al., 2018 (+/null) Healthy dairy, grains, snacks, and fruits and vegetables (fresh or canned)
Bird Jernigan et al., 2019 (null) Fruits and vegetables, protein, snacks
Shin et al., 2015 (+/−/null) Healthy snacks, grains, nuts and seeds, and fruits and vegetables
Trude et al., 2019 (null) Fruits and vegetables (fresh or canned)(+/null) Fruits and vegetables (fresh or canned)
Gittelsohn et al., 2013 (+/null) Healthy grains(+/null) Healthy grains
Surkan et al., 2016 (+) Healthy grains, beverages, fruits and vegetables, snacks, and dairy
Kannan et al., 2020 (null) Healthy beverages, legumes, vegetables (fresh), and grains(+) Healthy beverages, legumes, vegetables (fresh), and grains
Gustafson, Ng et al., 2019 (null) Fruits and vegetables (null) Fruits and vegetables
Jilcott Pitts et al., 2018 (null) Fruits and vegetables (fresh, canned, or frozen) (null) Fruits and vegetables (fresh, canned, or frozen)
Holmes et al., 2012 (+) Fruits and vegetables (fresh), Healthy grains, nuts and seeds, (−/null) Dairy and vegetables
Gittelsohn et al., 2010 (+) Healthy grains, dairy, fruits and vegetables, and beverages(+/null) Healthy grains, dairy, fruits and vegetables, and beverages
Mackenzie et al., 2019 (+) Fruits and vegetables (fresh or frozen)
Schultz & Litchfield, 2016 (+) Fruit and vegetables, healthy grains, and protein
Steeves et al., 2015 (+) Healthy grains, dairy, snacks, and fruits and vegetables
Ortega et al., 2016 (null) Fruits and vegetables(null) Fruits and vegetables
Albert et al., 2017 (null) Fruits and vegetables (fresh, canned, and frozen)(null) Fruits and vegetables (fresh, canned, and frozen)
Woodward-Lopez et al., 2018 (+/−/null) Fruits and vegetables(+) Fruits and vegetables
Boys et al., 2021 (+/null) Dairy products (+) Whole grains (+) nuts, high protein items (+) Fruits (+) Vegetables (−) Fruit Juice
Lawman et al., 2015 (null) Healthy grains, protein, snacks, and beverages(null) Healthy grains, protein, snacks, and beverages
Gudzune et al., 2015 (+) Fruits and vegetables (fresh)
Paluta et al., 2019(+) Healthy proteins, grains, and fruits and vegetables (fresh, canned, or frozen) *
Sutton et al., 2019 (null) Fruits and vegetables
Beckelman et al., 2020 (+/null) Fruit (fresh) and healthy beverages
Rushakoff et al., 2017 (+) Healthy beverages, dairy, grains, nuts and seeds, and fruits ad vegetables (fresh, canned, or frozen)(+) Vegetables (fresh)
Gustafson, McGladrey et al., 2019 (+) Fruits (+) Vegetables
Paek et al., 2014 (+) Healthy beverages, dairy, fruits and vegetables (fresh), grains, snacks, nuts and seeds, legumes(+/null) Healthy grains (+) Nuts and seeds, legumes (null) Beverages, dairy, fruits and vegetables (fresh)
Liu et al., 2017 (+) Fruits and vegetables(+) Fruits and vegetables
Davis et al., 2016 (+) Healthy grains, frozen meals, and dairy
Ayala et al., 2013 (+/null) Fruits and vegetables (fresh, canned, and frozen)
Banerjee & Nayak, 2018(+, null) Fruits and vegetables (fresh), grains
Dannefer et al., 2012 (+) Healthy dairy, snacks, grains, and fruits and vegetables (fresh and canned)
Note: Study outcomes are categorized as related to either sales, purchasing, or intake/consumption. Sales outcomes denote the use of store-level data tracking customer sales data. Purchasing outcomes are results obtained from the perspective of the customer often through post-shopping surveys. Intake/consumption outcomes are related to studies reporting healthy item consumption rates. Outcomes are coded as either positive (+), negative (−), or null based on the intervention’s effect on healthy item sales, purchasing, or intake/consumption. Positive results represent an increase in healthy item sales, purchasing, or consumption, negative results represent a decrease in healthy item sales, purchasing, or consumption and null results represent no change in healthy item sales, purchasing, or consumption. * Data were missing for one evaluation period. During this time, decreases in items sold were observed.

References

  1. Dixon, B.N.; Ugwoaba, U.A.; Brockmann, A.N.; Ross, K.M. Associations between the Built Environment and Dietary Intake, Physical Activity, and Obesity: A Scoping Review of Reviews. Obes. Rev. 2021, 22, e13171. [Google Scholar] [CrossRef] [PubMed]
  2. Dietary Guidelines for Americans, 2020–2025; U.S. Department of Agriculture: Washington, DC, USA; U.S. Department of Health and Human Services: Washington, DC, USA, 2010. Available online: https://www.dietaryguidelines.gov/sites/default/files/2020-12/Dietary_Guidelines_for_Americans_2020-2025.pdf (accessed on 14 August 2022).
  3. Swinburn, B.A.; Kraak, V.I.; Allender, S.; Atkins, V.J.; Baker, P.I.; Bogard, J.R.; Brinsden, H.; Calvillo, A.; De Schutter, O.; Devarajan, R.; et al. The Global Syndemic of Obesity, Undernutrition, and Climate Change: The Lancet Commission report. Lancet 2019, 393, 791–846. [Google Scholar] [CrossRef] [PubMed]
  4. Afshin, A.; Sur, P.J.; Fay, K.A.; Cornaby, L.; Ferrara, G.; Salama, J.S.; Mullany, E.C.; Abate, K.H.; Abbafati, C.; Abebe, Z.; et al. Health effects of dietary risks in 195 countries, 1990–2017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet 2019, 393, 1958–1972. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Gómez-Donoso, C.; Sacks, G.; Vanderlee, L.; Hammond, D.; White, C.M.; Nieto, C.; Bes-Rastrollo, M.; Cameron, A.J. Public support for healthy supermarket initiatives focused on product placement: A multi-country cross-sectional analysis of the 2018 International Food Policy Study. Int. J. Behav. Nutr. Phys. Act. 2021, 18, 1–12. [Google Scholar] [CrossRef]
  6. Cummins, S.; Macintyre, S. Food Environments and Obesity—Neighbourhood or Nation? Int. J. Epidemiol. 2006, 35, 100–104. [Google Scholar] [CrossRef]
  7. Glanz, K.; Bader, M.D.M.; Iyer, S. Retail Grocery Store Marketing Strategies and Obesity. Am. J. Prev. Med. 2012, 42, 503–512. [Google Scholar] [CrossRef]
  8. Escaron, A.L.; Meinen, A.M.; Nitzke, S.A.; Martinez-Donate, A.P. Supermarket and Grocery Store–Based Interventions to Promote Healthful Food Choices and Eating Practices: A Systematic Review. Prev. Chronic Dis. 2013, 10, 120156. [Google Scholar] [CrossRef] [Green Version]
  9. Laska, M.N.; Borradaile, K.E.; Tester, J.; Foster, G.D.; Gittelsohn, J. Healthy Food Availability in Small Urban Food Stores: A Comparison of Four US Cities. Public Health Nutr. 2010, 13, 1031–1035. [Google Scholar] [CrossRef] [Green Version]
  10. Krukowski, R.A.; West, D.S.; Harvey-Berino, J.; Elaine Prewitt, T. Neighborhood Impact on Healthy Food Availability and Pricing in Food Stores. J. Community Health 2010, 35, 315–320. [Google Scholar] [CrossRef] [Green Version]
  11. Hartmann-Boyce, J.; Bianchi, F.; Piernas, C.; Riches, S.P.; Frie, K.; Nourse, R.; Jebb, S.A. Grocery Store Interventions to Change Food Purchasing Behaviors: A Systematic Review of Randomized Controlled Trials. Am. J. Clin. Nutr. 2018, 107, 1004–1016. [Google Scholar] [CrossRef] [PubMed]
  12. Thornton, L.E.; Cameron, A.J.; McNaughton, S.A.; Worsley, A.; Crawford, D.A. The availability of snack food displays that may trigger impulse purchases in Melbourne supermarkets. BMC Public Health. 2012, 12, 194. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Cameron, A.J.; Charlton, E.; Ngan, W.W.; Sacks, G. A Systematic Review of the Effectiveness of Supermarket-Based Interventions Involving Product, Promotion, or Place on the Healthiness of Consumer Purchases. Curr. Nutr. Rep. 2016, 5, 129–138. [Google Scholar] [CrossRef]
  14. Bolen, E.; Wolkomir, E. SNAP Boosts Retailers and Local Economies; Center on Budget and Policy Priorities: Washington, DC, USA, 2022; Available online: https://www.cbpp.org/research/food-assistance/snap-boosts-retailers-and-local-economies (accessed on 17 August 2022).
  15. Monsivais, P.; Mclain, J.; Drewnowski, A. The Rising Disparity in the Price of Healthful Foods: 2004–2008. Food Policy 2010, 35, 514–520. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Gupta, S.; Hawk, T.; Aggarwal, A.; Drewnowski, A. Characterizing Ultra-Processed Foods by Energy Density, Nutrient Density, and Cost. Front. Nutr. 2019, 6, 70. [Google Scholar] [CrossRef] [Green Version]
  17. Darmon, N.; Drewnowski, A. Contribution of Food Prices and Diet Cost to Socioeconomic Disparities in Diet Quality and Health: A Systematic Review and Analysis. Nutr. Rev. 2015, 73, 643–660. [Google Scholar] [CrossRef]
  18. Karpyn, A.; McCallops, K.; Wolgast, H.; Glanz, K. Improving Consumption and Purchases of Healthier Foods in Retail Environments: A Systematic Review. Int. J. Environ. Res. Public Health 2020, 17, 7524. [Google Scholar] [CrossRef]
  19. RefWorks. Available online: https://knowledge.exlibrisgroup.com/RefWorks/Product_Documentation/RefWorks_User_Guide (accessed on 17 August 2022).
  20. Hendrix, I.C. RefWorks. J. Med. Libr. Assoc. 2004, 92, 111–113. [Google Scholar]
  21. Berning, J.P.; Chouinard, H.H.; McCluskey, J.J. Do Positive Nutrition Shelf Labels Affect Consumer Behavior? Findings from a Field Experiment with Scanner Data. Am. J. Agric. Econ. 2011, 93, 364–369. [Google Scholar] [CrossRef]
  22. Finkelstein, E.A.; Li, W.; Melo, G.; Strombotne, K.; Zhen, C. Identifying the Effect of Shelf Nutrition Labels on Consumer Purchases: Results of a Natural Experiment and Consumer Survey. Am. J. Clin. Nutr. 2018, 107, 647–651. [Google Scholar] [CrossRef] [Green Version]
  23. Bachman, J.L.; Arigo, D. Reported Influences on Restaurant-Type Food Selection Decision Making in a Grocery Store Chain. J. Nutr. Educ. Behav. 2018, 50, 555–563. [Google Scholar] [CrossRef] [PubMed]
  24. Cawley, J.; Sweeney, M.J.; Sobal, J.; Just, D.R.; Kaiser, H.M.; Schulze, W.D.; Wethington, E.; Wansink, B. The Impact of a Supermarket Nutrition Rating System on Purchases of Nutritious and Less Nutritious Foods. Public Health Nutr. 2015, 18, 8–14. [Google Scholar] [CrossRef]
  25. Gustafson, C.R.; Zeballos, E. The effect of ingredient-specific calorie information on calories ordered. Prev. Med. Rep. 2018, 12, 186–190. [Google Scholar] [CrossRef]
  26. Kiesel, K.; Villas-Boas, S.B. Can information costs affect consumer choice? nutritional labels in a supermarket experiment. Int. J. Ind. Organ. 2013, 31, 153–163. [Google Scholar] [CrossRef] [Green Version]
  27. Melo, G.; Zhen, C.; Colson, G. Does point-of-sale nutrition information improve the nutritional quality of food choices? Econ. Hum. Biol. 2019, 35, 133–143. [Google Scholar] [CrossRef] [PubMed]
  28. Nikolova, H.D.; Inman, J.J. Healthy Choice: The effect of simplified point-of-sale nutritional information on consumer food choice behavior. J. Mark. Res. 2015, 52, 817–835. [Google Scholar] [CrossRef]
  29. Rahkovsky, I.; Lin, B.-H.; Lin, C.-T.J.; Lee, J.-Y. Effects of the guiding stars program on purchases of ready-to-eat cereals with different nutritional attributes. Food Policy 2013, 43, 100–107. [Google Scholar] [CrossRef]
  30. Sutherland, L.A.; Kaley, L.A.; Fischer, L. Guiding Stars: The effect of a nutrition navigation program on consumer purchases at the supermarket. Am. J. Clin. Nutr. 2010, 91, 1090S–1094S. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Zhen, C.; Zheng, X. The Impact of NuVal Shelf Nutrition labels on food purchase. Appl. Econ. Perspect. Policy 2020, 42, 870–887. [Google Scholar] [CrossRef]
  32. Boys, K.A.; Haynes-Maslow, L.; McGuirt, J.T.; Ammerman, A.S.; Van Fleet, E.E.; Johnson, N.S.; Kelley, C.J.; Donadio, V.E.; Fleischhacker, S.E.; Truesdale, K.P.; et al. Perceived barriers and facilitators to participating in the north carolina healthy food small retailer program: A mixed-methods examination considering investment effectiveness. Public Health Nutr. 2021, 24, 6555–6565. [Google Scholar] [CrossRef]
  33. Haynes-Maslow, L.; Jilcott Pitts, S.B.; Boys, K.A.; McGuirt, J.T.; Fleischhacker, S.; Ammerman, A.S.; Johnson, N.; Kelley, C.; Donadio, V.E.; Bell, R.A.; et al. Qualitative perspectives of the north carolina healthy food small retailer program among customers in participating stores located in food deserts. BMC Public Health 2021, 21, 1459. [Google Scholar] [CrossRef]
  34. Jetter, K.M.; Cassady, D.L. Increasing fresh fruit and vegetable availability in a low-income neighborhood convenience store: A pilot study. Health Promot. Pract. 2010, 11, 694–702. [Google Scholar] [CrossRef]
  35. Jilcott Pitts, S.; Wu, Q.; Truesdale, K.; Haynes-Maslow, L.; McGuirt, J.; Ammerman, A.; Bell, R.; Laska, M. One-year follow-up examination of the impact of the north carolina healthy food small retailer program on healthy food availability, purchases, and consumption. Int. J. Environ. Res. Public Health 2018, 15, 2681. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Jilcott Pitts, S.B.; Wu, Q.; Truesdale, K.P.; Rafferty, A.P.; Haynes-Maslow, L.; Boys, K.A.; McGuirt, J.T.; Fleischhacker, S.; Johnson, N.; Kaur, A.P.; et al. A four-year observational study to examine the dietary impact of the north carolina healthy food small retailer program, 2017–2020. Int. J. Behav. Nutr. Phys. Act. 2021, 18, 44. [Google Scholar] [CrossRef]
  37. Laska, M.N.; Caspi, C.E.; Lenk, K.; Moe, S.G.; Pelletier, J.E.; Harnack, L.J.; Erickson, D.J. Evaluation of the first u.s. staple foods ordinance: Impact on nutritional quality of food store offerings, customer purchases and home food environments. Int. J. Behav. Nutr. Phys. Act. 2019, 16, 83. [Google Scholar] [CrossRef] [Green Version]
  38. Finnell, K.J.; John, R.; Thompson, D.M. 1% low-fat milk has perks!: An evaluation of a social marketing intervention. Prev. Med. Rep. 2017, 5, 144–149. [Google Scholar] [CrossRef] [PubMed]
  39. Woodward-Lopez, G.; Kao, J.; Kuo, E.S.; Rauzon, S.; Taylor, A.C.; Goette, C.; Collins, C.; Gonzalez, E.P.; Ronshausen, D.R.; Boyle, K.; et al. Changes in consumer purchases in stores participating in an obesity prevention initiative. Am. J. Prev. Med. 2018, 54, S160–S169. [Google Scholar] [CrossRef]
  40. Palacios, C.; Torres, M.; López, D.; Trak-Fellermeier, M.; Coccia, C.; Pérez, C. Effectiveness of the nutritional app “mynutricart” on food choices related to purchase and dietary behavior: A pilot randomized controlled trial. Nutrients 2018, 10, 1967. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Gustafson, C.R.; Kent, R.; Prate, M.R. Retail-based healthy food point-of-decision prompts (pdps) increase healthy food choices in a rural, low-income, minority community. PLoS ONE 2018, 13, e0207792. [Google Scholar] [CrossRef]
  42. Moran, A.J.; Khandpur, N.; Polacsek, M.; Thorndike, A.N.; Franckle, R.L.; Boulos, R.; Sampson, S.; Greene, J.C.; Blue, D.G.; Rimm, E.B. Make it fresh, for less! a supermarket meal bundling and electronic reminder intervention to promote healthy purchases among families with children. J. Nutr. Educ. Behav. 2019, 51, 400–408. [Google Scholar] [CrossRef]
  43. Biswas, D.; Szocs, C. The smell of healthy choices: Cross-modal sensory compensation effects of ambient scent on food purchases. J. Mark. Res. 2019, 56, 123–141. [Google Scholar] [CrossRef]
  44. Biswas, D.; Lund, K.; Szocs, C. Sounds like a healthy retail atmospheric strategy: Effects of ambient music and background noise on food sales. J. Acad. Mark. Sci. 2019, 47, 37–55. [Google Scholar] [CrossRef]
  45. Chapman, L.E.; Sadeghzadeh, C.; Koutlas, M.; Zimmer, C.; De Marco, M. Evaluation of three behavioural economics ‘nudges’ on grocery and convenience store sales of promoted nutritious foods. Public Health Nutr. 2019, 22, 3250–3260. [Google Scholar] [CrossRef] [PubMed]
  46. Payne, C.R.; Niculescu, M.; Just, D.R.; Kelly, M.P. Shopper marketing nutrition interventions: Social norms on grocery carts increase produce spending without increasing shopper budgets. Prev. Med. Rep. 2015, 2, 287–291. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Tal, A.; Wansink, B. An apple a day brings more apples your way: Healthy samples prime healthier choices: Healthy samples prime healthier choices. Psychol. Mark. 2015, 32, 575–584. [Google Scholar] [CrossRef]
  48. Weiss, S.; Davis, E.; Wojtanowski, A.C.; Foster, G.D.; Glanz, K.; Karpyn, A. Consumer taste tests and milk preference in low-income, urban supermarkets. Public Health Nutr. 2015, 18, 1419–1422. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  49. Bangia, D.; Shaffner, D.W.; Palmer-Keenan, D.M. A point-of-purchase intervention using grocery store tour podcasts about omega-3s increases long-term purchases of omega-3–rich food items. J. Nutr. Educ. Behav. 2017, 49, 475–480.e1. [Google Scholar] [CrossRef] [PubMed]
  50. Bangia, D.; Palmer-Keenan, D.M. Grocery store podcast about omega-3 fatty acids influences shopping behaviors: A pilot study. J. Nutr. Educ. Behav. 2014, 46, 616–620. [Google Scholar] [CrossRef]
  51. Liu, P.J.; Dallas, S.K.; Harding, M.; Fitzsimons, G.J. The Sales Impact of Featuring Healthy Foods, Indulgent Foods, or Both: Findings from a Large-Scale Retail Field Study. J. Assoc. Consum. Res. 2018, 3, 346–363. [Google Scholar] [CrossRef]
  52. Payne, C.; Niculescu, M. Can Healthy Checkout End-Caps Improve Targeted Fruit and Vegetable Purchases? Evidence from Grocery and SNAP Participant Purchases. Food Policy 2018, 79, 318–323. [Google Scholar] [CrossRef]
  53. Adjoian, T.; Dannefer, R.; Willingham, C.; Brathwaite, C.; Franklin, S. Healthy Checkout Lines: A Study in Urban Supermarkets. J. Nutr. Educ. Behav. 2017, 49, 615–622.e1. [Google Scholar] [CrossRef] [PubMed]
  54. Sutton, K.; Caldwell, J.; Yoshida, S.; Thompson, J.; Kuo, T. Healthy Food Marketing and Purchases of Fruits and Vegetables in Large Grocery Stores. Prev. Med. Rep. 2019, 14, 100861. [Google Scholar] [CrossRef]
  55. Milliron, B.-J.; Woolf, K.; Appelhans, B.M. A Point-of-Purchase Intervention Featuring In-Person Supermarket Education Affects Healthful Food Purchases. J. Nutr. Educ. Behav. 2012, 44, 225–232. [Google Scholar] [CrossRef] [Green Version]
  56. Surkan, P.J.; Tabrizi, M.J.; Lee, R.M.; Palmer, A.M.; Frick, K.D. Eat Right–Live Well! Supermarket Intervention Impact on Sales of Healthy Foods in a Low-Income Neighborhood. J. Nutr. Educ. Behav. 2016, 48, 112–121.e1. [Google Scholar] [CrossRef] [PubMed]
  57. Gittelsohn, J.; Song, H.-J.; Suratkar, S.; Kumar, M.B.; Henry, E.G.; Sharma, S.; Mattingly, M.; Anliker, J.A. An Urban Food Store Intervention Positively Affects Food-Related Psychosocial Variables and Food Behaviors. Health Educ. Behav. 2010, 37, 390–402. [Google Scholar] [CrossRef] [PubMed]
  58. MacKenzie, O.W.; George, C.V.; Pérez-Escamilla, R.; Lasky-Fink, J.; Piltch, E.M.; Sandman, S.M.; Clark, C.; Avalos, Q.J.; Carroll, D.S.; Wilmot, T.M.; et al. Healthy Stores Initiative Associated with Produce Purchasing on Navajo Nation. Curr. Dev. Nutr. 2019, 3, nzz125. [Google Scholar] [CrossRef]
  59. Martínez-Donate, A.P.; Riggall, A.J.; Meinen, A.M.; Malecki, K.; Escaron, A.L.; Hall, B.; Menzies, A.; Garske, G.; Nieto, F.J.; Nitzke, S. Evaluation of a Pilot Healthy Eating Intervention in Restaurants and Food Stores of a Rural Community: A Randomized Community Trial. BMC Public Health 2015, 15, 136. [Google Scholar] [CrossRef] [Green Version]
  60. Ayala, G.X.; Pickrel, J.L.; Baquero, B.; Sanchez-Flack, J.; Lin, S.-F.; Belch, G.; Rock, C.L.; Linnan, L.; Gittelsohn, J.; Ji, M.; et al. The El Valor de Nuestra Salud Clustered Randomized Controlled Trial Store-Based Intervention to Promote Fruit and Vegetable Purchasing and Consumption. Int. J. Behav. Nutr. Phys. Act. 2022, 19, 19. [Google Scholar] [CrossRef]
  61. Williams, M.B.; Wang, W.; Taniguchi, T.; Salvatore, A.L.; Groover, W.K.; Wetherill, M.; Love, C.; Cannady, T.; Grammar, M.; Standridge, J.; et al. Impact of a Healthy Retail Intervention on Fruits and Vegetables and Total Sales in Tribally Owned Convenience Stores: Findings From the THRIVE Study. Health Promot. Pract. 2021, 22, 796–805. [Google Scholar] [CrossRef] [PubMed]
  62. Gittelsohn, J.; Trude, A.; Poirier, L.; Ross, A.; Ruggiero, C.; Schwendler, T.; Anderson Steeves, E. The Impact of a Multi-Level Multi-Component Childhood Obesity Prevention Intervention on Healthy Food Availability, Sales, and Purchasing in a Low-Income Urban Area. Int. J. Environ. Res. Public Health 2017, 14, 1371. [Google Scholar] [CrossRef] [Green Version]
  63. Lent, M.R.; Vander Veur, S.S.; McCoy, T.A.; Wojtanowski, A.C.; Sandoval, B.; Sherman, S.; Komaroff, E.; Foster, G.D. A Randomized, Controlled Study of a Healthy Corner Store Initiative on the Purchases of Urban, Low-Income Youth: Healthy Corner Stores RCT. Obesity 2014, 22, 2494–2500. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  64. Trude, A.C.B.; Surkan, P.J.; Cheskin, L.J.; Gittelsohn, J. A Multilevel, Multicomponent Childhood Obesity Prevention Group-Randomized Controlled Trial Improves Healthier Food Purchasing and Reduces Sweet-Snack Consumption among Low-Income African-American Youth. Nutr. J. 2018, 17, 96. [Google Scholar] [CrossRef]
  65. Schultz, J.; Litchfield, R. Evaluation of Traditional and Technology-Based Grocery Store Nutrition Education. Am. J. Health Educ. 2016, 47, 355–364. [Google Scholar] [CrossRef]
  66. Steeves, E.A.; Penniston, E.; Rowan, M.; Steeves, J.; Gittelsohn, J. A Rural Small Food Store Pilot Intervention Creates Trends Toward Improved Healthy Food Availability. J. Hunger. Environ. Nutr. 2015, 10, 259–270. [Google Scholar] [CrossRef]
  67. Kannan, S.; Ranjit, N.; Ganguri, H.B.; Lasichak, A.; Sparks, A.; Scherer, H.; Schulz, A. Sisters Together: Lessons Learned from Implementing a Grocery Store Campaign in Michigan Communities at Risk for Neural Tube Defects. J. Health Care Poor Underserved 2020, 31, 301–324. [Google Scholar] [CrossRef] [PubMed]
  68. Paluta, L.; Kaiser, M.L.; Huber-Krum, S.; Wheeler, J. Evaluating the Impact of a Healthy Corner Store Initiative on Food Access Domains. Eval. Program Plan. 2019, 73, 24–32. [Google Scholar] [CrossRef] [PubMed]
  69. Lawman, H.G.; Vander Veur, S.; Mallya, G.; McCoy, T.A.; Wojtanowski, A.; Colby, L.; Sanders, T.A.; Lent, M.R.; Sandoval, B.A.; Sherman, S.; et al. Changes in Quantity, Spending, and Nutritional Characteristics of Adult, Adolescent and Child Urban Corner Store Purchases after an Environmental Intervention. Prev. Med. 2015, 74, 81–85. [Google Scholar] [CrossRef] [PubMed]
  70. Beckelman, T.; Sinclair-White, B.M.; McGurk, M.D.; Donohoe-Mather, C.; Vu, U.; Ching, L.K.; Pirkle, C.M.; Sentell, T.L.; Keliikoa, L.B. Encouraging Adults to Choose Healthy Now: A Hawai‘i Convenience Store Intervention. J. Nutr. Educ. Behav. 2020, 52, 330–334. [Google Scholar] [CrossRef] [PubMed]
  71. Rushakoff, J.A.; Zoughbie, D.E.; Bui, N.; DeVito, K.; Makarechi, L.; Kubo, H. Evaluation of Healthy2Go: A Country Store Transformation Project to Improve the Food Environment and Consumer Choices in Appalachian Kentucky. Prev. Med. Rep. 2017, 7, 187–192. [Google Scholar] [CrossRef] [PubMed]
  72. Wensel, C.; Trude, A.; Poirier, L.; Alghamdi, R.; Trujillo, A.; Anderson Steeves, E.; Paige, D.; Gittelsohn, J. B’more Healthy Corner Stores for Moms and Kids: Identifying Optimal Behavioral Economic Strategies to Increase WIC Redemptions in Small Urban Corner Stores. Int. J. Environ. Res. Public Health 2018, 16, 64. [Google Scholar] [CrossRef] [Green Version]
  73. Thorndike, A.N.; Bright, O.-J.M.; Dimond, M.A.; Fishman, R.; Levy, D.E. Choice Architecture to Promote Fruit and Vegetable Purchases by Families Participating in the Special Supplemental Program for Women, Infants, and Children (WIC): Randomized Corner Store Pilot Study. Public Health Nutr. 2017, 20, 1297–1305. [Google Scholar] [CrossRef] [Green Version]
  74. Bird Jernigan, V.B.; Salvatore, A.L.; Williams, M.; Wetherill, M.; Taniguchi, T.; Jacob, T.; Cannady, T.; Grammar, M.; Standridge, J.; Fox, J.; et al. A Healthy Retail Intervention in Native American Convenience Stores: The THRIVE Community-Based Participatory Research Study. Am. J. Public Health 2019, 109, 132–139. [Google Scholar] [CrossRef] [PubMed]
  75. Gustafson, A.; Ng, S.W.; Jilcott Pitts, S. The Association between the “Plate It Up Kentucky” Supermarket Intervention and Changes in Grocery Shopping Practices among Rural Residents. Transl. Behav. Med. 2019, 9, 865–874. [Google Scholar] [CrossRef] [PubMed]
  76. Ortega, A.N.; Albert, S.L.; Chan-Golston, A.M.; Langellier, B.A.; Glik, D.C.; Belin, T.R.; Garcia, R.E.; Brookmeyer, R.; Sharif, M.Z.; Prelip, M.L. Substantial Improvements Not Seen in Health Behaviors Following Corner Store Conversions in Two Latino Food Swamps. BMC Public Health 2016, 16, 389. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  77. Albert, S.L.; Langellier, B.A.; Sharif, M.Z.; Chan-Golston, A.M.; Prelip, M.L.; Elena Garcia, R.; Glik, D.C.; Belin, T.R.; Brookmeyer, R.; Ortega, A.N. A Corner Store Intervention to Improve Access to Fruits and Vegetables in Two Latino Communities. Public Health Nutr. 2017, 20, 2249–2259. [Google Scholar] [CrossRef] [Green Version]
  78. Gudzune, K.A.; Welsh, C.; Lane, E.; Chissell, Z.; Anderson Steeves, E.; Gittelsohn, J. Increasing Access to Fresh Produce by Pairing Urban Farms with Corner Stores: A Case Study in a Low-Income Urban Setting. Public Health Nutr. 2015, 18, 2770–2774. [Google Scholar] [CrossRef] [Green Version]
  79. Gustafson, A.; McGladrey, M.; Stephenson, T.; Kurzynske, J.; Mullins, J.; Peritore, N.; Cardarelli, K.; Vail, A. Community-Wide Efforts to Improve the Consumer Food Environment and Physical Activity Resources in Rural Kentucky. Prev. Chronic Dis. 2019, 16, 180322. [Google Scholar] [CrossRef] [Green Version]
  80. Paek, H.-J.; Oh, H.J.; Jung, Y.; Thompson, T.; Alaimo, K.; Risley, J.; Mayfield, K. Assessment of a Healthy Corner Store Program (FIT Store) in Low-Income, Urban, and Ethnically Diverse Neighborhoods in Michigan. Fam. Community Health 2014, 37, 86–99. [Google Scholar] [CrossRef]
  81. Liu, E.; Stephenson, T.; Houlihan, J.; Gustafson, A. Marketing Strategies to Encourage Rural Residents of High-Obesity Counties to Buy Fruits and Vegetables in Grocery Stores. Prev. Chronic Dis. 2017, 14, 170109. [Google Scholar] [CrossRef] [Green Version]
  82. Foster, G.D.; Karpyn, A.; Wojtanowski, A.C.; Davis, E.; Weiss, S.; Brensinger, C.; Tierney, A.; Guo, W.; Brown, J.; Spross, C.; et al. Placement and Promotion Strategies to Increase Sales of Healthier Products in Supermarkets in Low-Income, Ethnically Diverse Neighborhoods: A Randomized Controlled Trial. Am. J. Clin. Nutr. 2014, 99, 1359–1368. [Google Scholar] [CrossRef] [Green Version]
  83. Shin, A.; Surkan, P.J.; Coutinho, A.J.; Suratkar, S.R.; Campbell, R.K.; Rowan, M.; Sharma, S.; Dennisuk, L.A.; Karlsen, M.; Gass, A.; et al. Impact of Baltimore Healthy Eating Zones: An Environmental Intervention to Improve Diet Among African American Youth. Health Educ. Behav. 2015, 42, 97S–105S. [Google Scholar] [CrossRef] [PubMed]
  84. Holmes, A.S.; Estabrooks, P.A.; Davis, G.C.; Serrano, E.L. Effect of a Grocery Store Intervention on Sales of Nutritious Foods to Youth and Their Families. J. Acad. Nutr. Diet. 2012, 112, 897–901. [Google Scholar] [CrossRef]
  85. Davis, E.L.; Wojtanowski, A.C.; Weiss, S.; Foster, G.D.; Karpyn, A.; Glanz, K. Employee and Customer Reactions to a Healthy In-Store Marketing Intervention in Supermarkets. J. Food Res. 2016, 5, 107. [Google Scholar] [CrossRef] [Green Version]
  86. Trude, A.C.; Surkan, P.J.; Anderson Steeves, E.; Pollack Porter, K.; Gittelsohn, J. The Impact of a Multilevel Childhood Obesity Prevention Intervention on Healthful Food Acquisition, Preparation, and Fruit and Vegetable Consumption on African-American Adult Caregivers. Public Health Nutr. 2019, 22, 1300–1315. [Google Scholar] [CrossRef] [PubMed]
  87. Dannefer, R.; Williams, D.A.; Baronberg, S.; Silver, L. Healthy Bodegas: Increasing and Promoting Healthy Foods at Corner Stores in New York City. Am. J. Public Health 2012, 102, e27–e31. [Google Scholar] [CrossRef]
  88. Gittelsohn, J.; Kim, E.M.; He, S.; Pardilla, M. A Food Store–Based Environmental Intervention Is Associated with Reduced BMI and Improved Psychosocial Factors and Food-Related Behaviors on the Navajo Nation. J. Nutr. 2013, 143, 1494–1500. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  89. Banerjee, T.; Nayak, A. Believe It or Not: Health Education Works. Obes. Res. Clin. Pract. 2018, 12, 116–124. [Google Scholar] [CrossRef] [PubMed]
  90. Ayala, G.X.; Baquero, B.; Laraia, B.A.; Ji, M.; Linnan, L. Efficacy of a Store-Based Environmental Change Intervention Compared with a Delayed Treatment Control Condition on Store Customers’ Intake of Fruits and Vegetables. Public Health Nutr. 2013, 16, 1953–1960. [Google Scholar] [CrossRef] [PubMed]
  91. Gupta, A.; Alston, L.; Needham, C.; Robinson, E.; Marshall, J.; Boelsen-Robinson, T.; Blake, M.R.; Huggins, C.E.; Peeters, A. Factors Influencing Implementation, Sustainability and Scalability of Healthy Food Retail Interventions: A Systematic Review of Reviews. Nutrients 2022, 14, 294. [Google Scholar] [CrossRef]
Figure 1. Article Inclusion Process.
Figure 1. Article Inclusion Process.
Ijerph 19 16107 g001
Table 1. Description of Intervention Types Included in Review.
Table 1. Description of Intervention Types Included in Review.
Intervention TypeIntervention Description
Multi-component interventionsMulti-component interventions signify a retailer’s simultaneous usage of at least two of the intervention types listed below.
Nutrition ScoringNutrition scoring interventions involved the development of a scale to represent the healthfulness of certain food and beverage items throughout the food retail environment.
Increased stockingIncreased stocking interventions specifically stocked a higher quantity of healthy food items.
Nutritional messagingNutritional messaging interventions utilized signage, flyers, or other promotional materials specifically noting the healthful benefits of certain.
Non-nutritional messagingNon-nutritional messaging interventions tracked the effectiveness of non-traditional marketing strategies (e.g., scarcity labeling and floor labeling to guide shoppers to healthier store sections).
Food tasting and demonstrationsFood tasting and demonstrations represent interventions in which participants were given healthy food to sample and/or were shown how to prepare a recipe with healthier ingredients sold in-store.
Nutrition educationNutrition education interventions involved “any set of learning experiences designed to facilitate the voluntary adoption of eating and other nutrition-related behaviors conducive to health and well-being” (Washington State Department of Social and Health Services, 2022).
Endcaps and secondary placementEndcaps and secondary placement interventions promoted healthier items in a display placed at the end of an aisle or in other locations in addition to their primary placement throughout the store (e.g., in a newly installed refrigeration unit).
Point-of-sale interventionsPoint-of-sale interventions promoted healthy food items in retail checkout lines or counters (healthy checkout interventions). Point-of-sale interventions differ from point-of-purchase (POP) interventions, which often signify all in-store interventions and are not limited to the checkout vicinity.
Placement on shelfPlacement on shelf interventions promoted healthy food items by manipulating the location of healthy food items on store shelves, often moving healthier items to eye level or placing them on more prominent shelves within the store.
Table 2. Study Design Features of Included Articles by Intervention Type.
Table 2. Study Design Features of Included Articles by Intervention Type.
Study Design FeatureTotal (n = 73)Multi-Component Interventions (n = 40)Nutrition Scoring (n = 11)Increased Stocking (n = 6)Nutritional Messaging (n = 5)Non-Nutritional Messaging (n = 4)Food Tasting and Demos (n = 2)Nutrition Education (n = 2)Endcaps and Secondary Placement (n = 2)Point-of-Sale Interventions (n = 1)
Study Design
Experimental23 (32%)16 (40%)1 (9%)--2 (40%)2 (50%)1 (50%)--1 (50%)--
Quasi-experimental 25 (34%)11 (28%)5 (45%)3 (50%)2 (40%)2 (50%)----1 (50%)1 (100%)
Pre-experimental25 (34%) 13 (33%)5 (45%)3 (50%)1 (20%)--1 (50%)2 (100%)----
Objective data *43 (59%)17 (43%)10 (91%)4 (66%)4 (80%)4 (100%)1 (50%)--2 (100%)1 (100%)
Subjective data **30 (41%)23 (58%)1 (9%)2 (33%)1 (20%)--1 (50%)2 (100%)----
Population
SNAP enrollees24 (33%)14 (35%)--4 (66%)1 (20%)2 (50%)1 (50%)--1 (50%)1 (100%)
Low-income community20 (27%) 15 (38%) 2 (18%)1 (16%)1 (20%1 (25%)--------
Bodega or corner store7 (10%) 5 (13%)------------1 (50%) --
Major supermarket15 (21%)3 (8%)8 (73%)--1 (20%)1 (25%)--2 (100%)----
* Objective data refers to methodological tracking of sales and/or purchases of healthy items using store sales receipts or other store records. ** Subjective data refers to study participants’ self-reported sales, purchasing, and/or intake/consumption of healthy items.
Table 3. Intervention Types in Multi-Component Interventions.
Table 3. Intervention Types in Multi-Component Interventions.
RigorStudy NameIncreased StockingEndcaps & Secondary PlacementNutrition EducationFood Tasting and DemosNutritional MessagingNon-Nutritional MessagingPlacement on ShelfNutrition ScoringPoint-of-Sale
EXP + OBJMilliron, 2012 X X
Gittelsohn et al., 2017X XXX *
Williams et al., 2021XX X
Thorndike et al., 2017X XX
Wensel et al., 2019 XX X
Foster et al., 2014 X XX X
Banerjee & Nayak, 2018 XX
Total (EXP + OBJ) 334332210
EXP + SELFAyala et al., 2022 XXX X
Martinez-Donate et al., 2015X XX X
Lent et al., 2014X X XX
Trude et al., 2018X XXX
Bird Jernigan et al., 2019XX X
Shin et al., 2015 XX X
Trude et al., 2019X X
Ayala et al., 2013 X XX
Gittelsohn et al., 2013 XX
Total (EXP + SELF) 536542102
QUAS + OBJSurkan et al., 2016X XX X
Kannan et al., 2020 X X X
Gustafson, Ng et al., 2019 X X
Jilcott Pitts et al., 2018XX X
Holmes et al., 2012 X X X
Total (QUAS + OBJ) 232211310
QUAS + SELFGittelsohn et al., 2010 X X X
Mackenzie et al., 2019 X XX
Schultz & Litchfield, 2016 XX
Steeves et al., 2015X X X
Ortega et al., 2016XX X
Albert et al., 2017XX X
Total (QUAS + SELF) 332123021
PRE + OBJWoodward-Lopez et al., 2018 X X XX
Boys et al., 2021XX
Lawman et al., 2015X X X X
Gudzune et al., 2015X X
Paluta et al., 2019X X
Total (PRE + OBJ) 421122200
PRE + SELFSutton et al., 2019 XX X
Beckelman et al., 2020 XX X
Rushakoff et al., 2017 X XX
Gustafson, McGladrey et al., 2019 X XX
Paek et al., 2014X X X
Liu et al., 2017 X XX
Davis et al., 2016 X X X
Dannefer et al., 2012XX
Total (PRE + SELF) 241533410
Grand Total 1918161715131253
Note: The rigor column groups articles according to the study design and outcomes measured. Study design was coded as experimental, quasi-experimental, or pre-experimental. Outcome measures were coded as either using objective purchasing data or subjective purchasing data. An X indicates the intervention type utilized in a study. * Limited information about messaging type; category assumed.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Wolgast, H.; Halverson, M.M.; Kennedy, N.; Gallard, I.; Karpyn, A. Encouraging Healthier Food and Beverage Purchasing and Consumption: A Review of Interventions within Grocery Retail Settings. Int. J. Environ. Res. Public Health 2022, 19, 16107. https://doi.org/10.3390/ijerph192316107

AMA Style

Wolgast H, Halverson MM, Kennedy N, Gallard I, Karpyn A. Encouraging Healthier Food and Beverage Purchasing and Consumption: A Review of Interventions within Grocery Retail Settings. International Journal of Environmental Research and Public Health. 2022; 19(23):16107. https://doi.org/10.3390/ijerph192316107

Chicago/Turabian Style

Wolgast, Henry, McKenna M. Halverson, Nicole Kennedy, Isabel Gallard, and Allison Karpyn. 2022. "Encouraging Healthier Food and Beverage Purchasing and Consumption: A Review of Interventions within Grocery Retail Settings" International Journal of Environmental Research and Public Health 19, no. 23: 16107. https://doi.org/10.3390/ijerph192316107

APA Style

Wolgast, H., Halverson, M. M., Kennedy, N., Gallard, I., & Karpyn, A. (2022). Encouraging Healthier Food and Beverage Purchasing and Consumption: A Review of Interventions within Grocery Retail Settings. International Journal of Environmental Research and Public Health, 19(23), 16107. https://doi.org/10.3390/ijerph192316107

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop