Next Article in Journal
Burden of Mortality and Disease Attributable to Multiple Air Pollutants in Warsaw, Poland
Next Article in Special Issue
Factors Associated with Dental Caries in Primary Dentition in a Non-Fluoridated Rural Community of New South Wales, Australia
Previous Article in Journal
Climatic Variables and Malaria Morbidity in Mutale Local Municipality, South Africa: A 19-Year Data Analysis
Previous Article in Special Issue
Factors Influencing Early Feeding of Foods and Drinks Containing Free Sugars—A Birth Cohort Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 14
Issue 11
10.3390/ijerph14111361
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Development and Relative Validity of a Food Frequency Questionnaire to Assess Intakes of Total and Free Sugars in Australian Toddlers

by
Gemma Devenish
1,
Aqif Mukhtar
1,
Andrea Begley
1,
Loc Do
2 and
Jane Scott
1,*
1
School of Public Health, Curtin University, Perth 6102, Australia
2
Australian Research Centre for Population Oral Health, The University of Adelaide, Adelaide 5000, Australia
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2017, 14(11), 1361; https://doi.org/10.3390/ijerph14111361
Submission received: 22 September 2017 / Revised: 3 November 2017 / Accepted: 5 November 2017 / Published: 8 November 2017
(This article belongs to the Special Issue Epidemiology and Determinants of Dental Caries in Children)
Browse Figures
Versions Notes

Abstract

:
Background: Dental research into early childhood caries is hindered by a lack of suitable dietary assessment tools that have been developed and validated for the population and outcomes of interest. The aim of this study was to develop and investigate the relative validity and reproducibility of the Study of Mothers’ and Infants’ Life Events Food Frequency Questionnaire (SMILE-FFQ), to assess the total and free sugars intakes of Australian toddlers. Methods: The SMILE-FFQ was designed to capture the leading dietary contributors to dental caries risk in toddlers aged 18–30 months via a proxy report. Ninety-five parents of Australian toddlers completed the questionnaire online before and after providing three 24-h recalls (24HR), collected on non-consecutive days using the multipass method. Total and free sugars were compared between the two SMILE-FFQ administrations and between each SMILE-FFQ and the 24HR using multiple statistical tests and standardised validity criteria. Correlation (Pearson), mean difference (Wilcoxon rank test) and Bland Altman analyses were conducted to compare absolute values, with cross-classification (Chi-Square and Weighted Kappa) used to compare agreement across tertiles. Results: All reproducibility tests showed good agreement except weighted kappa, which showed acceptable agreement. Relative validity tests revealed a mix of good and acceptable agreement, with total sugars performing better at the individual level than free sugars. Compared to the 24HR, the SMILE-FFQ tended to underestimate absolute values at lower levels and overestimate them at higher levels. Conclusions: The combined findings of the various tests indicate that the SMILE-FFQ performs comparably to the 24HR for assessing both total and free sugars among individuals, is most effective for ranking participants rather than determining absolute intakes, and is therefore suitable for use in observational studies of Australian toddlers.

1. Introduction

Dental diseases are the most prevalent non-communicable disease globally [1] and the leading cause of preventable hospitalisations among children in Australia [2]. The prevalence of dental caries in Australian children began to decline significantly during the 1970s. However, in the mid-1990s, this downward trend reversed and caries prevalence has increased steadily ever since [3,4]. In addition, disparities in caries experience exist between children based on socio-economic status, remoteness and availability of fluoridated water [4].
The relationship between free sugar intakes and dental caries is well established, although the lower level of the effect is less clear [1,5,6,7]. In 2015, the World Health Organization (WHO) released the Guideline: Sugars intake for adults and children, with a focus on preventing dental disease and obesity [1]. Although the Guideline sets out a strong recommendation to reduce the intake of free sugars to less than 10% of total energy intake, the conditional recommendation to further reduce intake to less than 5% of total energy was limited by the low quality of evidence. The supporting evidence review highlights the lack of consistency and precision of dietary assessment methods in dental studies [5], leading to a call within the Guideline for new studies with an improved dietary assessment methodology [1].
Controlling for key dietary risk factors is a critical component of oral health research. However, dietary assessment is complex, and reviews of dental studies highlight the inconsistency and imprecision of approaches used for measuring dietary intakes and food behaviours [6,8,9]. Newly designed dietary assessment tools should be validated prior to use, and existing tools should be calibrated to each cohort [10,11,12]. An ongoing limitation of dental research is the lack of rigour in dietary assessment methodology, with many studies still not using validated dietary questionnaires to capture sugars intake, and fewer still undertaking internal calibration [5,8].
One reason for this lack of rigour is that an appropriate tool often does not exist [8,13]. Dietary assessment design and validation is a branch of nutrition research, and dental research teams do not always have the resources or nutrition expertise required to undertake validity studies. Childhood dietary intake is particularly challenging to capture, due to the participant age and varying ability to self-report intake, as well as the greater rate of fluctuation in dietary intake patterns throughout childhood compared to adults [10,14]. This is especially relevant throughout the first three years of life, as a child progresses from a newborn diet of breast milk or infant formula, via a range of textures, through to family foods. This transitioning diet combined with the unique dietary data requisites of dental research means that the tools used during the pre-school years need to be age-specific and developed purposely for dental research.
There are a number of methods used in dietary assessment, but the Food Frequency Questionnaire (FFQ) is considered the most appropriate data collection method for large, prospective studies [10,11,12]. Once developed, it is quick and inexpensive to administer and process, as it can be self-administered and rapidly analysed. Additionally, it has a lower subject burden than weighed food records, and captures usual dietary intakes over a longer period of time [10,11]. Although it is a commonly used method, the FFQ must be tailored to both the population and outcomes of interest if it is to produce useful data [10,11,12].
The aim of this study is twofold: firstly, to develop a FFQ to assess intakes of total and free sugars from major food and beverage sources, fluoride from non-water sources and other key foods relevant to dental caries in Australian toddlers aged 18–30 months via a proxy report. Secondly, to investigate the relative validity and reproducibility of this FFQ for total and free sugars against repeat 24-h recalls (24HR).

2. Materials and Methods

2.1. Study of Mothers’ and Infants’ Life Events Affecting Oral Health (SMILE)

The Study of Mothers’ and Infants’ Life Events Affecting Oral Health (SMILE) is a longitudinal birth cohort study that aims to identify and evaluate the relative importance and timing of critical factors that shape the oral health of young children and then evaluate those factors in terms of their interrelationship with socioeconomic influences [15]. A limited number of dietary assessment tools have been used to assess young children’s total dietary intake [16,17,18,19]; however, a tool for the dietary assessment of dental significance through the toddler years was not available. As such, a FFQ was designed to capture the leading dietary contributors to dental caries risk in toddlers aged 18–30 months via a proxy report for use in SMILE.

SMILE Food Frequency Questionnaire Development

Development of the SMILE-FFQ occurred in four stages, following the methods described by Willett [11] and Coulston et al. [20]. Firstly, a brief review of the literature identified the leading dietary contributors to dental caries risk, namely major food and beverage sources of total sugars, free sugars and fluoride, as well as protective foods including cheese and other milk products, chewing gum and xylitol [6]. The World Health Organization definition of free sugars was used, which “include(s) monosaccharides and disaccharides added to foods and beverages by the manufacturer, cook or consumer, and sugars naturally present in honey, syrups, fruit juices and fruit juice concentrates” [1].
Secondly, a food list was generated, incorporating all foods that would provide these dietary contributors. This list was then refined to exclude food items that: (a) contained only a small amount of the nutrient in question (for example, commercial pasta sauce containing approx. 5% total sugar); or (b) are not typically consumed by individuals within this age group (for example, chewing gum, alcohol, coffee, and energy drinks). During this process, foods were collapsed into line-items based on similarity of the food and of total and free sugar content and then grouped into blocks by food type. The final list (Supplementary Materials) consisted of 24 question blocks containing 89 line-items. Four additional questions were added to further split the coding of some food items (for example, “If your child eats yoghurt, do you choose reduced fat versions?”) and two questions were included to capture eating behaviours of dental significance (usual drinking vessels and typical number of eating occasions per day).
Thirdly, frequency and quantity response options appropriate to this age group were developed for each line-item. Seven frequency response options were used for all line-items, commencing with “never or rarely” before ranging from “1 time every 2 weeks” to “3 or more times per day”. Quantity response options were tailored to each question block, using a combination of household measures (teaspoon, tablespoon, cup) and typical portion sizes (piece, tub, pouch, etc.). Weight (g) or volume (ml) was also provided, except in blocks which only used teaspoon and tablespoon options, such as condiments.
Finally, a database was developed to analyse the SMILE-FFQ, linking scoring algorithms for all possible responses to nutrient values for total and free sugars, derived from representative foods in the AUSNUT 2011–13 food composition database [21].
The SMILE-FFQ was produced online using Qualtrics software, 2014 version (Qualtrics, Utah, USA, www.qualtrics.com). A copy of the questionnaire is provided in Supplementary Materials.

2.2. Food Frequency Questionnaire Validation

Curtin University Human Research Ethics Committee approved the study (Approval No. SPH-39-2014). The validation study was registered with the US National Cancer Institute Dietary Assessment Calibration/Validation (DACV) Register (Study ID 0293).

2.2.1. Participants and Recruitment

Inclusion criteria were parents with a child aged between 18 and 30 months at the time of recruitment, who lived in Australia and who had access to the internet, an email address, and telephone. No other inclusion or exclusion criteria were identified at this stage; however, one parent later withdrew from the study as they were unable to report their child’s dietary intake on any weekday.
A Facebook page was created to recruit participants, with a target sample size of 100, in line with recommendations by Willett [11] and comparable to similar studies [22,23,24,25]. The page included study information and a post with a link to an expression of interest, eligibility, and consent survey. The first page of the survey asked participants to report against the inclusion criteria. Those not meeting the eligibility requirements were diverted to an exit page, while eligible participants continued to the consent section. The survey was closed two weeks after the page launched, with a reach of 1753 people who saw the post, 229 who clicked the link, and 115 eligible participants who completed the survey (Figure 1). Fifteen people who completed the eligibility survey did not respond to any follow up communication or questionnaires, so the final number recruited was 100. Three participants withdrew after the first questionnaire, and the remaining 97 completed the study.

2.2.2. Data Collection

After recruitment, participants were entered into the study in three waves between August and October, to allow a single researcher to conduct all telephone diet recalls. Participants that were in the second or third wave were informed of the delay and the expected date of commencement. Data collection for all participants occurred between August and December, with one participant’s final FFQ completed in early January (the greatest duration from commencement at a total of 15 weeks).
Data collection, depicted in Figure 1, occurred over a minimum six-week period for each participant. Parents were sent a link to the online questionnaire (SMILE-FFQ1) and followed-up via weekly reminder emails which contained an unsubscribe option. One week after SMILE-FFQ1 was completed, participants entered the 24-h diet recall phase (24HR), consisting of three recalls conducted via telephone on non-consecutive days by the principal researcher (GD) who is a dietitian. A random number generator was used to allocate three days within a four week period for the participant recalls; two weekdays and one weekend day. On the day of a recall, participants were sent an SMS to inform them they would be receiving a call and were able to nominate a suitable time of day for the call. If the child had been unwell or the parent was unaware of intake on the previous day (e.g., child in childcare), the recall was not conducted, and a new day was randomly allocated. If the parent was able to report intake despite the child being in the care of others, the allocated recall day was retained. The five-step multipass method [26] was used to conduct the 24HR, using a modified forgotten foods list (the question about alcohol was substituted for “any rusks or teething foods?”) and a uniquely designed probing protocol with specific questions for toddler foods in the detail cycle phase. One week after the final 24HR, and at least six weeks after SMILE-FFQ1, participants were sent a new link to the online questionnaire (SMILE-FFQ2). Demographic data were collected on the final page of the SMILE-FFQ2, with the child’s date of birth also captured in the eligibility survey.
The 24HR were entered into FoodWorks version 8 (Xyris Software, 2012–2017, Brisbane, Australia) using the current Australian food composition database, AUSNUT 2011–13. As FoodWorks does not provide free sugars values, these were later imputed in Microsoft Access version 15 (Microsoft Corporation, 2013, Washington, DC, USA) using a dataset developed to accompany AUSNUT 2011–13. This dataset was developed by Food Standards Australia and New Zealand using the methodology by Louie et al. [27], which employed the WHO definition of free sugars [1]. Mean daily intakes for total and free sugars were obtained for each participant. Descriptive statistics were used to identify outliers for the weight of food and energy consumed and the data were checked for entry errors and resolved.
The survey responses for the SMILE-FFQs were exported from Qualtrics in Excel format and linked to the scoring database using Microsoft Access. Mean SMILE-FFQ1 and SMILE-FFQ2 daily intakes of total and free sugars were then generated for each participant.
Two participants were identified as reporting implausibly high intakes of both energy and total sugars and were excluded from the analysis. Implausible intake was defined as values greater than 3 SD above the mean for both nutrients in at least one questionnaire [28].

2.2.3. Data Analysis

Data were exported from Microsoft Access to Microsoft Excel and then analysed using SPSS version 24 (IBM SPSS Statistics for Windows, New York, NY, USA). The data were not normally distributed, so natural log transformation was performed and used with all but the Bland Altman analyses. The data were not energy adjusted, as the SMILE-FFQ was not designed to assess total diet and therefore total energy intakes could not be obtained.
As recommended by Lombard et al. [29], multiple statistical tests were employed to assess different facets of validity at both the group and individual level. Correlation (Pearson) and Cross-classification (Chi-Square and Weighted Kappa) assessed validity at the individual level. Comparison of means (Wilcoxon rank test) compared agreement between the two tools at a group level. Bland-Altman analyses of the raw data, including calculation of the Limits of Agreement and Bland Altman Index, investigated individual differences from the group mean and the extent of bias at a group level. Linear regression analysis was undertaken to investigate whether the slope of mean bias was significantly different from zero, indicating proportional bias.
Outcome cut-offs were determined using criteria described in Lombard et al. [29]. For correlation coefficient (Pearson), good agreement was ≥0.50, acceptable from 0.20 to 0.49, and poor <0.20 [29,30]. For cross-classification (Chi Square), good agreement was when ≥50% of participants were similarly classified (same tertile) and ≤10% dissimilarly classified (opposite tertile); and poor agreement was when <50% were similarly classified and/or >10% dissimilarly classified [29,30]. For the Weighted Kappa, good agreement was ≥0.61, acceptable from 0.20 to 0.60, and poor <0.20 [29,30]. For comparison of means (Wilcoxon), good agreement was at p > 0.05 and poor was p ≤ 0.05. A Bland Altman Index score of ≤5% was considered good agreement [31,32].

3. Results

3.1. Participant Characteristics

Data from 95 participants were analysed. Respondents were predominantly university-educated mothers, with a mean age of 34.2 ± 4.4 years at the time of the study (Table 1).

3.2. Total and Free Sugars Intakes

Table 2 provides mean values of total and free sugars in grams per day as measured by the SMILE-FFQ1, SMILE-FFQ2, and 24HR, along with further descriptive statistics and measures of normality before and after log transformation. SMILE-FFQ1 recorded the highest mean intake and the widest range of responses for both total and free sugars.

3.3. Relative Validity of the SMILE-FFQ

Relative validity was assessed by comparing total and free sugars values obtained by 24HR to those determined by SMILE-FFQ1 and SMILE-FFQ2. Agreement at the individual level (Table 3) was either good (Chi-Square, Pearson for total sugars) or acceptable (Weighted Kappa, Pearson for free sugars). Cross-classifications demonstrated good agreement between the two methods in classifying participants into tertiles, with less than 10% of participants misclassified into an opposite tertile for all methods except free sugars SMILE-FFQ2, which was just over at 10.6%.
Comparison at the group level (nonparametric) found no significant difference in mean intake between each pair of methods, except for the difference in the mean intake of total sugars between SMILE-FFQ1 and 24HR (Table 4).
The Bland Altman Index showed either 5.3% or 7.4% of participants falling outside of the limits of agreement, with total sugars close to the acceptable cut-off of 5%. The limits of agreement appear to be wide when compared to the mean, SD, and range of reported intakes (Table 2), and were greatest for total sugars between SMILE-FFQ1 and 24HR.
Visual inspection of Bland-Altman plots suggested a positive slope and a tendency to increase in variability as the magnitude of the measure increases (Figure 2). Linear regression verified that the slope of bias was significantly different from zero for all four measures (p < 0.001), indicating that proportional bias was present. These results suggest that agreement between the SMILE-FFQ and 24HR was better amongst participants with moderate total and free sugars intakes, but the SMILE-FFQ tended to underestimate sugars for participants with lower reported 24HR intakes and overestimate sugars for participants with higher 24HR intakes.

3.4. Reproducibility of the SMILE-FFQ

Reproducibility was assessed by comparing total and free sugars values obtained by SMILE-FFQ1 and SMILE-FFQ2. Agreement at the individual level (Table 5) was either good (Chi-Square, Pearson) or acceptable (Weighted Kappa). Cross-classifications demonstrated good agreement between the two methods in classifying participants into tertiles, with less than 10% of participants misclassified into an opposite tertile for both total and free sugars.
Comparison at the group level found no significant difference between the two FFQs for mean free sugars intakes, indicating good agreement; however, for total sugars, there was a significant difference, indicating poor agreement (Table 6).
The Bland Altman Index showed either 5.3% or 6.3% of participants falling outside of the limits of agreement, with total sugars close to the acceptable cut-off of 5%. The limits of agreement appear to be wide when compared to the mean, SD, and range of reported intakes (Table 2).
Bland-Altman plots demonstrated good agreement between repeat administrations of the SMILE-FFQ, as the data were scattered with no proportional bias (Figure 3). Linear regression also indicated that the slope of bias was not significantly different from zero for both measures (p > 0.01), suggesting there was no trend in differences in performance between the FFQs at varying levels of intake.

4. Discussion

4.1. Validity

Food Frequency Questionnaires aim to provide estimates of long-term dietary intake, and are generally considered unsuited to estimating absolute daily intakes of nutrients or capturing short-term dietary intakes [10,11]. The findings of this validation study reflect this, as the weakest agreement was observed in comparisons of absolute intake.
Combining the findings of the various tests, it is evident that individual level validity is more consistently rated as good than group level validity, and that the tool is most effective for ranking participants but not necessarily for determining absolute intakes of total and free sugars. The tendency of the tool to underestimate intakes in participants with lower 24HR values and overestimate intakes in those with higher values is consistent with other FFQs [23,33], and has a lesser impact on the accuracy of ranking than of absolute intakes. These results indicate that the SMILE-FFQ performs comparably to the 24HR as a measure of total and free sugars of individuals and to a lesser extent the group, and is therefore acceptable for use in observational studies of Australian toddlers if absolute sugars intakes is not an explanatory or outcome measure.
At present, the number of available tools which assess dietary intakes during early childhood is very small, and fewer still capture total sugars. To our knowledge, this is the first FFQ designed to measure free sugars in this age group (18–30 months), and the first Australian FFQ that has been validated for total or free sugars in any pre-school aged children (0–5 years). As such, comparison data in this age group are scarce. The National Infant Feeding Survey focuses on milk-feeding methods in the first two years of life rather than total diet [18], and although the recent Australian Health Survey included participants as young as two, reporting of results for this age group is incomplete [34].
Bell et al. [35] recently developed a short food-group-based dietary risk assessment tool for use with Australian toddlers (aged 1–3 years). This 19-item screener generates a dietary risk score between 0 and 100, but is not designed to obtain nutrient values. Flood et al. [25] developed a 17-item screener in FFQ style to rank Australian pre-school aged children (2–5 years) based on key dietary habits. Four questions pertained to major contributors of free sugars (biscuits, confectionery, soft-drink, fruit juice); however, two of these performed poorly against the reference method. Burrows et al. [36] developed the 70-item Australian Recommended Food Score for Pre-schoolers (ARFS-P), but to date, this has only provided validation reporting against the FFQ from which it was developed, and scoring occurs at the food group rather than nutrient level, with a focus on core foods.
A FFQ has been developed for 12-month-olds in the United Kingdom [22] and modified for use in New Zealand with toddlers aged 12–24 months [24,37]; however, it was not designed to capture free sugars, and the New Zealand validation studies do not report findings for total sugars. Nevertheless, it is rarely appropriate to use a tool designed for another population without adaptation and further validation, as food supply, composition, and terminology differ between countries. A notable example is the eight-item screener developed to assess the consumption of sugar snacks in Ugandan school-aged children (mean 12.4 years of age) [38]. This tool lists tea and coffee as two of the eight food items included and as such is not transferable to Australian children. Although tea and coffee are significant contributors to sugar consumption of Ugandan children, the 2007 Australian National Children’s Nutrition and Physical Activity Survey reports a very low prevalence of consumption of tea and coffee in this age group, at 6.6% and 2.3%, respectively [39], and so the tool is not compatible with Australian research.

4.2. Challenges in Dietary Assessment of Pre-School Aged Children

Dietary assessment in early childhood presents some unique challenges. Young children are unable to self-report, so proxy reporting must be used to collect the data from a surrogate [10,14,40,41]. Parents are not always aware of their child’s complete dietary intake as feeding roles may be shared amongst other caregivers and child-care providers [14,41]. In addition, toddlers tend to have higher amounts of plate waste than older children and adults, which makes estimating actual consumption more cognitively complex [10,14].
In addition, toddlers experience a unique food landscape. Portion sizes are small and highly variable, and the frequency of meals and snacks tends to be greater than for older children and adults. Young children also have strong appetite cues [42], and experience irregular patterns of rapid physical and cognitive growth. As a result, food intake and preferences have greater variability than that of adults in terms of both day-to-day intake and overall eating pattern, and change rapidly across the pre-school years [10,14,41]. An FFQ for this age needs to offer a wide range of lower-size portion options that include very small values. These small amounts seem negligible to overall nutritional intake; however, if lower end options are not provided, reporters may select the lowest available value, which may be several times larger than what was actually consumed. Also, due to the smaller energy and nutrient requirements of pre-school aged children, the small, frequent “tastes” of food may add up to a significant proportion of overall intake. The development of FFQs for this age bracket has often occurred by testing the validity of a tool originally designed for adults or older children, the portion sizes of which may be inappropriate and contribute to the common finding of overestimation [23,33,43,44].
These challenges limit the accuracy of most dietary assessment methods, including the SMILE-FFQ. The design of this FFQ attempts to minimise error from these challenges by targeting a narrow age range, providing portion size options specifically suited to toddlers and including only those foods of relevance to total and free sugars intakes rather than total diet. To reduce the difficulties arising from the high variability of the toddler diet, participants were asked to report “usual intake”, rather than over a specified period of weeks. However, some participants may have found this more challenging as usual intake may be difficult to define.
The FFQ is well suited to comparing food group contribution to nutrient intake, as further interrogation of FFQ data can be conducted at the line item or block level. However, FFQs are unable to separate out individual sources at the single food level. For example, the SMILE-FFQ is able to identify major sources of sugar by food type (milk, yoghurt, biscuits, chocolate, breakfast cereal, table sugar, etc.), but is unable to describe precisely where, when, and in what combinations these foods are consumed.

4.3. Limitations of the Study

Participation in the 24HR may have increased parent awareness of their child’s food intake and influenced their responses to SMILE-FFQ2. This influence could partially explain the wide limits of agreement between repeat administrations of SMILE-FFQ, and why the total sugars results from SMILE-FFQ2 showed greater agreement with 24HR, a finding consistent with other studies of this type [45].
It is also likely that social desirability bias played a greater role in the 24HR, which were conducted by a dietitian via telephone interview, than the FFQs, which were self-administered online. This may explain some of the proportional bias; that participants who reported the highest sugars intakes via the online SMILE-FFQ provided more conservative estimates when reporting to the dietitian than those with lower SMILE-FFQ sugars intakes. Other studies support this theory, suggesting social desirability bias influences parents’ reporting of their children’s intakes, particularly in relation to their child’s weight or higher levels of foods that are perceived as unhealthy [46,47]. The recent focus on reducing the intake of free sugars by the scientific community, coupled with current food fads in popular media that demonise sugar intake from any and all sources, may have exacerbated the rates of underreporting of foods high in sugars via telephone interviews. This does not, however, account for the underestimation at lower intakes.
There is some evidence to indicate that seasonal variation may affect dietary assessment responses; however, these effects are generally considered minor, with the greatest variation between different types of fruit and vegetables [48,49]. This FFQ does not differentiate between types of fruit, but rather asks participants to estimate the overall frequency of consumption of any fruit. Additionally, the reference method (24HR) was administered in the middle of the data collection phase, which for most participants (90%), was completed within two months. This is likely to have ameliorated the effect of seasonal change on questionnaire responses.
A further limitation of this study design is the absence of a biomarker for validation. A urinary sugars biomarker based on a 24-h urine collection could be a suitable reference measure in future studies; however, research to date is limited by small sample sizes [50] and a 24-h urine collection in 18–30 month olds may be particularly challenging. In order to provide an estimate of usual intake, multiple collections would be required, resulting in a high cost and subject burden. In the absence of this, there is no reference instrument that can provide an accurate measure of a person’s usual dietary intake. Repeat 24-h recalls are accepted as a suitable but imperfect reference instrument, considered to contain less systematic error than FFQs [11,12]. The collection of intake data on three non-consecutive days in the 24HR may not have been sufficient to represent usual intake for some participants. Use of this imperfect reference instrument may have resulted in correlated errors between the two methods. Conversely, discrepancy between the SMILE-FFQ and 24HR may be the result of departure from true intake by either or both questionnaires.

4.4. Opportunities

As an online tool, the SMILE-FFQ is easily administered and shows potential for use in future Australian dental observational studies involving toddlers and pre-schoolers. If, however, the cohort is substantially different from the one described here, or the tool is modified in any way, an external or internal validation study is required to assess validity [10,11,12]. Further information about the questionnaire is available from the researchers upon request. The SMILE-FFQ has recently been administered to a cohort of approximately 1600 families as part of the NHMRC-funded Study of Mothers and Infants Life Events affecting oral health (SMILE) [15].

5. Conclusions

The SMILE-FFQ has been developed to capture total and free sugars intake within the unique food landscape of Australian toddlers by parent report. Investigations of relative validity and repeatability suggest that this tool performs similarly to repeat 24-h recalls at ranking participants, with a tendency to underestimate intakes in participants with lower reported 24HR values and overestimate intakes in those with higher reported values. The findings of this study suggest that the SMILE-FFQ is suitable for use in observational studies of Australian toddlers wanting to use ranked total and free sugars intakes. As with all FFQs, it is recommended that future administrations of the tool are accompanied with internal or external calibration or further validity testing if administered to a dissimilar cohort.

Supplementary Materials

The following are available online at www.mdpi.com/1660-4601/14/11/1361/s1, Supplementary Material: SMILE Food Frequency Questionnaire.

Acknowledgments

This research is part of the NHMRC funded Study of Mothers and Infants Life Events affecting oral health (SMILE) project (APP1046219).

Author Contributions

Gemma Devenish and Jane Scott conceived and designed the study; Gemma Devenish drafted the FFQ in consultation with Jane Scott, Andrea Begley, and Loc Do; Gemma Devenish and Aqif Mukhtar designed the Access database for pairing the FFQ and nutrient data; Gemma Devenish, Aqif Mukhtar, and Jane Scott analysed the data; Gemma Devenish wrote the paper in consultation with all authors.

Conflicts of Interest

The authors declare no conflict of interest. The funding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; and in the decision to publish the results.

References

  1. World Health Organization. Guideline: Sugars Intake for Adults and Children; WHO: Geneva, Switzerland, 2015. [Google Scholar]
  2. Australian Institute of Health and Welfare. Australia’s Health 2012; AIHW: Canberra, Australia, 2012.
  3. Australian Institute of Health and Welfare. Oral Health and Dental Care in Australia: Key Facts and Figures 2015; AIHW: Canberra, Australia, 2015.
  4. Armfield, J.M.; Spencer, A.J. Quarter of a century of change: Caries experience in Australian children, 1977–2002. Aust. Dent. J. 2008, 53, 151–159. [Google Scholar] [CrossRef] [PubMed]
  5. Moynihan, P.J.; Kelly, S.A. Effect on Caries of Restricting Sugars Intake: Systematic Review to Inform WHO Guidelines. J. Dent. Res. 2014, 93, 8–18. [Google Scholar] [CrossRef] [PubMed]
  6. Moynihan, P.; Petersen, P.E. Diet, nutrition and the prevention of dental diseases. Public Health Nutr. 2004, 7, 201–226. [Google Scholar] [CrossRef] [PubMed]
  7. Sheiham, A.; James, W.P. A new understanding of the relationship between sugars, dental caries and fluoride use: Implications for limits on sugars consumption. Public Health Nutr. 2014, 17, 2176–2184. [Google Scholar] [CrossRef] [PubMed]
  8. Harris, R.; Nicoll, A.D.; Adair, P.M.; Pine, C.M. Risk factors for dental caries in young children: A systematic review of the literature. Community Dent. Health 2004, 21, 71–85. [Google Scholar] [PubMed]
  9. Valaitis, R.; Hesch, R.; Passarelli, C.; Sheehan, D.; Sinton, J. A systematic review of the relationship between breastfeeding and early childhood caries. Can. J. Public Health 2000, 91, 411–417. [Google Scholar] [PubMed]
  10. Thompson, F.E.; Subar, A.F. Dietary Assessment Methodology. In Nutrition in the Prevention and Treatment of Disease, 3rd ed.; Coulston, A.M., Boushey, C., Ferruzzi, M., Eds.; Academic Press: London, UK, 2013. [Google Scholar]
  11. Willett, W. Nutritional Epidemiology, 3rd ed.; Oxford University Press: Oxford, UK, 2013. [Google Scholar]
  12. National Institutes of Health; National Cancer Institute. Dietary Assessment Primer. Available online: http://dietassessmentprimer.cancer.gov (accessed on 22 June 2017).
  13. Evans, E.W.; Hayes, C.; Palmer, C.A.; Bermudez, O.I.; Naumova, E.N.; Cohen, S.A.; Must, A. Development of a pediatric cariogenicity index. J. Public Health Dent. 2013, 73, 179–186. [Google Scholar] [CrossRef] [PubMed]
  14. Ortiz-Andrellucchi, A.; Henríquez-Sánchez, P.; Sánchez-Villegas, A.; Peña-Quintana, L.; Mendez, M.; Serra-Majem, L. Dietary assessment methods for micronutrient intake in infants, children and adolescents: A systematic review. Br. J. Nutr. 2010, 102, S87–S117. [Google Scholar] [CrossRef] [PubMed]
  15. Do, L.G.; Scott, J.A.; Thomson, W.M.; Stamm, J.W.; Rugg-Gunn, A.J.; Levy, S.M.; Wong, C.; Devenish, G.; Ha, D.H.; Spencer, A.J. Common risk factor approach to address socioeconomic inequality in the oral health of preschool children—A prospective cohort study. BMC Public Health 2014, 14, 429. [Google Scholar] [CrossRef] [PubMed]
  16. Scott, J.A.; Binns, C.W.; Aroni, R.A. The influence of reported paternal attitudes on the decision to breast-feed. J. Paediatr. Child Health 1997, 33, 305–307. [Google Scholar] [CrossRef] [PubMed]
  17. Scott, J.A.; Binns, C.W.; Oddy, W.H.; Graham, K.I. Predictors of Breastfeeding Duration: Evidence from a Cohort Study. Pediatrics 2006, 117, e646–e655. [Google Scholar] [CrossRef] [PubMed]
  18. Australian Institute of Health and Welfare. 2010 Australian National Infant Feeding Survey: Indicator Results; AIHW: Canberra, Australia, 2011.
  19. Daniels, L.; Magarey, A.; Battistutta, D.; Nicholson, J.; Farrell, A.; Davidson, G.; Cleghorn, G. The NOURISH randomised control trial: Positive feeding practices and food preferences in early childhood—A primary prevention program for childhood obesity. BMC Public Health 2009, 9, 387. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Coulston, A.M.; Boushey, C.J.; Ferruzzi, M. Nutrition in the Prevention and Treatment of Disease, 3rd ed.; Academic Press: London, UK, 2013. [Google Scholar]
  21. Food Standards Australia New Zealand. AUSNUT 2011–13—Food Composition Database. 2014. Available online: http://www.foodstandards.gov.au/science/monitoringnutrients/ausnut/Pages/default.aspx (accessed on 22 June 2017).
  22. Marriott, L.D.; Inskip, H.M.; Borland, S.E.; Godfrey, K.M.; Law, C.M.; Robinson, S.M. What do babies eat? Evaluation of a food frequency questionnaire to assess the diets of infants aged 12 months. Public Health Nutr. 2009, 12, 967–972. [Google Scholar] [CrossRef] [PubMed]
  23. Andersen, L.F.; Lande, B.; Trygg, K.; Hay, G. Validation of a semi-quantitative food-frequency questionnaire used among 2-year-old Norwegian children. Public Health Nutr. 2004, 7, 757–764. [Google Scholar] [CrossRef] [PubMed]
  24. Watson, E.O.; Heath, A.L.; Taylor, R.W.; Mills, V.C.; Barris, A.C.; Skidmore, P.M. Relative validity and reproducibility of an FFQ to determine nutrient intakes of New Zealand toddlers aged 12–24 months. Public Health Nutr. 2015, 18, 3265–3271. [Google Scholar] [CrossRef] [PubMed]
  25. Flood, V.M.; Wen, L.M.; Hardy, L.L.; Rissel, C.; Simpson, J.M.; Baur, L.A. Reliability and validity of a short FFQ for assessing the dietary habits of 2–5-year-old children, Sydney, Australia. Public Health Nutr. 2013, 1–12. [Google Scholar] [CrossRef] [PubMed]
  26. Australian Bureau of Statistics. Australian Health Survey: Users’ Guide, 2011–13: 24-Hour Dietary Recall. 2014. Available online: http://www.abs.gov.au/ausstats/[email protected]/Lookup/4363.0.55.001Chapter65022011-13 (accessed on 22 June 2017).
  27. Louie, J.C.Y.; Moshtaghian, H.; Boylan, S.; Flood, V.M.; Rangan, A.M.; Barclay, A.W.; Brand-Miller, J.C.; Gill, T.P. A systematic methodology to estimate added sugar content of foods. Eur. J. Clin. Nutr. 2015, 69, 154–161. [Google Scholar] [CrossRef] [PubMed]
  28. Atkins, L.A.; McNaughton, S.A.; Campbell, K.J.; Szymlek-Gay, E.A. Iron intakes of Australian infants and toddlers: Findings from the Melbourne Infant Feeding, Activity and Nutrition Trial (InFANT) Program. Br. J. Nutr. 2016, 115, 285–293. [Google Scholar] [CrossRef] [PubMed]
  29. Lombard, M.J.; Steyn, N.P.; Charlton, K.E.; Senekal, M. Application and interpretation of multiple statistical tests to evaluate validity of dietary intake assessment methods. Nutr. J. 2015, 14, 40. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Masson, L.F.; McNeill, G.; Tomany, J.O.; Simpson, J.A.; Peace, H.S.; Wei, L.; Grubb, D.A.; Bolton-Smith, C. Statistical approaches for assessing the relative validity of a food-frequency questionnaire: Use of correlation coefficients and the kappa statistic. Public Health Nutr. 2003, 6, 313–321. [Google Scholar] [CrossRef] [PubMed]
  31. Bland, J.M.; Altman, D.G. Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement. Lancet 1986, 327, 307–310. [Google Scholar] [CrossRef]
  32. Bland, J.M.; Altman, D.G. Measuring agreement in method comparison studies. Stat. Methods Med. Res. 1999, 8, 135–160. [Google Scholar] [CrossRef] [PubMed]
  33. Huybrechts, I.; De Backer, G.; De Bacquer, D.; Maes, L.; De Henauw, S. Relative Validity and Reproducibility of a Food-Frequency Questionnaire for Estimating Food Intakes among Flemish Preschoolers. Int. J. Environ. Res. Public Health 2009, 6, 382–399. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Australian Bureau of Statistics. Australian Health Survey 2011–13; ABS: Canberra, Australia, 2014.
  35. Bell, L.K.; Golley, R.K.; Magarey, A.M. A short food-group-based dietary questionnaire is reliable and valid for assessing toddlers’ dietary risk in relatively advantaged samples. Br. J. Nutr. 2014, 112, 627–637. [Google Scholar] [CrossRef] [PubMed]
  36. Burrows, T.L.; Collins, K.; Watson, J.; Guest, M.; Boggess, M.M.; Neve, M.; Rollo, M.; Duncanson, K.; Collins, C.E. Validity of the Australian Recommended Food Score as a diet quality index for Pre-schoolers. Nutr. J. 2014, 13, 87. [Google Scholar] [CrossRef] [PubMed]
  37. Mills, V.C.; Skidmore, P.M.L.; Watson, E.O.; Taylor, R.W.; Fleming, E.A.; Heath, A.-L.M. Relative Validity and Reproducibility of a Food Frequency Questionnaire for Identifying the Dietary Patterns of Toddlers in New Zealand. J. Acad. Nutr. Diet. 2015, 115, 551–558. [Google Scholar] [CrossRef] [PubMed]
  38. Kiwanuka, S.N.; Åstrøm, A.N.; Trovik, T.A. Sugar snack consumption in Ugandan schoolchildren: Validity and reliability of a food frequency questionnaire. Community Dent. Oral Epidemiol. 2006, 34, 372–380. [Google Scholar] [CrossRef] [PubMed]
  39. Commonwealth Scientific Industrial Research Organisation. 2007 Australian National Children’s Nutrition and Physical Activity Survey: Main Findings; CSIRO: Canberra, Australia, 2008.
  40. Magarey, A.; Watson, J.; Golley, R.K.; Burrows, T.; Sutherland, R.; McNaughton, S.A.; Denney-Wilson, E.; Campbell, K.; Collins, C. Assessing dietary intake in children and adolescents: Considerations and recommendations for obesity research. Int. J. Pediatr. Obes. 2011, 6, 2–11. [Google Scholar] [CrossRef] [PubMed]
  41. Livingstone, M.B.; Robson, P.J.; Wallace, J.M. Issues in dietary intake assessment of children and adolescents. Br. J. Nutr. 2004, 92, S213–S222. [Google Scholar] [CrossRef] [PubMed]
  42. Gahagan, S. Development of Eating Behavior: Biology and Context. J. Dev. Behav. Pediatr. 2012, 33, 261–271. [Google Scholar] [CrossRef] [PubMed]
  43. Kobayashi, T.; Kamimura, M.; Imai, S.; Toji, C.; Okamoto, N.; Fukui, M.; Date, C. Reproducibility and validity of the food frequency questionnaire for estimating habitual dietary intake in children and adolescents. Nutr. J. 2011, 10, 27. [Google Scholar] [CrossRef] [PubMed]
  44. Fumagalli, F.; Pontes Monteiro, J.; Sartorelli, D.S.; Vieira, M.N.C.M.; de Lourdes Pires Bianchi, M. Validation of a food frequency questionnaire for assessing dietary nutrients in Brazilian children 5 to 10 years of age. Nutrition 2008, 24, 427–432. [Google Scholar] [CrossRef] [PubMed]
  45. Margetts, B.M.; Nelson, M. Design Concepts in Nutritional Epidemiology, 2nd ed.; Oxford University Press: New York, NY, USA, 1997. [Google Scholar]
  46. Börnhorst, C.; Huybrechts, I.; Ahrens, W.; Eiben, G.; Michels, N.; Pala, V.; Molnár, D.; Russo, P.; Barba, G.; Bel-Serrat, S.; et al. Prevalence and determinants of misreporting among European children in proxy-reported 24 h dietary recalls. Br. J. Nutr. 2012, 109, 1257–1265. [Google Scholar] [CrossRef] [PubMed]
  47. Radnitz, C.; Todd, L.E. Social desirability may explain why some caregivers of overweight children report less frequent high calorie food intake. Eat. Behav. 2016, 23, 48–51. [Google Scholar] [CrossRef] [PubMed]
  48. Mohammadifard, N.; Omidvar, N.; Houshiarrad, A.; Neyestani, T.; Naderi, G.-A.; Soleymani, B. Validity and reproducibility of a food frequency questionnaire for assessment of fruit and vegetable intake in Iranian adults. J. Res. Med. Sci. 2011, 16, 1286–1297. [Google Scholar] [PubMed]
  49. Subar, A.F.; Frey, C.M.; Harlan, L.C.; Kahle, L. Differences in reported food frequency by season of questionnaire administration: The 1987 National Health Interview Survey. Epidemiology 1994, 5, 226–233. [Google Scholar] [CrossRef] [PubMed]
  50. Tasevska, N. Urinary Sugars—A Biomarker of Total Sugars Intake. Nutrients 2015, 7, 5816–5833. [Google Scholar] [CrossRef] [PubMed]
Ijerph 14 01361 g001 550
Figure 1. Recruitment and data collection for the SMILE-FFQ validation study. SMILE-FFQ: Study of Mothers’ and Infants’ Life Events Food Frequency Questionnaire; 24HR: 24-h recalls.
Figure 1. Recruitment and data collection for the SMILE-FFQ validation study. SMILE-FFQ: Study of Mothers’ and Infants’ Life Events Food Frequency Questionnaire; 24HR: 24-h recalls.
Ijerph 14 01361 g001
Ijerph 14 01361 g002 550
Figure 2. Bland Altman Plots comparing total and free sugars measured by the SMILE-FFQ compared with repeat 24-h recalls: (a) Total Sugars FFQ1 compared with 24HR; (b) Total Sugars FFQ2 compared with 24HR; (c) Free Sugars FFQ1 compared with 24HR; (d) Free Sugars FFQ2 compared with 24HR. FFQ 1 and 2 refers to the same tool (SMILE-FFQ) administered before (FFQ1) and after (FFQ2) the recalls.
Figure 2. Bland Altman Plots comparing total and free sugars measured by the SMILE-FFQ compared with repeat 24-h recalls: (a) Total Sugars FFQ1 compared with 24HR; (b) Total Sugars FFQ2 compared with 24HR; (c) Free Sugars FFQ1 compared with 24HR; (d) Free Sugars FFQ2 compared with 24HR. FFQ 1 and 2 refers to the same tool (SMILE-FFQ) administered before (FFQ1) and after (FFQ2) the recalls.
Ijerph 14 01361 g002
Ijerph 14 01361 g003 550
Figure 3. Bland Altman Plots comparing total and free sugars measured by repeat administrations of the SMILE-FFQ: (a) Total Sugars FFQ1 compared with FFQ2; (b) Free Sugars FFQ1 compared with FFQ2.
Figure 3. Bland Altman Plots comparing total and free sugars measured by repeat administrations of the SMILE-FFQ: (a) Total Sugars FFQ1 compared with FFQ2; (b) Free Sugars FFQ1 compared with FFQ2.
Ijerph 14 01361 g003
Table
Table 1. Demographic characteristics of study participants (n = 95).
Table 1. Demographic characteristics of study participants (n = 95).
CharacteristicsMean (SD)
Age of mother (years)34.2 (±4.4)
Age of child (months)25 (±4.1)
Gender of child: female (%)43.2
Mothers’ education: completed university (%)70.6
Table
Table 2. Total and Free sugars as measured by SMILE-FFQ1 a, SMILE-FFQ2, and repeat 24-h recalls.
Table 2. Total and Free sugars as measured by SMILE-FFQ1 a, SMILE-FFQ2, and repeat 24-h recalls.
MethodMean ± SDMedianRangePercentileNormality
RawTransformed b
25th75thSkewK-S c pSkewK-S c p
Total Sugars:
FFQ1 (g)74.8 ± 30.866.518.9–149.751.296.30.570.001−0.410.200
FFQ2 (g)68.7 ± 28.665.222.6–148.349.183.60.900.018−0.130.200
24HR (g)65.9 ± 19.364.830.0–130.452.279.00.720.200−0.110.200
Free Sugars:
FFQ1 (g)22.0 ± 15.519.01.0–75.611.126.91.310.000−0.630.160
FFQ2 (g)21.3 ± 15.018.11.0–71.810.229.21.260.009−0.930.004
24HR (g)18.3 ± 9.717.32.1–58.710.924.91.050.036−0.720.091
a FFQ 1 and 2 refers to the same tool (SMILE-FFQ) administered before (FFQ1) and after (FFQ2) the recalls; b Natural Log Transformation; c Kolmogorov–Smirnov test for normality. FFQ: Food Frequency Questionnaire; 24HR: 24-h recalls.
Table
Table 3. Relative validity of the SMILE-FFQ a compared with repeat 24-h recalls.
Table 3. Relative validity of the SMILE-FFQ a compared with repeat 24-h recalls.
MethodCorrelationCross-Classification
Pearson bChi-Square cκw d
Similarly Classified (%)Adjacently Classified (%)Dissimilarly Classified (%)
Total Sugars: FFQ10.532 **54.735.89.50.387
FFQ20.642 **55.840.04.20.454
Free Sugars: FFQ10.392 **52.637.99.50.359
FFQ20.484 **50.438.910.60.320
a FFQ 1 and 2 refers to the same tool (SMILE-FFQ) administered before (FFQ1) and after (FFQ2) the recalls; b Outcome criteria correlation coefficient (Pearson): Good: ≥0.50; Acceptable: 0.20–0.49; Poor: <0.20 [29,30]; c Outcome criteria cross-classification (Chi Square): Good: ≥50% similarly classified and ≤10% dissimilarly classified; poor <50% similarly classified and >10% dissimilarly classified [29,30]; d Outcome criteria Weighted Kappa: good ≥ 0.61; acceptable 0.20–0.6; poor < 0.20 [29,30]. ** p < 0.01.
Table
Table 4. Bland Altman statistics for the SMILE-FFQ a compared with repeat 24-h recalls.
Table 4. Bland Altman statistics for the SMILE-FFQ a compared with repeat 24-h recalls.
MethodMean Difference ± SD (g)Agreement p Value bLimits of Agreement LOA (g)Bland-Altman Index c (%)
Total Sugars: FFQ18.9 ± 26.80.008−43.7, 61.45.3
FFQ22.8 ± 22.60.596−41.6, 47.25.3
Free Sugars: FFQ13.7 ± 15.70.180−27.1, 34.57.4
FFQ23.1 ± 15.40.490−27.2, 33.37.4
a FFQ 1 and 2 refers to the same tool (SMILE-FFQ) administered before (FFQ1) and after (FFQ2) the recalls; b Outcome criteria nonparametric agreement (Wilcoxon): Good: p > 0.05; Poor: p ≤ 0.05 [29]; c Bland Altman index % of persons outside of LOA. Values ≤ 5% indicate good agreement [31,32].
Table
Table 5. Reproducibility of the SMILE-FFQ a.
Table 5. Reproducibility of the SMILE-FFQ a.
CorrelationCross-Classification
Pearson bChi-Square cκw d
Similarly Classified (%)Adjacently Classified (%)Dissimilarly Classified (%)
Total Sugars0.494 **51.540.08.50.359
Free Sugars0.693 **59.034.86.30.464
a Reproducibility was assessed by comparing results from the SMILE-FFQ administered to the same cohort a minimum of six weeks apart; b Outcome criteria correlation coefficient (Pearson): Good: ≥0.50; Acceptable: 0.20–0.49; Poor: <0.20 [29,30]; c Outcome criteria cross-classification (Chi Square): Good: ≥50% similarly classified and ≤10% dissimilarly classified; Poor: <50% similarly classified and >10% dissimilarly classified [29,30]; d Outcome criteria Weighted Kappa: Good: ≥0.61; Acceptable: 0.20–0.60; Poor: <0.20 [29,30]. ** p < 0.01.
Table
Table 6. Bland Altman statistics for repeat administrations of the SMILE-FFQ a.
Table 6. Bland Altman statistics for repeat administrations of the SMILE-FFQ a.
Mean Difference ± SD (g)Agreement p Value b Limits of Agreement LOA (g)Bland-Altman Index c (%)
Total Sugars6.1 ± 28.50.027−49.8, 61.95.3
Free Sugars0.6 ± 12.00.583−22.9, 24.26.3
a Reproducibility was assessed by comparing results from the SMILE-FFQ administered to the same cohort a minimum of six weeks apart; b Outcome criteria nonparametric agreement (Wilcoxon): Good: p > 0.05; Poor: p ≤ 0.05 [29]; c Bland Altman index % of persons outside of LOA. Values ≤ 5% indicate good agreement [31,32].

Share and Cite

MDPI and ACS Style

Devenish, G.; Mukhtar, A.; Begley, A.; Do, L.; Scott, J. Development and Relative Validity of a Food Frequency Questionnaire to Assess Intakes of Total and Free Sugars in Australian Toddlers. Int. J. Environ. Res. Public Health 2017, 14, 1361. https://doi.org/10.3390/ijerph14111361

AMA Style

Devenish G, Mukhtar A, Begley A, Do L, Scott J. Development and Relative Validity of a Food Frequency Questionnaire to Assess Intakes of Total and Free Sugars in Australian Toddlers. International Journal of Environmental Research and Public Health. 2017; 14(11):1361. https://doi.org/10.3390/ijerph14111361

Chicago/Turabian Style

Devenish, Gemma, Aqif Mukhtar, Andrea Begley, Loc Do, and Jane Scott. 2017. "Development and Relative Validity of a Food Frequency Questionnaire to Assess Intakes of Total and Free Sugars in Australian Toddlers" International Journal of Environmental Research and Public Health 14, no. 11: 1361. https://doi.org/10.3390/ijerph14111361

APA Style

Devenish, G., Mukhtar, A., Begley, A., Do, L., & Scott, J. (2017). Development and Relative Validity of a Food Frequency Questionnaire to Assess Intakes of Total and Free Sugars in Australian Toddlers. International Journal of Environmental Research and Public Health, 14(11), 1361. https://doi.org/10.3390/ijerph14111361

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop