Next Article in Journal
Grouped Bees Algorithm: A Grouped Version of the Bees Algorithm
Next Article in Special Issue
Traffic Priority-Aware Adaptive Slot Allocation for Medium Access Control Protocol in Wireless Body Area Network
Previous Article in Journal / Special Issue
Static Human Detection and Scenario Recognition via Wearable Thermal Sensing System
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Wearable Food Intake Monitoring Technologies: A Comprehensive Review

1
Department of Computer Science and Engineering, University at Buffalo, Buffalo, NY 14228, USA
2
Department of Preventive Medicine, Northwestern University, Evanston, IL 60208, USA
3
Department of Computer Science, Northwestern University, Evanston, IL 60208, USA
*
Author to whom correspondence should be addressed.
Computers 2017, 6(1), 4; https://doi.org/10.3390/computers6010004
Submission received: 23 September 2016 / Revised: 19 December 2016 / Accepted: 28 December 2016 / Published: 24 January 2017
(This article belongs to the Special Issue Theory, Design and Prototyping of Wearable Electronics and Computing)

Abstract

:
Wearable devices monitoring food intake through passive sensing is slowly emerging to complement self-reporting of users’ caloric intake and eating behaviors. Though the ultimate goal for the passive sensing of eating is to become a reliable gold standard in dietary assessment, it is currently showing promise as a means of validating self-report measures. Continuous food-intake monitoring allows for the validation and refusal of users’ reported data in order to obtain more reliable user information, resulting in more effective health intervention services. Recognizing the importance and strength of wearable sensors in food intake monitoring, there has been a variety of approaches proposed and studied in recent years. While existing technologies show promise, many challenges and opportunities discussed in this survey, still remain. This paper presents a meticulous review of the latest sensing platforms and data analytic approaches to solve the challenges of food-intake monitoring, ranging from ear-based chewing and swallowing detection systems that capture eating gestures to wearable cameras that identify food types and caloric content through image processing techniques. This paper focuses on the comparison of different technologies and approaches that relate to user comfort, body location, and applications for medical research. We identify and summarize the forthcoming opportunities and challenges in wearable food intake monitoring technologies.

1. Introduction

Eating is essential for survival, but to live a high quality lifestyle, healthy eating is crucial. Good quality food intake has the potential to directly prevent metabolic disturbances resulting in poor human body function [1], and prevent eating-related diseases such as diabetes [2], obesity [3] and cardiovascular disease [4]. Eating-related diseases such as obesity not only affect physical health but also result in significant economic (diagnosis, treatment of diseases, etc.) and societal (loss of productivity, disability pensions, etc.) costs [5]. According to [6], for an individual with a BMI (body mass index) of 40 kg/m 2 , a weight loss of 5% is expected to result in a $402 reduction in annual prescription drug costs, a 10% weight loss saves $679, a 15% weight loss saves $869, and a 20% weight loss saves $999. Moreover, cost of lost productivity due to obesity is $12,988.51 annually [7].
To address this problem of maintaining healthy food intake, several approaches have been investigated since the mid-20th century. Manual approaches with handwritten self-reports of dietary intake were among the first accepted measures of food-intake recording [8]. Recently, approaches have been advanced with the ubiquitous use of smartphones. Several diet-diary applications have been developed [9]. However, these note-based approaches rely heavily on user memory and recall, recall which is not practical for many, especially handicapped users with memory disorders. Therefore, it is necessary to develop a more effective approach with the help of technology for automatic food intake monitoring. During the past decade, a wide range of approaches have been investigated in multiple studies employing different devices. These devices comprise varying sensing modalities, such as acoustic, visual, inertial, EGG (electroglottography), EMG (electromyography), capacitive and piezoelectric sensors. Some approaches use a combined fusion of sensors. Since each sensing technology has its advantage and limitation for food-intake monitoring depending on its application, we conduct this in-depth review focusing on the most recent and promising wearable technologies. We hope this review study can provide insightful observations and recommendations for the field of passive sensing of eating.

1.1. Goals of The Review

  • Up-to-date review of recent novel approaches: There is a lack of comprehensive survey papers surrounding the current state-of-the-art in wearable food intake monitoring. The wide range of food intake monitoring and wearable dietary assessment approaches carry different advantages and drawbacks. Considering that increasingly more research is aimed towards objective dietary assessment, up-to-date reviews of technology and methodology are needed for this field.
    Some previous surveys in this field have been created in the past. Amft [10], and Fontana and Sazonov [11] each bring about a thorough survey that addresses a wide range of topics on chewing and swallowing detection using wearable sensors. Amft and Troster [12] also generated a short survey comparing sensors in different body locations. Although the reviews mentioned above provided an important milestone for food intake detection and characterization, these reviews mostly focus on sensor hardware, and do not provide detailed comparison between wearable technologies in food intake monitoring needed to make an informed decision on which sensing platform to use. As a result, we build on prior surveys to cover the more recent approaches in wearable sensors on food intake monitoring.
    Recently, there have been a few other surveys related to food intake measures. Burke et al. [13] focus on note-based approaches including a discussion on electronic diaries (personal digital assistant), but not wearable sensor technology. Fischer [14] provides a fair amount of prior work, however focuses mainly on automated chew event monitoring using chewing sounds. Stumbo [15] provides a detailed review of visual (camera-based) approaches of food intake monitoring. Kalantarian et al. [16] honed in on piezoelectric- and audio-based approaches for food intake monitoring. However, these surveys fall short of discussing the trade-offs of each sensing approach needed to make an informed decision, including sensing algorithms and comfort assessment.
  • Application-oriented review: In this review, we focus not only on modalities but also on food intake applications. Approaches are classified based on these applications: eating behavior determination, food type classification, and volume/weight estimation. By using this differentiation, a comprehensive comparison can be established comparing drawbacks/advantages as well as signal processing analytic approaches. Real-life applications are likewise mentioned creating a clearer connection between the application and the corresponding wearable sensor.
  • Signal processing and machine learning-oriented review: Recognizing that signal processing plays a vital role in analyzing data from wearable sensors for food intake monitoring, we will focus a section on analyzing signal processing methods to investigate various approaches of data acquisition and feature extraction. Schemes are classified based on food intake monitoring applications: eating behavior/food intake detection, volume estimation and food type classification. In these signal processing procedures, we will provide an exhaustive survey of specific classification algorithms. Tamura and Kimura [17] provide a review on wearable food intake monitoring but do not significantly address signal processing.

1.2. Organization of this Survey

This survey will cover a full range of wearable food intake monitoring systems. The following Section 2 describes the background literature surrounding food intake monitoring. Then we classify the sensor systems based on applications in medical fields (Section 3) and based on sensing approaches (e.g., acoustic, camera, pressure, inertial, EMG/EGG and combination) (Section 5). Between these two classifications, there is a Section 4 that covers food intake mechanisms to provide a deeper understanding of modality and body locations for each sensor. Section 6 describes the signal processing methodology with related food intake applications. Section 7 discusses the challenges and potential research focus in the field of automated food intake monitoring. Finally, we conclude our work in Section 8.

2. Background

2.1. Food Intake

Food intake and eating behaviors directly affects one’s health. Food intake is essential in energy balance which is represented by Calories [18]. Furthermore, Schwartz et al. [19] emphasized the importance of food intake on energy homeostasis. Food intake controlling the energy homeostasis, the match of energy intake and energy expenditure, have shown to effect obesity of rodents , highlighting its importance on human health [19]. Note that energy expenditure is also an important variable in maintaining optimal health. Energy expenditure monitoring using passive sensors have been extensively studied, such as activity monitor-based [20], accelerometer-based [21], and multi-sensor armband-based [22]. Since this paper focuses on food intake monitoring, energy expenditure monitoring will not be investigated in depth. Besides energy control, food intake has a proportional relationship with liver stiffness levels affecting liver failure and chronic liver disease. Specifically, liver stiffness increases immediately following food intake for up to 60 min (p = 0.01). After 180 min, the liver stiffness level returned to a steady-state. Consequently, food intake has been shown to increase liver stiffness in patients with Hepatitis C virus infection and with healthy controls [23].
Mastication behavior, including bolus size and chewing rates, also plays an important part in food digestion [24]. Poor masticatory system performance results in ulcers and gastric carcinoma [25]. Otherwise, proper mastication habits facilitates the digestive processes. Blisset [26] states that natural bite sizes (the size of food that constitutes an ordinary mouthful) have been reported to weigh between 5.5 and 7.4 g for males and between 3.6 and 5.4 g for females when consuming peanuts and doughnuts, respectively. Bite sizes larger than these ranges greatly effect masticatory efficiency [27]. Chewing rate influences not only digestive system but also masticatory muscle. Karibe [28] showed that a slow rate of chewing benefits the patients with chronic masticatory muscle pain. Moreover, chewing rate is a stress indicator with its effects on salivary cortisol levels [29]; hence, it is crucially important to control chewing rates and bolus size/weight during food intake.

2.2. Food Intake Monitoring

Food intake monitoring plays a vital role in ensuring normal bodily growth. This normal growth is determined by amino acids (AA) [30], which are utilized for the synthesis of protein and many other low-molecular-weight compounds. Moreover, non-essential amino acids (NEAA), which are synthesized by the human body, are responsible for many of our physiological functions, represented in Figure 1. Humans must have NEAA in their diet for optimal growth, development, lactation, reproduction, and health [1]. However, amino acid synthesis is vulnerable to limitation due to inappropriate diet and food intake [31]. Accordingly, proper monitoring of food intake ensures the control of amino acid metabolism for meeting the needs of maximal human growth and optimal health. Food intake monitoring effects major physiologic functions mentioned in Figure 1.
Obesity is a major risk factor for many diseases, and approximately one-third of American adults are obese [32]. Figure 2 shows that overall more than 35% of U.S men and women were obese in 2009–2010 [33], accounting for 200 billion US dollars in health care cost annual [2] because it is implicated in many chronic illnesses including diabetes, heart disease, stroke, and some types of cancer. Maintaining adequate quality of food intake serves as an essential component to preventing obesity caused by other than genetic factors. Jean et al. [3] stated that with proper regulation of food intake and exercise, the excessive energy stored in the form of fat will be prevented when energy intake balances energy expenditure.

2.3. Related Works on the Manual Approach

  • Conventional self-report method: There have been several attempts to monitor food intake by self-report on paper. Block [34] comprehensively analyzed the validation, agreements with known results and conclusions regarding self-report methods, such as the history method, the 24-h dietary recall method and the seven-day method. Bingham [35] represented a clearer comparison between methods instituted by showing the percentages of errors in each method in people who finally completed the survey. As in the review, the percentages misclassification of results into quartiles of the distribution of nutrition intake are around 10%, and only 15% of participants completed the method. Moreover, self-report methods discourages people from taking part in food intake monitoring due to lack of convenience [35].
  • Self-report method using smart phones: With the popularity of smart phones, self-report approaches have improved in convenience and time to complete self-monitoring. The network capability on a smart phone also allows this approach to have more features, such as a quick response from doctors and real-time self-monitoring. Wohler [36] developed a food intake survey system on the Motorola Q9h smart phone. The system allows participants to record full days of meals and to send the data to an SQLite database for analysis in the background. The system provides improved user experience than the paper self-report approach. A similar approach by Tsai et al. [37] had a comparable process to Wohler et al. [36], but focuses particularly on calorie intake.
  • Food frequency questionnaire: The manual approach in dietary assessment also involves using food frequency questionnaires (FFQ), using questionnaire systems (e.g., applications, websites, etc.) to collect food consumption frequency of foods and beverages over a period of time, but not for every single meal. For example, Fellaize [38] developed an FFQ system called Food4Me to gather food intake data for every four-day period. Besides, Green [39] also generated an FFQ method named 3-d weighed food record (3d-WFR) with a three-day frequency.
Overall, manual approaches are subject to subjective and recall bias since it depends on reports rather than on true objective food intake information. Self-reporting or the manual approach is believed to be imprecise and extremely biased [40]. For example, in obese people, the bias is too high to obtain a valid estimation of dietary energy intake.

2.4. Existing Works on the Automatic Approach

  • Pressure method: The pressure method takes advantage of force sensors embedded in a tablecloth or underneath a table. This approach is not wearable, but is novel in food intake monitoring. The earliest study about the pressure approach was performed by Chang et al. [41]. The research exploits a weight sensor enclosed under the table. This weighing surface can measure the amount of food transferred between parts of the table and the amount of food consumed by each person at the table when there are changes in the plates’ weight. In order to detect and distinguish how people eat a weight matching algorithm is designed, taking into account changes in weight on different plates. The accuracy of this method is approximately 80%, showing a promising start for a novel approach. Recent research using pressure sensors was conducted by Zhou et al. [42], shown in Figure 3. This system is basically a pressure sensor with the upper layer serving as both protection and a dining surface. Specific eating actions (e.g., cutting, scooping and transporting food) are detected when someone dines on this tablecloth. The sensor is less of a burden than the one in Chang et al. [41]. However, it is capable of detecting only eating movements without either weight estimation or food classification.
  • Surveillance-video method: In contrast to wearable camera approaches, which will be discussed in Section 5.2, the surveillance-video approach employs an external camera for automatic food intake detection. Cadavid et al. [43] developed an active appearance model system for chewing detection based on captured images of the human face from an external camera during the eating process. The chewing detection is marked by the quasi-periodic nature of chewing fetched in the model parameters.
  • Doppler sensing method: Another existing automatic approach to food intake monitoring is using microwave Doppler motion sensors. Tanigawa et al. [45] explored the use of the Doppler effect in their system to sense the Doppler signal of mastication produced from vertical jaw movements. The distinct moving speed of the jaw during mastication is determined from the equation of the relationship between Doppler frequency and moving speed.
The pressure, surveillance-video and Doppler sensing approaches are limited in detecting eating habits by weighing and indetecting solid foods that involve jaw movement. Current achievements using these three approaches have not enabled them to detect specific types of food or food composition. Moreover, the non-wearable limitation of these approaches restrains the mobility of the sensor system for food intake monitoring, restricting their utility in detecting activity in free-living populations.

2.5. Why a Wearable-Based Approach?

Since manual approaches are proven to be inaccurate and biased [46], we need a more objective and reliable system for dietary assessment. Wearable sensors are potential solutions since they do not rely on people’s subjective judgments; moreover, wearable sensors provide real-time food intake monitoring. Data collected can be instantly synchronized with an information gateway, such as a smart phone application, for useful feedback. Another essential advantage for wearable sensors is creating understanding of behavior in free-living people. It provides convenience for users, removing the burden of self-report, which can be helpful in medical applications, especially when the users are patients. The applications for wearable food intake monitoring systems are broad, which will be assessed in a later section of this paper.
In addition, the accuracy of the automated wearable sensor approach is able to be improved by limiting the unwanted effects of external factors such as noises from the environment. For instance, the sensor-based approach requires denoising of the signal, which can even fully eliminate noises embedded in the collected data. With advancing technology, researchers have been able to develop satisfying denoising signal processing techniques. Ervin et al. [47] succeeded in developing a procedure to denoise dual-axis swallowing accelerometry signals with significant improvements in comparison to previous approaches.

3. Applications of Food Intake Monitoring

3.1. Objective Measure of Caloric Intake

The main applications of automatic food intake monitoring systems on nutrition intake are to avoid subjective influences in manual reports and to provide a comfortable way for controlling food intake in daily life. Related studies manifest that subjective influences in manual reports are impacted by respondent’s motivation, ability to memorize, literate capabilities, desirability and self-awareness [48]. Therefore, food intake sensors have the potential to play a crucial part in maintaining nutrition intake with food recognition and classification.
Amft [49] came up with a wearable ear pad sensor system shown in Figure 4a for food classification. It makes use of acoustic sensors to detect chewing sounds. It is capable of recognizing 19 types of foods with a decent performance of 86.6%. Even though it is not applicable yet in real-life scenarios with a variety of food types, the device demonstrates the feasibility of using sensors in recognizing respondent’s food selection. Accordingly, this automatic food selection or classification will lead to the objectiveness of caloric intake’s measurement.

3.2. Detection of Eating Behavior

Eating behavior is affected by habit, whereby some participants exhibit night eating [57], evening overeating [58], as well as weekend overeating [59], as well as other external aspects, such as environmental factors [60], and social factors [61] or by proximal determinants such as stress [62] due to job pressure [63]. Hence, an objective system is necessary for controlling food intake, as well as human weight.
The most popular approach in weight management is the acoustic approach, which makes full use the sounds produced from chewing and swallowing events for food intake detection. Sazanov et al. [64] proposed an ingestive behavior monitoring device that employed methods based on mel-scale Fourier spectrum, wavelet packets and support vector machines to differentiate swallowing events from respiration, intrinsic speech, head movements, food ingestion and ambient noise. Even though these devices still cannot either classify food or quantify energy consumption, they can characterize food intake behaviors such as number and frequency of chews and swallows. Their applications include diagnosing potential patterns that can lead to weight gain.
Another approach developed by Liu et al. [54] merges both the acoustic approach (microphone) and the visual approach (camera) to detect chewing sounds in applications for obesity. It is demonstrated further in Section 5.6 and Figure 4f. With the assistance of the camera, this system is capable of detecting food portion by time during the eating process through image processing of snapshots of the meal over time. Integrating this with information regarding food type, it can provide helpful data related to eating behavior, which directly impacts obesity.
Chang et al. [41] embedded a weighing sensor and radio-frequency-identification (RFID) reader as a two-layer sensor surface beneath a dining table, to trace the weight of food intake, as shown in Section 2. Based on this system, it is able to track food consumption by measuring the weight of the food content, along with the food movement movement patterns detected using the RFID antenna, which is also used to classify the food items on the table.

3.3. Inpatient Hospital Care

Food intake becomes extremely crucial for patients in a hospital setting [65], and an on-going concern for medical centers. Only 25% of patients receive sufficient amount of energy and protein, and 20% exhibit nutrition risks [66]. In addition to the crisis, hospital nutrition management is lacking in several aspects including: planning and managing nutritional care, nutrition education among medical staff, limited influence on patient behavior [67]. These are all factors beyond the control of wearable sensors, however, automated sensing can help create an objective and reliable measure of food intake, facilitating a solution that helps solve hospital nutrition management.
The eButton from Sun et al. [50] is a creative solution among existing approaches, utilizing a wearable camera embedded in a wearable button for capturing images of food on a plate. The system also includes a smart phone to act as a control and information gateway. This device is capable of classifying food types, estimating the volume of the food and exporting the output of the calories and nutrients of the meal. The wearable ability of this device is satisfactory for use in a hospital. Nutritionists and nurses can place it on patients’ chests for automatic food data collection. This data can be stored in a local database for monitoring patients’ nutrition intake.

4. Food Intake Mechanisms

To understand why and how to employ specific types of sensors and to investigate the impact of their positions on the body, we need to first comprehend the food intake mechanism or digestion process. Because this section serves as a guide for classifying and recognizing food intake sensors’ positions and applications, it will analyze only the brief processes in digestion that relate contiguously to these concerns.
The digestion process is divided into two processes: mechanical digestion and chemical digestion (Figure 5) [68]. One type of chemical digestion is protein digestion, in which your body breaks down dietary protein into usable amino acids, then makes this nutrient available to your cells in support of muscle maintenance, immune function, hormone synthesis, red blood cell formation and tissue repair [69]. One commercial product for protein digestion monitoring is the ITSI Biosciences protein digestion monitoring kit [70], though wearable sensors do not contribute much in this area. In this paper, we will focus more on the mechanical digestion process, since it encloses more inputs, which can be detected via sensors (e.g., vision, sound, pressure, etc.).
  • Mechanism of bite: Bite is recognized as the very first stage in food intake. It involves two main steps: movement of wrist and hand to transport food to the mouth and movement of jaw muscles to take a bite. Wrist movements are typified by characteristic rolling motions (pronation and supination) to deliver food from cutlery to mouth [51]. Following this is the jaw movements for the mouth to obtain the food. The lateral pterygoid muscle is responsible for the abduction of jaw (opening mouth), while temporalis, masseter and medial pterygoid muscles assist the adduction (closing mouth, moving mouth side to side and grinding teeth) [71].
  • Mechanisms of chewing (mastication): The mechanical digestion involves the physical degradation of food [72]. This digestion occurs mainly from the mouth, passing through the pharynx and esophagus. It starts in the oral digestion with the mastication process, or chewing, to produce bolus, which is later swallowed down into the esophagus [73]. In greater detail, during the oral process, the ingested food is moved from the front of the mouth to the teeth so that it can be broken down by crushing and grinding with the teeth [74]. The jaw’s muscles’ movements in this stage are similar to those in the mechanisms of bite with continuously indicative motions of lateral pterygoid and temporalis muscles to open and close the mouth [71].
  • Mechanisms of swallowing: The process begins with the voluntary phase. The bolus, after being formed by folding and manipulating the mixture of food particles and saliva with the tongue, is moved into the back of the tongue to prepare for swallowing [27]. In the involuntary pharyngeal phase, the swallowing reflex is triggered to open and close the upper esophageal sphincter. During this phase, pharyngeal constrictor muscles push the bolus into the esophagus via a stripping action. The last phase, the involuntary esophageal phase, consists of continuous peristalsis by sequential contraction of the superior, middle and inferior pharyngeal constrictor muscles of the esophagus to push the bolus to the stomach [75,76]. These movements of smooth and striated muscles in pharynx and esophagus play a vital role in the source of input for wearable sensors.
These processes contribute to the chewing and swallowing signal properties of the food, which can be detected by various wearable sensing approaches, which will be scrutinized in depth in Section 5. The chemical digestion, which mostly involves the enzymatic breakdown of the food particles, is out of the scope for this paper.

5. Wearable Sensing Approaches

In this section, we provide a detailed review of various sensors with their associated food intake applications and significant studies. Food intake applications include food type classification, eating behavior and volume/weight estimation, described in Figure 6. Different types of sensors will aim at specific food intake detecting applications.

5.1. Acoustic Approach

  • Chewing detection: Detecting and assessing chewing sound by acoustic means has become one of the most appealing topics in food intake monitoring. It attracts increasingly more research in this field day-by-day. Overall, existing approaches focus on using wearable systems, such as microphones, to accumulate sound waves, employing a certain methodology for classifying and analyzing chewing events. Amft [49] specified their vibration-based detecting system using a condenser microphone embedded in an ear pad for food classification, which is demonstrated in Figure 4a. Amft et al. [77] further expanded the food classification to chewing and bite weight recognition. The chewing recognition procedure employs the chewing sequences, waveforms of sounds indicating chewing captured by ear pad sensors. The chewing sequences are distinguished by the feature similarity search event recognition algorithm. This chewing detection later leads to the bite weight recognition from micro-structure variables. Their sound-based recognition system for bite weight shows high accuracy results with only 19% error on one type of food: apple. However, the study only includes three kinds of food (apple, potato chips and lettuce), which is insufficient to make comprehensive conclusions about the effectiveness of this approach. Another study from Amft et al. [78] goes into an in-depth analysis of chewing sounds, as well as specifies the methodology of the procedure and the most appropriate position of the microphone (inner ear, directed towards the eardrum).
    There is a number of other research studies relating to automatic chewing detection. One of them is from Yatani and Truong [79], which is adept at differentiating chewing events from, as well as the softness of the food from the collected sounds. In addition, Olubanjo et al. [80] concentrated on differentiating chewing acoustic events from noisy surroundings with a template-matching method. The study shows a positive result of up to 83.3% on relatively sizeable data samples (13 different tasks, including five solid, four liquid food and four tracheal and non-tracheal events) to support the validity of the result. A similar approach developed by Pabler and Fischer [81] is also capable of detecting chewing sounds. Pabler and Fischer [82] further utilized this method by developing a low computational cost algorithm for detecting food intake sound. Similarly, Pabler et al. [83] took advantage of chewing sound in the breakdown process detected by the sensor system in the outer ear canal. Other studies that are likewise capable of detecting chewing events with the count numbers of chewing and food texture classification are proposed in [84,85].
    The microphone is not the singular sensor for this approach. A strain sensor is also feasible for detecting vibration in chewing events. Sazonov and Fontana [86] embedded a strain sensor below the outer ear to capture the motion of the lower jaw caused by the posterior border of the mandible’s ramus relative to the temporal bone [87].
  • Swallowing detection: Swallowing detection is basically parallel to chewing detection since swallowing is the next phase of chewing in mechanical digestion. The acoustic approach is also feasible for this detection. As mentioned in Section 4, swallowing consists of the open/close of the upper esophageal sphincter and the pushing of bolus by peristaltic waves, which are significant sources of vibration.
    Several significant studies on wearable sensor have been proposed on this type of food intake detection. Sazonov et al. [87] developed a system containing a throat microphone placed over the laryngopharynx in the throat. The signal is strong in this position because it is close to the origin of the swallowing sound. Moreover, the system includes the hardware/software parts for capturing and scoring sounds’ data. Sazonov et al. [64] analyzed in greater detail the methodologies for the acoustic detection of swallowing events. The proposed methods for detection are based on the following time-frequency decompositions: msFS (mel-scale Fourier spectrum) and WPD (wavelet packet decomposition) with classification performed by support vector machine (SVM). Overall, the method shows sufficient results of approximately 86.7% with a test size of 80 subjects.
  • Combined chewing/swallowing detection: Since chewing and swallowing processes correlate with each other, fusion approaches can be employed to detect their events continuously. Lopez-Meyer et al. [88] proposed an SVM model-based approach to detect food intake period from information extracted from the instantaneous swallowing frequency (ISF) signal and the binary indicators of chewing. In another study, Kalantarian et al. [89] generated spectrograms for audio clips corresponding to four types of swallowing and chewing events (sandwich chews/swallows, water swallows and no action) and afterward applying feature extraction for distinguishing those events. Other methods do not focus on chewing and swallowing events, but on food classification. AutoDietary from Bi et al. [90] is a significant example. It is shown to have positive performance in distinguishing solid food and liquid food with the results of 99.7% and 97.8% accuracy, respectively.

5.2. Visual/Camera Approach

The visual approach, unlike the acoustic approach, does not focus on the eating process to extract data, but takes advantages of food’s image and video capture. With the on-going advance of smart phones, as well as cameras and image processing, this approach is getting more and more attention in the field of automatic food intake monitoring with decent amounts of studies. The applications of this approach vary from food classification to portion evaluation of food consumption. It has been proven to be a valid and convenient method together with the effect of reducing dietary under-reporting [91,92]. One of the most famous methods in this approach is developed by Sun et al. [50]: the eButton. As mentioned in Section 3.3 and in Figure 4c, it addresses properly the major concern of this approach: wearability. As the camera is embedded in a minuscule button, it is convenient enough to wear on the chest for food intake collection. Jia et al. [93], Chen et al. [94] and Bai et al. [95] further analyzed insight into the methodology for this system. Zhu et al. [96,97] developed a dietary assessment system based on the smart phone’s camera to detect food portion (volume estimation before and after a meal), as well as food classification. During the food portion estimation process, the camera calibration step and 3D reconstruction step will perform image processing to get the output of food volume. In details, the input is apparent in the image before the camera calibration step to estimate camera parameters. The fiducial marker is used for the scale and the pose. The system divides objects into “geometric classes”. A 3D volume is reconstructed by the unprojected points from feature point extraction based on the parameters of the geometric class. Once the volume estimate for a food item is obtained, the nutrient intake consumed is derived from the estimate based on the USDA Food and Nutrient Database for Dietary Studies [98]. Zhu et al. [98,99] further described the use of SVM and image segmentation in food classification.
In an analogous approach, Puri et al. [100] again focused on the volume estimation and food recognition using captured images from cameras; however, its approach in 3D reconstruction is different. It employs the “point-cloud” technique shown to extract the information of food volume: the heights of the pixels based on the darkness of the pixels are investigated to indicate the food volume [100].
Shang et al. [101] developed the Dietary Data Recording System (DDRS), which is another way for 3D reconstruction to estimate food volume for a visual approach. It imposes a laser beam on the system for generating a laser grid on the food scene. This laser grid assists the depth images’ generation, as well as the creation of the 3D structure from these depth images in different directions. Unlike the mentioned approaches, the DDRS has the advantage of not carrying a fiducial marker for image detection [15]. However, they have a common possibility to provide food composition information from their food database. Another system for the visual approach is called DietCam, described in Kong and Tan [102]. The system primarily focuses on calorie estimation from food portion for obesity prevention. It also features the segmentation technique for image processing and food classification, which is more deeply analyzed. Some other approaches for dietary assessment are proposed in O’Loughlin et al. [103], Kikunaga et al. [104] and Wang et al. [105]. However, all of them engaged the dietitians for food calculation from images instead of developing an automatic system for dietary assessment, utilizing cameras just for capturing pictures. For the main purpose of this paper, we will not analyze these approaches any further.

5.3. Inertial Approach

The inertial approach targets the motion of the eating process, especially wrist motion, since eating goes along with the gesturing of hands. This approach is emerging with the widespread availability of gyroscopes and accelerometers. In this section, we will focus on the gyroscope and accelerometer in wrist motion tracking of food intake.
  • Gyroscope: MEMS (microelectromechanical systems) gyroscopes are suitable for the wrist motion in automatic food intake monitoring because they belong to the nano-scaled vibrating structure gyroscope that is identical to rotating movements. One of the most pioneering method in this approach is developed by Dong et al. [51,106,107,108,109]. The device in this method, which is illustrated in Figure 4c, applies the MEMS gyroscope to automatically track wrist motion every time the users take a bite of food from the radical velocity [51]. The system is capable of getting a positive result with 80% accuracy for detecting bite events across 139 meals with restricted food selection in the laboratory condition. In another method, Scisco et al. [110] took advantage of the relationship between wrist motion and bite and likewise employed the gyroscope in the device.
  • Accelerometer: The accelerometer is also taken into account for the inertial approach. Since motions and the types of this sensor vary, different research in this field is performed. For example, Thomaz et al. [111] described a method using a three-axis accelerometer embedded in a smart watch, which attempts to recognize the eating process, food types and food intake amount. A similar approach, developed by Mendi et al. [112], is even adept at sending data from the accelerometer to the smart phone via Bluetooth. In another advance in the most recent study, Ye et al. [113,114] proposed an automatic structure using two accelerometers simultaneously: wrist-worn and head-mounted accelerometers on such devices as the Pebble Watch and Google Glass to detect chewing events and eating duration with high accuracy: 89.5% in a dataset of 12,003 epochs in which 3325 epochs are chewing-related under the laboratory condition. Another similar inertial system developed by Amft et al. [115] employed four motion sensors at two positions on each arm (upper and lower) to detect drinking and eating gestures and further to detect chewing and swallowing sounds. The accuracy of this system is significant, of approximately 94% among four types of gestures (cutlery, hand, spoon and drink) with two subjects in its experiment. Besides being utilized in the arm positions, a three-axis accelerometer can be embedded in the temple of glasses to detect the intake of crunchy food [116]. In addition, apart from the three-axis accelerometer, Wang et al. [117] exploited a single axis accelerometer embedded in flexible types of wearable objects (glasses, filler, earphone) to detect chewing frequency generated by the temporal muscle.
  • Gyroscope-accelerometer combination: A combination of gyroscope and accelerometer provides an alternative method for the inertial approach. Eskandar [118] embedded two three-axis accelerometers and a two-axis gyroscope for bite detection. However, despite utilizing six axes, the accuracy is humble, with only 46% for single bite recognition across 22,383 manually-labeled bites. It proves that the amount of gyroscope/accelerometer with the related number of axes is not proportional to the detection’s accuracy.

5.4. Physiological Approach

The physiological approach is comprised of two main sensor types: electroglottography (EGG) and electromyography (EMG). EGG sensors have long been used for the investigation of vocal cords’ movements with the implementation of electrodes to detect electrical impedance across the neck near the larynx [119]. This should not be confused with electrogastrography, which has the same abbreviation, but completely different specification (for gastric activity) [120]. EMG likewise employs electrodes, but specifically targets skeletal muscles’ activities [121]. For such specifications, EMG and EGG are applicable to a variety of research in the field of automatic dietary assessment.
  • EGG sensor: The EGG sensor utilizes the motion-induced variations of the electrical impedance, as the amplitude and/or phase change of high-frequency voltage, between two electrodes positioned on the larynx [119]. Because of the characteristic of contacting with the larynx, EGG has a high potential for wearable food intake monitoring. Farooq et al. [52] proposed a testing scheme to evaluate the validity of practicing EGG in food intake detection by setting up the Laryngograph (EGG-D200 from Laryngograph, Ltd., Wallington, U.K. (illustrated in Figure 4d)) around the participant’s neck during the experiment. The swallowing frequency signal will then be recorded by two electrodes embedded in the Laryngograph to detect eating events. The results afterward would be evaluated comparatively with the acoustic method using a microphone placed over the laryngopharynx. Accordingly, the EGG method presents greater accuracy results: 89.7% over 81.1% on female subjects and 90.3% over 85.2% on male subjects among 25 subjects (13 females and 12 males in selected meal choice under the laboratory condition).
  • EMG sensor: EMG has widely been employed in food intake monitoring research. Because of its efficiency in detecting chewing and swallowing signals, EMG focuses on mastication with the bolus size and hardness of the food intake, rather than food classification. For example, Woda et al. [122,123] employed EMG to scrutinize the effect of food hardness, bolus size, chewing cycles and sequence duration in specific food types and testing subject classes. On the other hand, Kohyama et al. [124] took into account mastication efforts for finely-cut foods with the use of the EMG. A further research work of Kohyama et al. featuring EMG investigates more in-depth how chewing behavior is influenced by dental status and age with different types of food [125].
To summarize, this approach of using EGG and EMG is promising, especially in mastication properties’ analysis and food intake detection. However, it still needs more efforts and future work in order to expand its applications for food intake classification.

5.5. Piezoelectric Approach

Another novel approach that is available in the field of wearable food intake monitoring is the piezoelectric sensor approach. It is based on piezoelectricity: the electric charge that leads to voltage production in response to mechanical stress in some solid materials [126].
  • Chewing detection: Besides the application in swallowing detection, the piezoelectric approach is harnessed in chewing detection. Farooq and Sazonov [127] presented the utilization of piezoelectric film sensors for identifying jaw movements during the chewing process. The experimental process contains 12 subjects under no restricted laboratory condition while wearing the sensors for one day. The study more forwardly investigates the detection accuracy over a range of different ensemble classifiers with base learners (more details in Section 6). The average Z-score accuracy of ensembles is 90.76%. Farooq and Sazonov [53] furthermore worked on the the comparison between piezoelectric strain sensor (demonstrated in Figure 4e) and plotter drawn strain sensor in automatic chew counting. Two sensors are positioned below one ear for capturing jaw movements. This study proves a comparatively similar effectiveness in chewing detection between piezoelectric and printed strain sensors (error rates of 8.09% and 8.29%, respectively). Their latter work on this piezoelectric approach specifies the jaw movements on the temporalis muscle as the source of the input signal to the piezoelectric film sensor [128]. Additional works on the piezoelectric method merge it with other approaches for expanding the detected aspects besides chewing and swallowing.
  • Swallowing detection: Taking advantage of this specification, Kalantarian et al. [129] used piezoelectric sensors to capture the motion of the throat’s skin during the swallowing process. Similar to the system in capacitive sensing, this method embeds sensors in a necklace, worn around the user’s neck, for swallowing detection. The study is capable of classifying three types of food (chip, sandwich and water) besides recognizing the state of food (solid or liquid) with an average accuracy of 86% from a population of 10 subjects in a lab-controlled environment.

5.6. Fusion Approach

The fusion approach stands for blending the use of two or more wearable approaches in food intake monitoring. Its purposes vary, ranging from increasing accuracy, synchronizing utilization, simplifying and quickening signal processing to eliminate the drawbacks of a singular sensor approach.
The most evident fusion approach one can derive from previous sections is the acoustic-visual approach. Liu et al. [54] proposed a wearable device that embeds both a camera and a microphone for dietary assessment shown in Figure 4f. In this system, the microphone is utilized to detect chewing or eating events, triggering the camera for food detection and classification, which leads to a more simplified sound signal processing without a complicated denoising pre-process. In addition, this method succeeds in not only being more feasible in real-life scenarios, but also providing a convenient wearable device from the user feedback.
Another fusion approach from Sen et al. [130] making use of the visual approach is the inertial-visual approach. It is similar to the acoustic-visual approach in the that instead of using an acoustic sensor, it applies the accelerometer and gyroscope as the trigger for the camera process. The system consists of three components: smart phone, smart-watch and backend server. While the smart-watch contains all of the participating sensors (camera, accelerometer and gyroscope), the smart phone serves as the gateway between the smart watch and the backend server, which is used to store data and to identify food type from the images.
Besides those relating to the visual approach, the combination approach merges the piezoelectric, inertial and proximity approaches. Fontana et al. [131] employed this method on their automatic ingestion monitor (AIM). The AIM consists of a firm piezoelectric, accelerometer and proximity sensors to discern jaw movements, body motion and hand-to-mouth gestures, respectively, during ingestion. Twelve subjects in the evaluation phase of this study were allowed to carry out the experiment outside the laboratory condition. This approach has high potential of integrating the advantages in wearability, out-of-laboratory applicability and with an acceptable accuracy of 89.8% for food intake detection.

5.7. Other Approaches

5.7.1. Electrical Proximity Sensing

Electrical proximity sensing exploits the physics effects relating to electromagnetic fields to detect the presence of objects. Capacitive sensors are one of the widespread proximity sensing approaches, previously used in various technologies, especially in touchscreen mobile phones and tablets [132]. For eating detection, capacitive sensing exploits the capacitance changes in the human body resulting from muscle motion. Despite being an advanced and novel method, this approach has been conducted by several studies. Cheng et al. [133] pioneered this technology by discovering multiple activity-recognition applications, including chewing and swallowing. This paper set up a basic understanding for the capacitive approach. Further work from Cheng et al. [55] specified the design of this system represented in Figure 4g, consisting of a textile neckband with four capacitors. This novel approach eliminates the previous drawbacks in traditional approaches. For example, in comparison with acoustic and inertial methods, it does not require direct contact with the skin, an inconvenience for users, for detection. Moreover, it can be implemented on a wide range of body locations and is not limited to the neck alone. However, the accuracy of this approach requires improvement, since it is still at a humble 84.4%. Later studies embraced the idea of a system of multiple proximity sensors to improve the accuracy. Bedri et al. [134,135] generated the outer ear interface (OEI) system assembled in an ear pad consisting of three proximity sensors. Its evaluation on 23 participants to test the capability of classifying eating events from non-eating events shows a positive performance of 95.3% in the laboratory condition.

5.7.2. Respiratory Inductance Plethysmography Sensing

In addition to the approaches that utilize conventional food intake monitoring signals, such as chewing/swallowing sounds, jaw movements and the food’s image, this novel method bestows an indirect respiratory signal for eating detection. Respiratory inductance plethysmography (RIP) utilizes Faraday’s law and Lenz’s law in induced electromotive force that generates the signal from the change of area enclosed by the loop [136]. It represents the change in cross-sectional area of the body when breathing. Dong et al. [136] and Dong and Biswas [56,137] constructed an RIP belt shown in Figure 4h containing a breathing signal sensing system that embeds swallowing signatures in this respiratory signal for swallowing detection. The experimental procedure for this RIP belt consists of only six subjects and is limited to the laboratory condition, and its evaluation result has a moderate accuracy of 88% to 73.33% with an SVM classifier; however, this approach has the advantage of no contact with skin and shows an exceptional novelty in wearable food intake monitoring. It enlarges the possibility of discovering more innovative methods that do not rely on conventional food intake signals.

5.8. Comparison

Table 1 provides an overall comparison among modalities, considering the following aspects: body positions, comfort, applications, accuracy, advantages and challenges.

6. Signal Processing Scheme

The methodologies and algorithms used in signal processing vary among studies. For the analysis in this section, we divide it based on the most significant food intake applications, whether for detecting eating behavior, for classifying food types/states or for estimating food volume.

6.1. Eating Behavior/Food Intake Detection

As briefly presented in Section 4, eating behavior consists of chewing and swallowing detection with related factors, such as the rates of chewing, bolus size and duration. These food intake applications run parallel with food intake detection and eating behavior with chewing and swallowing as the mark of food intake. Moreover, with features extracted from the signal processing, such as eating duration and time of each meal, eating behavior can further help with determining a person’s eating lifestyle. As a result, the signal processing scheme is similar for eating behavior, food intake detection and other applications.
The overall signal processing procedure for these applications is represented in Figure 7. Depending on the studies and approaches, different models and classifiers for event detection will be utilized. We take into account Bi et al. [90] and Lopez-Meyer et al. [88]. In these systems, the first step is undeniably sound recording with an 8000-Hz sampling rate with an amplifier. The next step includes framing to produce sound frames for event detection afterward. Bi et al. [90] employed the hidden Markov model (HMM) for event detection. This model has long been utilized for sound recognition [138,139]. Among 4000 signals collected, 86.6% of those are accurately classified into chewing/swallowing events. Besides the HMM, SVM is likewise a popular model for food intake detection. For example, Lopez-Meyer et al. [88] employed SVM on an instantaneous swallowing frequency signal (difference in time of frequencies between two consecutive swallows). This model has a high accuracy of more than 95%; however, it depends considerably on the time duration of the experiment. SVM is used not only in the acoustic approach, but also in the visual approach [98]. In addition, other different classifiers are employed for distinctive approaches. For example, for the inertial approach, Dong et al. [106] adopted the naive Bayes classifier with the naive assumption of the independence of features for the classification problem. Farooq et al. [140] proposed the artificial neural network (ANN) classifier for the AIM system mentioned in Section 5.6 and compared it with the SVM. The ANN shows better performance in this study. However, this is not a universal assumption. Fontana et al. [141] further dug into the use of random forest classification for this AIM system for studying the importance of the time domain and frequency domain features in food intake monitoring detection. Other signal processing schemes even consist of stacked classifiers in multiple stages. Zhou et al. [42] introduced the two-stage scheme of the confidence-based AdaBoost algorithm ConfAdaBoost.M1 with decision trees. Farooq and Sazonov [127] later expanded the investigation of this scheme into six stacked ensembles. In this study, the first stage contains base learners, including decision trees, Fishers linear discriminate analysis (LDA) and logistic regression analysis. The second stage is comprised of bootstrap aggregation and AdaBoost. The Z-score results differ for each stack; however, bootstrap aggregation with Fisher LDA has the best performance.

6.2. Food Type Classification

Food type classification is another crucial application in food intake. It focuses on determining certain types of food in each meal. It enables the wearable sensor systems to discover nutrient information. Because of the complication in the signal processing scheme, however, this application is still scarce in studies in the wearable food intake monitoring field.
Signal processing schemes for food type classification are more limited than for others. They are dependent on the number of food types, dataset and approach; however, we can consider the signal processing introduced by Amft et al. [77] for acoustic food type recognition, which is shown in Figure 8.
The recognition procedure starts with data collection as usual. Then, the author employs food similarity search (FFS), an online event recognition algorithm, to detect temporal boundaries of chewing events in sound data. Generally speaking, the FFS is utilized to detect the food intake event (chewing) for later food type classification, as well as to eliminate overlapping temporal events and to retain the events with the largest food classification confidences. A Fisher’s linear discriminant feature transformation-based classifier is trained for the classification step [77,142]. A majority vote among all classified events in one chewing sequence will later determine the final food type.

6.3. Volume Estimation

Another important branch of food intake classification is the volume estimation. It deals with the determination of the food volume in dishes in each meal. It is crucial in indicating the amount of food, as well as nutrient quantity. Its signal processing procedure is mostly based on the camera-based approach because of its possibility to capture a visual image of food. Its framework for signal processing, represented in Figure 9, is general for most methods. Starting with the image collected, camera calibration is performed to estimate the camera’s extrinsic and intrinsic parameters (principal points, distortion, orientation, etc.). Next, 3D volume reconstruction, which is the key step for volume estimation, will transform the 2D image to a 3D model. This step differs in different studies. For example, Sun et al. [50] employed the wire mesh method, while Puri et al. [100] and Shang et al. [101] applied the point-cloud extraction technique and the laser grid, respectively, for constructing the 3D model of the food. The 3D reconstructing structure from images reveals the food’s geometric properties. The integrated information of these results in the final volume estimation of the food.

7. Discussion

There still exists a variety of challenges and obstacles in wearable food intake monitoring. One of them is the applicability of wearable devices. Most of the approaches are currently for research purposes only and need expanding for a real-life scenario. Further work in this field should be taken into account, such as improving the user interface and compliance for real-life applications. Nonetheless, the biggest challenge lays in the improvement of accuracy associated with signal processing procedure. The average accuracy in food intake detection is approximately 90% in the laboratory condition in which signal processing schemes still rely on a variety of factors, such as the duration of eating time and the number of food samples. For such an approach as the acoustic approach, accuracy also depends on the surrounding noise, which requires an acceptable denoising process. Future work requires a more complete signal processing structure to improve accuracy covering a wide range of external conditions in real-life scenarios.
Another concern correlates with a specific application, i.e., food type classification. It is considered to be the final problem to tackle in most studies. However, this application has many burdens to deal with, including food collection for the database, the accuracy decrease relative to the increase in the number of foods and the reliable signal processing scheme. Real-life scenarios have vast numbers of foods with different properties. It is a hindering process to record all the sound or image properties of each food for food type classification. Moreover, the greater the number of foods, the more complicated the signal processing scheme becomes and the less accurate the system is. For example, Amft et al. [77] are able to classify between three types of food with 94% accuracy. However, Amft [49], categorizing between four types of food, obtained an accuracy of only 86.6%. This aspect is determined by the signal processing structure. Nonetheless, signal analysis for this application is not yet consolidated and limited. Further work on food type categorization is needed to pursue a strong signal analyzing approach to tackle a large amount of food types, but still be able to maintain sufficient accuracy.
Besides food type classification, food composition determination is another concern for future work, since the ultimate purpose of dietary assessment is to figure out in detail what we intake from food. Nevertheless, this application is extremely difficult, and signals from current external wearable sensors seem insufficient to determine this factor. Current wearable sensors are limited in calories, with studies in [50,102] of the visual approach containing a direct formula between food volume and calories estimation [143]. Future works in food composition determination need to cover more important food nutrient information.
In this review paper, we also scrutinize the signal processing aspect of food intake monitoring with associated algorithms and methodologies. This directly affects the effectiveness and capability of related food intake applications. In consideration of multiple classifiers having been employed for food intake detection across several approaches (acoustic, inertial, physiological, etc.), SVM is the most widely used, while stacked ensembles, such as bootstrap aggregation with Fisher LDA, provide a solution to improve classification and recognition accuracy. On the other hand, apart from food intake detection, other applications (food type classification and volume estimation) are in need for studies on efficient signal processing procedures to develop more effective systems on these food intake monitoring aspects.
Last, but not least, since the whole food intake process also expands through the esophagus to the stomach and colon, further studies should take this process into account in their assessment. At present, there exists only the electrogastrography approach to capture the stomach muscles’ activities [120]. Assessment of food intake digestion in this latter phase is limited.

8. Conclusions

This paper presents a review of current wearable technologies of food intake monitoring, a crucial process in daily human activities determining wide ranges of bodily functions, such as amino acid production, eating behavior and eating-related diseases. To address various applications, different approaches have been employed in existing studies, including acoustic, visual, EMG/EGG, inertial, fusion, piezoelectric, proximity and inductance plethysmography approaches. They differ from each other in many aspects: body positions, comfort, application, accuracy, advantages, limitations, current applicability and obtrusiveness. Signal processing schemes or signal analysis structures mentioned also vary, depending on the applications. Overall, there is currently a growing potential in exploring wearable food intake monitoring to tackle existing obstacles step-by-step, such as improving accuracy, breaking out of the restricted laboratory condition, enhancing the comfort level, resolving efficient food type classification and advancing an effective signal processing procedure. If the existing challenges can be addressed, wearable sensors in food intake monitoring will play a significant role in improving overall human healthcare.

Author Contributions

Wenyao Xu and Nabil Alshurafa generated the idea, proposed the outline for this paper and supervised the project, Tri Vu conducted main researches and analysis on this topic and Feng Lin contributed to the editing of final manuscript. All authors have gone through, proofread and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hou, Y.; Yin, Y.; Wu, G. Dietary essentiality of “nutritionally non-essential amino acids” for animals and humans. Exp. Biol. Med. 2015, 240, 997–1007. [Google Scholar] [CrossRef] [PubMed]
  2. Volkow, N.D.; Wang, G.J.; Baler, R.D. Reward, dopamine and the control of food intake: Implications for obesity. Trends Cogn. Sci. 2011, 15, 37–46. [Google Scholar] [CrossRef] [PubMed]
  3. Mayer, J.; Thomas, D.W. Regulation of food intake and obesity. Science 1967, 156, 328–337. [Google Scholar] [CrossRef] [PubMed]
  4. Bazzano, L.A.; He, J.; Ogden, L.G.; Loria, C.M.; Vupputuri, S.; Myers, L.; Whelton, P.K. Fruit and vegetable intake and risk of cardiovascular disease in US adults: The first National Health and Nutrition Examination Survey Epidemiologic Follow-up Study. Am. J. Clin. Nutr. 2002, 76, 93–99. [Google Scholar] [PubMed]
  5. Seidell, J.C. Societal and personal costs of obesity. Exp. Clin. Endocrinol. Diabetes 1998, 106, 7–10. [Google Scholar] [CrossRef] [PubMed]
  6. Cawley, J.; Meyerhoefer, C.; Biener, A.; Hammer, M.; Wintfeld, N. Savings in Medical Expenditures Associated with Reductions in Body Mass Index Among US Adults with Obesity, by Diabetes Status. Pharmacoeconomics 2015, 33, 707–722. [Google Scholar] [CrossRef] [PubMed]
  7. Sonntag, D.; Ali, S.; De Bock, F. Lifetime indirect cost of childhood overweight and obesity: A decision analytic model. Obesity 2016, 24, 200–206. [Google Scholar] [CrossRef] [PubMed]
  8. Basiotis, P.P.; Welsh, S.O.; Cronin, F.J.; Kelsay, J.L.; Mertz, W. Number of days of food intake records required to estimate individual and group nutrient intakes with defined confidence. J. Nutr. 1987, 117, 1638–1641. [Google Scholar] [PubMed]
  9. Darby, A.; Strum, M.W.; Holmes, E.; Gatwood, J. A Review of Nutritional Tracking Mobile Applications for Diabetes Patient Use. Diabetes Technol. Ther. 2016, 18, 200–212. [Google Scholar] [CrossRef]
  10. Amft, O. Automatic Dietary Monitoring Using on-Body Sensors. Ph.D. Thesis, Chemnitz University of Technology, Chemnitz, Germany, 2008. [Google Scholar]
  11. Fontana, J.M.; Sazonon, E. Detection and characterization of food intake by wearable sensors. In Wearable Sensors: Fundamentals, Implementation and Applications, 1st ed.; Sazonov, E., Neuman, M.R., Eds.; Academic Press: New York, NY, USA, 2014; Chapter 7.4; pp. 591–616. [Google Scholar]
  12. Amft, O.; Tröster, G. On-body sensing solutions for automatic dietary monitoring. IEEE Pervasive Comput. 2009, 8, 62–70. [Google Scholar] [CrossRef]
  13. Burke, L.E.; Warziski, M.; Starrett, T.; Choo, J.; Music, E.; Sereika, S.; Stark, S.; Sevick, M.A. Self-monitoring dietary intake: Current and future practices. J. Ren. Nutr. 2005, 15, 281–290. [Google Scholar] [CrossRef] [PubMed]
  14. Päßler, S.; Fischer, W.J. Food intake monitoring: Automated chew event detection in chewing sounds. IEEE J. Biomed. Health Inform. 2014, 18, 278–289. [Google Scholar] [CrossRef] [PubMed]
  15. Stumbo, P.J. New technology in dietary assessment: A review of digital methods in improving food record accuracy. Proc. Nutr. Soc. 2013, 72, 70–76. [Google Scholar] [CrossRef] [PubMed]
  16. Kalantarian, H.; Mortazavi, B.; Alshurafa, N.; Sideris, C.; Le, T.; Sarrafzadeh, M. A comparison of piezoelectric-based inertial sensing and audio-based detection of swallows. Obes. Med. 2016, 1, 6–14. [Google Scholar] [CrossRef]
  17. Tamura, T.; Kimura, Y. Review of monitoring devices for food intake. CICSJ Bull. 2016, 34, 73. [Google Scholar]
  18. Morton, G.; Cummings, D.; Baskin, D.; Barsh, G.; Schwartz, M. Central nervous system control of food intake and body weight. Nature 2006, 443, 289–295. [Google Scholar] [CrossRef] [PubMed]
  19. Schwartz, M.W.; Woods, S.C.; Porte, D.; Seeley, R.J.; Baskin, D.G. Central nervous system control of food intake. Nature 2000, 404, 661–671. [Google Scholar] [PubMed]
  20. Heil, D.P. Predicting activity energy expenditure using the Actical® activity monitor. Res. Q. Exerc. Sport 2006, 77, 64–80. [Google Scholar] [CrossRef] [PubMed]
  21. Chen, K.Y.; Sun, M. Improving energy expenditure estimation by using a triaxial accelerometer. J. Appl. Physiol. 1997, 83, 2112–2122. [Google Scholar] [PubMed]
  22. Fruin, M.L.; Rankin, J.W. Validity of a multi-sensor armband in estimating rest and exercise energy expenditure. Med. Sci. Sports Exerc. 2004, 36, 1063–1069. [Google Scholar] [CrossRef] [PubMed]
  23. Mederacke, I. Food intake increases liver stiffness in patients with chronic or resolved hepatitis C virus infection. Liver Int. 2009, 29, 1500–1506. [Google Scholar] [CrossRef] [PubMed]
  24. Buschang, P.; Throckmorto, G.; Travers, K.H.; Johnson, G. The effects of bolus size and chewing rate on masticatoryperformance with artificial test foods. J. Oral Rehabil. 1997, 24, 522–526. [Google Scholar] [CrossRef] [PubMed]
  25. Sognnaes, R. Studies on masticatory efficiency. PartsI-IV. Am. J. Orthod. 1941, 27, 309. [Google Scholar]
  26. Blisset, A. Effect of bolus size on chewing, swallowing, oral soft tissue and tongue movement. J. Oral Rehabil. 2007, 34, 572–582. [Google Scholar] [CrossRef] [PubMed]
  27. Prinz, J.F.; Heath, M.R. Bolus dimensions in normal chewing. J. Oral Rehabil. 2000, 27, 765–768. [Google Scholar] [CrossRef] [PubMed]
  28. Karibe, H.; Goddard, G.; Gear, R.W. Sex Differences in Masticatory Muscle Pain after Chewing. J. Dent. Res. 2003, 82, 112–116. [Google Scholar] [CrossRef] [PubMed]
  29. Tasaka, A.; Tahara, Y.; Sugiyama, T.; Sakurai, K. Influence of chewing rate on salivary stress hormone levels. J. Jpn. Prosthodont. Soc. 2008, 52, 482–487. [Google Scholar] [CrossRef]
  30. Young, V.R. Nutritional Balance Studies: Indicators of Human Requirements or of Adaptive Mechanisms? J. Nutr. 1986, 116, 700–703. [Google Scholar] [PubMed]
  31. Reeds, P.J. Dispensable and Indispensable Amino Acids for Humans. J. Nutr. 2000, 130, 1835S–1840S. [Google Scholar] [PubMed]
  32. Ogden, C.L.; Carroll, M.D.; Curtin, L.R.; McDowell, M.A.; Tabak, C.J.; Flegal, K.M. Prevalence of overweight and obesity in the United States, 1999–2004. JAMA 2006, 295, 1549–1555. [Google Scholar] [CrossRef] [PubMed]
  33. Ogden, C.L.; Carroll, M.D.; Kit, B.K.; Flegal, K.M. Prevalence of obesity in the United States, 2009–2010. NCHS Data Brief 2012, 131, 1–8. [Google Scholar]
  34. Block, G. A review of validations of dietary assessment methods. Am. J. Epidemiol. 1982, 115, 492–505. [Google Scholar] [PubMed]
  35. Bingham, S.; Gill, C.; Welch, A.; Day, K.; Cassidy, A.; Khaw, K.; Sneyd, M.; Key, T.; Roe, L.; Day, N. Comparison of dietary assessment methods in nutritional epidemiology: Weighed records v. 24 h recalls, food-frequency questionnaires and estimated-diet records. Br. J. Nutr. 1994, 72, 619–643. [Google Scholar] [CrossRef] [PubMed]
  36. Wohlers, E.M.; Sirard, J.R.; Barden, C.M.; Moon, J.K. Smart phones are useful for food intake and physical activity surveys. In Proceedings of the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Minneapolis, MN, USA, 3–6 September 2009; pp. 5183–5186.
  37. Tsai, C.C.; Lee, G.; Raab, F.; Norman, G.J.; Sohn, T.; Griswold, W.G.; Patrick, K. Usability and feasibility of PmEB: A mobile phone application for monitoring real time caloric balance. Mob. Netw. Appl. 2007, 12, 173–184. [Google Scholar] [CrossRef]
  38. Fallaize, R.; Forster, H.; Macready, A.L.; Walsh, M.C.; Mathers, J.C.; Brennan, L.; Gibney, E.R.; Gibney, M.J.; Lovegrove, J.A. Online dietary intake estimation: Reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record. J. Med. Internet Res. 2014, 16, e190. [Google Scholar] [CrossRef] [PubMed]
  39. Green, T.J.; Allen, O.B.; O’Connor, D.L. A three-day weighed food record and a semiquantitative food-frequency questionnaire are valid measures for assessing the folate and vitamin B-12 intakes of women aged 16 to 19 years. J. Nutr. 1998, 128, 1665–1671. [Google Scholar] [PubMed]
  40. Schoeller, D.A. The Role of Dexfenfluramine in the Regulation of Energy Balance Limitations in the assessment of dietary energy intake by self-report. Metabolism 1995, 44, 18–22. [Google Scholar] [CrossRef]
  41. Chang, K.H.; Liu, S.Y.; Chu, H.H.; Hsu, J.Y.J.; Chen, C.; Lin, T.Y.; Chen, C.Y.; Huang, P. The Diet-aware Dining Table: Observing Dietary Behaviors over a Tabletop Surface. In Proceedings of the 4th International Conference on Pervasive Computing (PERVASIVE’06), Dublin, Ireland, 7–10 May 2006; pp. 366–382.
  42. Zhou, B.; Cheng, J.; Sundholm, M.; Reiss, A.; Huang, W.; Amft, O.; Lukowicz, P. Smart table surface: A novel approach to pervasive dining monitoring. In Proceedings of the 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom), St. Louis, MO, USA, 23–27 March 2015; pp. 155–162.
  43. Cadavid, S.; Abdel-Mottaleb, M.; Helal, A. Exploiting visual quasi-periodicity for real-time chewing event detection using active appearance models and support vector machines. Pers. Ubiquitous Comput. 2012, 16, 729–739. [Google Scholar] [CrossRef]
  44. Zhou, B.; Cheng, J.; Sundholm, M.; Lukowicz, P. From smart clothing to smart table cloth: Design and implementation of a large scale, textile pressure matrix sensor. In Proceedings of the International Conference on Architecture of Computing Systems, Lubeck, Germany, 25–28 February 2014; pp. 159–170.
  45. Tanigawa, S.; Nishihara, H.; Kaneda, S.; Haga, H. Detecting mastication by using microwave doppler sensor. In Proceedings of the 1st International Conference on Pervasive Technologies Related to Assistive Environments, Athens, Greece, 16–19 July 2008; p. 88.
  46. Westerterp, K.R.; Goris, A.H. Validity of the assessment of dietary intake: Problems of misreporting. Curr. Opin. Clin. Nutr. Metab. Care 2002, 5, 489–493. [Google Scholar] [CrossRef] [PubMed]
  47. Sejdić, E.; Steele, C.M.; Chau, T. A procedure for denoising dual-axis swallowing accelerometry signals. Physiol. Meas. 2010, 31, N1. [Google Scholar] [CrossRef] [PubMed]
  48. Hill, R.J.; Davies, P.S.W. The validity of self-reported energy intakeas determined using the doubly labelled water technique. Br. J. Nutr. 2001, 85, 415–430. [Google Scholar] [CrossRef] [PubMed]
  49. Amft, O. A wearable ear pad sensor for chewing monitoring. In Proceedings of the IEEE Sensors, Waikoloa, HI, USA, 1–4 November 2010; pp. 222–227.
  50. Sun, M.; Burke, L.E.; Mao, Z.H.; Chen, Y.; Chen, H.C.; Bai, Y.; Li, Y.; Li, C.; Jia, W. eButton: A Wearable Computer for Health Monitoring and Personal Assistance. In Proceedings of the 51st Annual Design Automation Conference, San Francisco, CA, USA, 1–5 June 2014.
  51. Dong, Y.; Hoover, A.; Scisco, J.; Muth, E. A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion Tracking. Appl. Psychophysiol. Biofeedback 2012, 37, 205–215. [Google Scholar] [CrossRef]
  52. Farooq, M.; Fontana, J.M.; Sazonov, E. A novel approach for food intake detection using electroglottography. Physiol. Meas. 2014, 35, 739. [Google Scholar] [CrossRef] [PubMed]
  53. Farooq, M.; Sazonov, E. Comparative testing of piezoelectric and printed strain sensors in characterization of chewing. In Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 25–29 August 2015; pp. 7538–7541.
  54. Liu, J.; Johns, E.; Atallah, L.; Pettitt, C.; Lo, B.; Frost, G.; Yang, G.Z. An intelligent food-intake monitoring system using wearable sensors. In Proceedings of the 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks, London, UK, 9–12 May 2012; pp. 154–160.
  55. Cheng, J.; Zhou, B.; Kunze, K.; Rheinländer, C.C.; Wille, S.; Wehn, N.; Weppner, J.; Lukowicz, P. Activity recognition and nutrition monitoring in every day situations with a textile capacitive neckband. In Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, Zurich, Switzerland, 8–12 September 2013; pp. 155–158.
  56. Dong, B.; Biswas, S. Wearable diet monitoring through breathing signal analysis. In Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 1186–1189.
  57. Stunkard, A.J.; Grace, W.; Wolff, H. Night eating syndrome. In Eating Disorders and Obesity: A Comprehensive Handbook; Guilford Press: New York, NY, USA, 2002; pp. 183–188. [Google Scholar]
  58. Kant, A.K.; Ballard-Barbash, R.; Schatzkin, A. Evening eating and its relation to self-reported body weight and nutrient intake in women, CSFII 1985-86. J. Am. Coll. Nutr. 1995, 14, 358–363. [Google Scholar] [CrossRef] [PubMed]
  59. Haines, P.S.; Hama, M.Y.; Guilkey, D.K.; Popkin, B.M. Weekend eating in the United States is linked with greater energy, fat, and alcohol intake. Obes. Res. 2003, 11, 945–949. [Google Scholar] [CrossRef] [PubMed]
  60. Popkin, B.M.; Duffey, K.; Gordon-Larsen, P. Environmental influences on food choice, physical activity and energy balance. Physiol. Behav. 2005, 86, 603–613. [Google Scholar] [CrossRef] [PubMed]
  61. Pliner, P.; Mann, N. Influence of social norms and palatability on amount consumed and food choice. Appetite 2004, 42, 227–237. [Google Scholar] [CrossRef] [PubMed]
  62. Wardle, J.; Gibson, E. Impact of Stress on Diet: Processes and Implications. In Stress and the Heart: Psychosocial Pathways to Coronary Heart Disease; Stansfeld, S., Marmot, M.G., Eds.; BMJ Books: London, UK, 2002; pp. 124–149. [Google Scholar]
  63. World Health Organization. The World Health Report 2001: Mental Health: New Understanding, New Hope; World Health Organization: Geneva, Switzerland, 2001. [Google Scholar]
  64. Sazonov, E.; Makeyev, O.; Schuckers, S.; Lopez-Meyer, P.; Melanson, E.L.; Neuman, M.R. Automatic Detection of Swallowing Events by Acoustical Means for Applications of Monitoring of Ingestive Behavior. IEEE Trans. Biomed. Eng. 2010, 57, 626–633. [Google Scholar] [CrossRef] [PubMed]
  65. Kondrup, J. Can food intake in hospitals be improved? Clin. Nutr. 2001, 20, 153–160. [Google Scholar] [CrossRef]
  66. Kondrup, J.; Johansen, N.; Plum, L.; Bak, L.; Larsen, I.H.; Martinsen, A.; Andersen, J.; Baernthsen, H.; Bunch, E.; Lauesen, N. Incidence of nutritional risk and causes of inadequate nutritional care in hospitals. Clin. Nutr. 2002, 21, 461–468. [Google Scholar] [CrossRef] [PubMed]
  67. Beck, A.M.; Balkn, U.; Fürst, P.; Hasunen, K.; Jones, L.; Keller, U.; Melchior, J.; Mikkelsen, B.E.; Schauder, P.; Sivonen, L.; et al. Food and nutritional care in hospitals: How to prevent undernutrition–report and guidelines from the Council of Europe. Clin. Nutr. 2001, 20, 455–460. [Google Scholar] [CrossRef] [PubMed]
  68. Maton, A. Human Biology and Health; Prentice Hall: Englewood Cliffs, NJ, USA, 1997. [Google Scholar]
  69. Protein Digestion Inside the Human Body. Available online: http://healthyeating.sfgate.com/protein-digestion-inside-human-body-6044.html (accessed on 30 December 2016).
  70. Protein Digestion Monitoring Kit. Available online: http://www.itsibio.com/prodm.html (accessed on 30 December 2016).
  71. Muscles of Mastication. Available online: http://www.musclesused.com/muscles-mastication-chewin/ (accessed on 30 December 2016).
  72. Hoebler, C.; Karinthi, A.; Devaux, M.F.; Guillon, F.; Gallant, D.; Bouchet, B.; Melegari, C.; Barry, J.L. Physical and chemical transformations of cereal food during oral digestion in human subjects. Br. J. Nutr. 1998, 80, 429–436. [Google Scholar] [CrossRef] [PubMed]
  73. Bornhorst, G.M.; Singh, R.P. Bolus formation and disintegration during digestion of food carbohydrates. Compr. Rev. Food Sci. Food Saf. 2012, 11, 101–118. [Google Scholar] [CrossRef]
  74. Hiiemae, K.M.; Palmer, J. Food transport and bolus formation during complete feeding sequences on foods of different initial consistency. Dysphagia 1999, 14, 31–42. [Google Scholar] [CrossRef] [PubMed]
  75. Pedersen, A.; Bardow, A.; Jensen, S.B.; Nauntofte, B. Saliva and gastrointestinal functions of taste, mastication, swallowing and digestion. Oral Dis. 2002, 8, 117–129. [Google Scholar] [CrossRef] [PubMed]
  76. Chen, J. Food oral processing—A review. Food Hydrocoll. 2009, 23, 1–25. [Google Scholar] [CrossRef]
  77. Amft, O.; Kusserow, M.; Tröster, G. Bite weight prediction from acoustic recognition of chewing. IEEE Trans. Biomed. Eng. 2009, 56, 1663–1672. [Google Scholar] [CrossRef] [PubMed]
  78. Amft, O.; Stäger, M.; Lukowicz, P.; Tröster, G. Analysis of chewing sounds for dietary monitoring. In Proceedings of the International Conference on Ubiquitous Computing, Osaka, Japan, 7–11 September 2005; pp. 56–72.
  79. Yatani, K.; Truong, K.N. BodyScope: A Wearable Acoustic Sensor for Activity Recognition. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, PA, USA, 5–8 September 2012; pp. 341–350.
  80. Olubanjo, T.; Moore, E.; Ghovanloo, M. Detecting food intake acoustic events in noisy recordings using template matching. In Proceedings of the 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, USA, 25–27 February 2016; pp. 388–391.
  81. Päßler, S.; Fischer, W.J. Acoustical method for objective food intake monitoring using a wearable sensor system. In Proceedings of the 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops, Dublin, Ireland, 23–26 May 2011; pp. 266–269.
  82. Paßler, S.; Fischer, W.J. Food Intake Activity Detection Using a Wearable Microphone System. In Proceedings of the 2011 7th International Conference on Intelligent Environments (IE), Nottingham, UK, 25–28 July 2011; pp. 298–301.
  83. Päßler, S.; Wolff, M.; Fischer, W.J. Food intake monitoring: An acoustical approach to automated food intake activity detection and classification of consumed food. Physiol. Meas. 2012, 33, 1073. [Google Scholar] [CrossRef] [PubMed]
  84. Nishimura, J.; Kuroda, T. Eating habits monitoring using wireless wearable in-ear microphone. In Proceedings of the 3rd International Symposium on Wireless Pervasive Computing, Santorini, Greece, 7–9 May 2008; pp. 130–132.
  85. Shuzo, M.; Komori, S.; Takashima, T.; Lopez, G.; Tatsuta, S.; Yanagimoto, S.; Warisawa, S.; Delaunay, J.J.; Yamada, I. Wearable Eating Habit Sensing System Using Internal Body Sound. J. Adv. Mech. Des. Syst. Manuf. 2010, 4, 158–166. [Google Scholar] [CrossRef]
  86. Sazonov, E.S.; Fontana, J.M. A Sensor System for Automatic Detection of Food Intake Through Non-Invasive Monitoring of Chewing. IEEE Sens. J. 2012, 12, 1340–1348. [Google Scholar] [CrossRef] [PubMed]
  87. Sazonov, E.; Schuckers, S.; Lopez-Meyer, P.; Makeyev, O.; Sazonova, N.; Melanson, E.L.; Neuman, M. Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. Physiol. Meas. 2008, 29, 525. [Google Scholar] [CrossRef] [PubMed]
  88. Lopez-Meyer, P.; Schuckers, S.; Makeyev, O.; Sazonov, E. Detection of periods of food intake using Support Vector Machines. In Proceedings of the 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, Argentina, 1–4 September 2010; pp. 1004–1007.
  89. Kalantarian, H.; Alshurafa, N.; Pourhomayoun, M.; Sarin, S.; Le, T.; Sarrafzadeh, M. Spectrogram-based audio classification of nutrition intake. In Proceedings of the IEEE Healthcare Innovation Conference (HIC), Seattle, WA, USA, 8–10 October 2014; pp. 161–164.
  90. Bi, Y.; Lv, M.; Song, C.; Xu, W.; Guan, N.; Yi, W. AutoDietary: A Wearable Acoustic Sensor System for Food Intake Recognition in Daily Life. IEEE Sens. J. 2016, 16, 806–816. [Google Scholar] [CrossRef]
  91. Chang, U.J.; Ko, S. A study on the dietary intake survey method using a cameraphone. Korean J. Community Nutr. 2007, 12, 198–205. [Google Scholar]
  92. Gemming, L.; Rush, E.; Maddison, R.; Doherty, A.; Gant, N.; Utter, J.; Ni Mhurchu, C. Wearable cameras can reduce dietary under-reporting: Doubly labelled water validation of a camera-assisted 24 h recall. Br. J. Nutr. 2015, 113, 284–291. [Google Scholar] [CrossRef] [PubMed]
  93. Jia, W.; Yue, Y.; Fernstrom, J.D.; Zhang, Z.; Yang, Y.; Sun, M. 3D localization of circular feature in 2D image and application to food volume estimation. In Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA, 28 August–1 September 2012; pp. 4545–4548.
  94. Chen, H.C.; Jia, W.; Yue, Y.; Li, Z.; Sun, Y.N.; Fernstrom, J.D.; Sun, M. Model-based measurement of food portion size for image-based dietary assessment using 3D/2D registration. Meas. Sci. Technol. 2013, 24, 105701. [Google Scholar] [CrossRef] [PubMed]
  95. Bai, Y.; Jia, W.; Mao, Z.H.; Sun, M. Automatic eating detection using a proximity sensor. In Proceedings of the 2014 40th Annual Northeast Bioengineering Conference (NEBEC), Boston, MA, USA, 25–27 April 2014; pp. 1–2.
  96. Zhu, F.; Bosch, M.; Boushey, C.J.; Delp, E.J. An image analysis system for dietary assessment and evaluation. In Proceedings of the 2010 IEEE International Conference on Image Processing, Hong Kong, China, 26–29 September 2010; pp. 1853–1856.
  97. Zhu, F.; Bosch, M.; Schap, T.; Khanna, N.; Ebert, D.S.; Boushey, C.J.; Delp, E.J. Segmentation assisted food classification for dietary assessment. In Proceedings of the IS&T/SPIE Electronic Imaging, San Francisco, CA, USA, 23–27 January 2011.
  98. Zhu, F.; Bosch, M.; Woo, I.; Kim, S.; Boushey, C.J.; Ebert, D.S.; Delp, E.J. The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation. IEEE J. Sel. Top. Signal Process. 2010, 4, 756–766. [Google Scholar] [PubMed]
  99. Zhu, F.; Bosch, M.; Khanna, N.; Boushey, C.J.; Delp, E.J. Multilevel segmentation for food classification in dietary assessment. In Proceedings of the 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA), Dubrovnik, Croatia, 4–6 September 2011; pp. 337–342.
  100. Puri, M.; Zhu, Z.; Yu, Q.; Divakaran, A.; Sawhney, H. Recognition and volume estimation of food intake using a mobile device. In Proceedings of the 2009 Workshop on Applications of Computer Vision (WACV), Snowbird, UT, USA, 7–8 December 2009; pp. 1–8.
  101. Shang, J.; Sundara-Rajan, K.; Lindsey, L.; Mamishev, A.; Johnson, E.; Teredesai, A.; Kristal, A. A pervasive Dietary Data Recording System. In Proceedings of the 2011 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), Seattle, WA, USA, 21–25 March 2011; pp. 307–309.
  102. Kong, F.; Tan, J. DietCam: Automatic dietary assessment with mobile camera phones. Pervasive Mob. Comput. 2012, 8, 147–163. [Google Scholar] [CrossRef]
  103. O’Loughlin, G.; Cullen, S.J.; McGoldrick, A.; O’Connor, S.; Blain, R.; O’Malley, S.; Warrington, G.D. Using a Wearable Camera to Increase the Accuracy of Dietary Analysis. Am. J. Prev. Med. 2013, 44, 297–301. [Google Scholar] [CrossRef] [PubMed]
  104. Kikunaga, S.; Tin, T.; Ishibashi, G.; Wang, D.H.; Kira, S. The Application of a Handheld Personal Digital Assistant with Camera and Mobile Phone Card (Wellnavi) to the General Population in a Dietary Survey. J. Nutr. Sci. Vitaminol. 2007, 53, 109–116. [Google Scholar] [CrossRef] [PubMed]
  105. Wang, D.H.; Kogashiwa, M.; Ohta, S.; Kira, S. Validity and Reliability of a Dietary Assessment Method: The Application of a Digital Camera with a Mobile Phone Card Attachment. J. Nutr. Sci. Vitaminol. 2002, 48, 498–504. [Google Scholar] [CrossRef] [PubMed]
  106. Dong, Y.; Hoover, A.; Muth, E. A Device for Detecting and Counting Bites of Food Taken by a Person during Eating. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM ’09), Washington, DC, USA, 1–4 November 2009; pp. 265–268.
  107. Dong, Y.; Scisco, J.; Wilson, M.; Muth, E.; Hoover, A. Detecting Periods of Eating During Free-Living by Tracking Wrist Motion. IEEE J. Biomed. Health Inform. 2014, 18, 1253–1260. [Google Scholar] [CrossRef] [PubMed]
  108. Dong, Y.; Hoover, A.; Scisco, J.; Muth, E. Detecting eating using a wrist mounted device during normal daily activities. In Proceedings of the International Conference on Embedded Systems and Applications, Las Vegas, NV, USA, 18–21 July 2011.
  109. Dong, Y. Tracking Wrist Motion to Detect and Measure the Eating Intake of Free-Living Humans. Ph.D. Thesis, Clemson University, Clemson, SC, USA, 2012. [Google Scholar]
  110. Scisco, J.L.; Muth, E.R.; Hoover, A.W. Examining the Utility of a Bite-Count–Based Measure of Eating Activity in Free-Living Human Beings. J. Acad. Nutr. Diet. 2014, 114, 464–469. [Google Scholar] [CrossRef] [PubMed]
  111. Thomaz, E.; Essa, I.; Abowd, G.D. A Practical Approach for Recognizing Eating Moments with Wrist-mounted Inertial Sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15), Osaka, Japan, 7–11 September 2015; pp. 1029–1040.
  112. Mendi, E.; Ozyavuz, O.; Pekesen, E.; Bayrak, C. Food intake monitoring system for mobile devices. In Proceedings of the 2013 5th IEEE International Workshop on Advances in Sensors and Interfaces (IWASI), Bari, Italy, 13–14 June 2013; pp. 31–33.
  113. Ye, X.; Chen, G.; Gao, Y.; Wang, H.; Cao, Y. Assisting Food Journaling with Automatic Eating Detection. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’16), San Jose, CA, USA, 7–12 May 2016; pp. 3255–3262.
  114. Ye, X.; Chen, G.; Cao, Y. Automatic Eating Detection using head-mount and wrist-worn accelerometers. In Proceedings of the 2015 17th International Conference on E-Health Networking, Application Services (HealthCom), Dalian, China, 12–15 October 2015; pp. 578–581.
  115. Amft, O.; Junker, H.; Troster, G. Detection of eating and drinking arm gestures using inertial body-worn sensors. In Proceedings of the Ninth IEEE International Symposium on Wearable Computers (ISWC’05), Osaka, Japan, 18–21 October 2005; pp. 160–163.
  116. Biallas, M.; Andrushevich, A.; Kistler, R.; Klapproth, A.; Czuszynski, K.; Bujnowski, A. Feasibility Study for Food Intake Tasks Recognition Based on Smart Glasses. J. Med. Imaging Health Inform. 2015, 5, 1688–1694. [Google Scholar] [CrossRef]
  117. Wang, S.; Zhou, G.; Hu, L.; Chen, Z.; Chen, Y. CARE: Chewing activity recognition using noninvasive single axis accelerometer. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, Osaka, Japan, 7–11 September 2015; pp. 109–112.
  118. Eskandari, S. Bite Detection and Differentiation Using Templates of Wrist Motion. Master’s Thesis, Clemson University, Clemson, SC, USA, 2013. [Google Scholar]
  119. Lecluse, F.L.E.; Brocaar, M.P.; Verschuure, J. The Electroglottography and its Relation to Glottal Activity. Folia Phoniatr Logop 1975, 27, 215–224. [Google Scholar] [CrossRef]
  120. Abell, T.L.; Malagelada, J.R. Electrogastrography. Dig. Dis. Sci. 1988, 33, 982–992. [Google Scholar] [CrossRef] [PubMed]
  121. Kamen, G. Electromyographic kinesiology. In Robertson, DGE et al. Research Methods in Biomechanics; Human Kinetics Publ.: Champaign, IL, USA, 2004. [Google Scholar]
  122. Woda, A.; Mishellany, A.; Peyron, M. The regulation of masticatory function and food bolus formation. J. Oral Rehabil. 2006, 33, 840–849. [Google Scholar] [CrossRef] [PubMed]
  123. Woda, A.; Foster, K.; Mishellany, A.; Peyron, M. Adaptation of healthy mastication to factors pertaining to the individual or to the food. Physiol. Behav. 2006, 89, 28–35. [Google Scholar] [CrossRef] [PubMed]
  124. Kohyama, K.; Nakayama, Y.; Yamaguchi, I.; Yamaguchi, M.; Hayakawa, F.; Sasaki, T. Mastication efforts on block and finely cut foods studied by electromyography. Food Qual. Preference 2007, 18, 313–320. [Google Scholar] [CrossRef]
  125. Kohyama, K.; Mioche, L.; Bourdio, P. Influence of age and dental status on chewing behaviour studied by EMG recordings during consumption of various food samples. Gerodontology 2003, 20, 15–23. [Google Scholar] [CrossRef] [PubMed]
  126. Holler, F.J.; Skoog, D.A.; Crouch, S.R. Principles of Instrumental Analysis; Saunders College: Philadelphia, PA, USA, 2007. [Google Scholar]
  127. Farooq, M.; Sazonov, E. Detection of chewing from piezoelectric film sensor signals using ensemble classifiers. In Proceedings of the 2016 IEEE 38th Annual International Conference of Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016; pp. 4929–4932.
  128. Farooq, M.; Sazonov, E. A novel wearable device for food intake and physical activity recognition. Sensors 2016, 16, 1067. [Google Scholar] [CrossRef]
  129. Kalantarian, H.; Alshurafa, N.; Sarrafzadeh, M. A wearable nutrition monitoring system. In Proceedings of the IEEE 2014 11th International Conference on Wearable and Implantable Body Sensor Networks, Zurich, Switzerland, 16–19 June 2014; pp. 75–80.
  130. Sen, S.; Subbaraju, V.; Misra, A.; Balan, R.K.; Lee, Y. The case for smart watch-based diet monitoring. In Proceedings of the 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), St. Louis, MI, USA, 23–27 March 2015; pp. 585–590.
  131. Fontana, J.M.; Farooq, M.; Sazonov, E. Automatic ingestion monitor: A novel wearable device for monitoring of ingestive behavior. IEEE Trans. Biomed. Eng. 2014, 61, 1772–1779. [Google Scholar] [CrossRef] [PubMed]
  132. Dietz, P.; Leigh, D. DiamondTouch: A Multi-user Touch Technology. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST ’01), Orlando, FL, USA, 11–14 November 2001; pp. 219–226.
  133. Cheng, J.; Amft, O.; Lukowicz, P. Active Capacitive Sensing: Exploring a NewWearable Sensing Modality for Activity Recognition. In Pervasive Computing, Proceedings of the 8th International Conference Pervasive 2010, Helsinki, Finland, 17–20 May 2010; Floréen, P., Krüger, A., Spasojevic, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp. 319–336. [Google Scholar]
  134. Bedri, A.; Verlekar, A.; Thomaz, E.; Avva, V.; Starner, T. Detecting Mastication: A Wearable Approach. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle, WA, USA, 9–13 November 2015; pp. 247–250.
  135. Bedri, A.; Verlekar, A.; Thomaz, E.; Avva, V.; Starner, T. A wearable system for detecting eating activities with proximity sensors in the outer ear. In Proceedings of the 2015 ACM International Symposium on Wearable Computers, Osaka, Japan, 7–11 September 2015; pp. 91–92.
  136. Dong, B.; Biswas, S.; Gernhardt, R.; Schlemminger, J. A mobile food intake monitoring system based on breathing signal analysis. In Proceedings of the 8th International Conference on Body Area Networks, Boston, MA, USA, 30 September–2 October 2013; pp. 165–168.
  137. Dong, B.; Biswas, S. Wearable sensing for liquid intake monitoring via apnea detection in breathing signals. Biomed. Eng. Lett. 2014, 4, 378–387. [Google Scholar] [CrossRef]
  138. Zhou, X.; Zhuang, X.; Liu, M.; Tang, H.; Hasegawa-Johnson, M.; Huang, T. HMM-based acoustic event detection with AdaBoost feature selection. In Multimodal Technologies for Perception of Humans; Springer: Berlin/Heidelberg, Germany, 2008; pp. 345–353. [Google Scholar]
  139. Zieger, C. An HMM based system for acoustic event detection. In Multimodal Technologies for Perception of Humans; Springer: Berlin/Heidelberg, Germany, 2008; pp. 338–344. [Google Scholar]
  140. Farooq, M.; Fontana, J.M.; Boateng, A.F.; Mccrory, M.A.; Sazonov, E. A Comparative Study of Food Intake Detection Using Artificial Neural Network and Support Vector Machine. In Proceedings of the 12th International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA, 4–7 December 2013.
  141. Fontana, J.M.; Farooq, M.; Sazonov, E. Estimation of feature importance for food intake detection based on Random Forests classification. In Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 6756–6759.
  142. Duda, R.O.; Hart, P.E.; Stork, D.G. Pattern Classification; John Wiley & Sons: New York, NY, USA, 2012. [Google Scholar]
  143. US Department of Agriculture, ARS. USDA National Nutrient Database for Standard Reference, Release 22; US Department of Agriculture: Washington, DC, USA, 2009.
Figure 1. Major physiologic functions in human cells by non-essential amino acids (NEAA) in particular. Beside those mentioned in the figure, amino acids, as well as NEAA also take part in other functions, such as RNA and DNA synthesis, endocrine status, energy substrates, antioxidative defense, pigmentation and metamorphosis. Adapted from [1].
Figure 1. Major physiologic functions in human cells by non-essential amino acids (NEAA) in particular. Beside those mentioned in the figure, amino acids, as well as NEAA also take part in other functions, such as RNA and DNA synthesis, endocrine status, energy substrates, antioxidative defense, pigmentation and metamorphosis. Adapted from [1].
Computers 06 00004 g001
Figure 2. Obesity prevalence among adults by sex and age in the U.S. Adapted from [33].
Figure 2. Obesity prevalence among adults by sex and age in the U.S. Adapted from [33].
Computers 06 00004 g002
Figure 3. Smart tablecloth’s layers. The overall layout of the system with two protecting sheets to cover the sensing layer; the main sensor hardware part consisting of a carbon polymer fabric sheet, a master FPGA as a controller, a slave FPGA with fast analog switches, 24-bit analog-to-digital converters (ADCs) and individual commercial force sensitive resistors (FSRs). Adapted from [42,44].
Figure 3. Smart tablecloth’s layers. The overall layout of the system with two protecting sheets to cover the sensing layer; the main sensor hardware part consisting of a carbon polymer fabric sheet, a master FPGA as a controller, a slave FPGA with fast analog switches, 24-bit analog-to-digital converters (ADCs) and individual commercial force sensitive resistors (FSRs). Adapted from [42,44].
Computers 06 00004 g003
Figure 4. Overview of the prototype in each sensing approach. (a) Acoustic approach: sensor prototype with ear pad cushion (left) with the attachment of the foam cushion to the sensor (right) (adapted from [49]); (b) Visual approach: eButton system with a camera embedded in a button (left) and the smart phone (right) (adaped from [50]); (c) Inertial approach: MEMS gyroscope prototype on the wristband for wrist motion tracking. This self-contained system includes a microprocessor, battery, gyroscope, LCD, memory and USB port connection (adapted from [51]); (d) EMG/EGG-based approach: EGG sensors are attached to a neoprene collar (left) and the collar is fastened to the neck of the subject by the Velcro (right) (adapted from [52]); (e) Piezoelectric approach: piezoelectric film-embedded necklace to detect muscular contractions during swallowing (adapted from [53]); (f) Fusion approach: an example of the vision-acoustic approach - the microphone is embedded in the housing for detecting chewing/eating events and the camera faces toward the food plate for food detection and classification (adapted from [54]); (g) Electrical proximity sensing: the prototype consists of a textile neckband with 4 capacitive sensors for the detection of single swallows [55]; (h) Respiratory inductance plethysmography: the prototype is worn on the chest to collect the breathing signal and and to transfer it to a smart phone through Bluetooth (adapted from [56]).
Figure 4. Overview of the prototype in each sensing approach. (a) Acoustic approach: sensor prototype with ear pad cushion (left) with the attachment of the foam cushion to the sensor (right) (adapted from [49]); (b) Visual approach: eButton system with a camera embedded in a button (left) and the smart phone (right) (adaped from [50]); (c) Inertial approach: MEMS gyroscope prototype on the wristband for wrist motion tracking. This self-contained system includes a microprocessor, battery, gyroscope, LCD, memory and USB port connection (adapted from [51]); (d) EMG/EGG-based approach: EGG sensors are attached to a neoprene collar (left) and the collar is fastened to the neck of the subject by the Velcro (right) (adapted from [52]); (e) Piezoelectric approach: piezoelectric film-embedded necklace to detect muscular contractions during swallowing (adapted from [53]); (f) Fusion approach: an example of the vision-acoustic approach - the microphone is embedded in the housing for detecting chewing/eating events and the camera faces toward the food plate for food detection and classification (adapted from [54]); (g) Electrical proximity sensing: the prototype consists of a textile neckband with 4 capacitive sensors for the detection of single swallows [55]; (h) Respiratory inductance plethysmography: the prototype is worn on the chest to collect the breathing signal and and to transfer it to a smart phone through Bluetooth (adapted from [56]).
Computers 06 00004 g004
Figure 5. Overview of food intake mechanism classification with corresponded technologies and studies. These will be examined further in Section 5.
Figure 5. Overview of food intake mechanism classification with corresponded technologies and studies. These will be examined further in Section 5.
Computers 06 00004 g005
Figure 6. Application overview of food intake monitoring concerns.
Figure 6. Application overview of food intake monitoring concerns.
Computers 06 00004 g006
Figure 7. Signal processing scheme for chewing and swallowing detection.
Figure 7. Signal processing scheme for chewing and swallowing detection.
Computers 06 00004 g007
Figure 8. Signal processing scheme for food type classifications. Adapted from [77].
Figure 8. Signal processing scheme for food type classifications. Adapted from [77].
Computers 06 00004 g008
Figure 9. Signal processing scheme for food volume estimation.
Figure 9. Signal processing scheme for food volume estimation.
Computers 06 00004 g009
Table 1. Overall comparison among modalities.
Table 1. Overall comparison among modalities.
Body PositionsComfortApplicationsHighest AccuracyAdvantagesChallengesCurrent ApplicabilityObtrusiveness
Acoustic approachNeck/outer ear/inner ear-toward eardrumModerate. Contact with skin sometimes requiredChewing, swallowing events detection85% for swallowing and chewing event detection [80]; 98.5% for food state classification [64]Simple to implement; acceptable accuracy; most suitable for food intake detection, chewing ratesDenoise pre-process needed; accuracy decreased in a noisy environmentMostly still under testing in laboratory conditionObtrusive
Visual approachUpper body (neck, chest, etc.)/external by smart phoneHigh. Attach on clothes/taking pictures with smart phoneVolume estimation, food type classification90.6% for food type classification; 94.3% for volume estimation with large data of images [100]So far the most suitable with the highest accuracy on food type classificationDependent on battery, especially those using the smart phone’s cameraApplicable in real-life scenarios (eButton [50])Unobtrusive
Inertial approachWrist-neck-head-upper and lower armsModerate. Contact with skin requiredBite event detection, food type classification, food amounts89.5% for eating detection [113,114]; 94% on eating gesture detection [115]Able to be embedded in highly wearable items groceries, bracelets, etc.Limited by eating gestures and bites of food; contact makes it uncomfortableReal-life scenarios are still questionable [51]; high accuracy in the laboratory conditionUnobtrusive
EMG/EGG-based approachNeckLow. Contact with skin needed, especially with the neckSwallowing, chewing events detection, bolus size, food hardness86.6% for EGG’s swallowing detection [52]EGG: novel approach, but still has high accuracy; EMG: traditional approach dealing with mastication’s propertiesHighly limited at the neck; EMG sometimes needs exact positions on neck to detect signalLimited to the laboratory scenarioObtrusive
Piezoelectric approachNeckLow. Close contact with skin needed, especially with the neckSwallowing event recognition86% for food states and food type classification (only three kinds of foods) [129]Limited amount of noises in collected data leads to a less complicated frameworkClosed neckband needed for data acquisitionLab-controlled environment onlyObtrusive
Fusion approachFlexibly (ear, wrist, external, etc.)High. Existing methods embedded in comfortable devices (smart watch, ear pad)Swallowing, chewing events detection, food type classification, weight and volume estimation92.0% for eating gesture detection [130]; 85.5% for eating mode detection [131]Combines applications (food types, chewing/swallowing detection); increases accuracy with appropriate framework; high comfort; flexible positionsActual accuracy has not yet met expectationsCan be tested beyond the lab-controlled environment with acceptable accuracy [127]Unobtrusive
Other approach
Electrical proximity sensingNeckHigh. Flexible to implement in different body locationsSwallowing and chewing events recognition95.3% for eating event detection [134,135]Does not need direct contact with the skin; flexible positionsCombination of multiple proximity sensors needed for high accuracyStill in the laboratory circumstanceObtrusive
Respiratory inductance plethysmography sensingWaistModerate. Worn as a beltSwallowing detection88% for swallowing detection [56,137]Does not require contact with skin; utilizes an indirect eating signalAccuracy needs improvingStill limited to the laboratory conditionUnobtrusive

Share and Cite

MDPI and ACS Style

Vu, T.; Lin, F.; Alshurafa, N.; Xu, W. Wearable Food Intake Monitoring Technologies: A Comprehensive Review. Computers 2017, 6, 4. https://doi.org/10.3390/computers6010004

AMA Style

Vu T, Lin F, Alshurafa N, Xu W. Wearable Food Intake Monitoring Technologies: A Comprehensive Review. Computers. 2017; 6(1):4. https://doi.org/10.3390/computers6010004

Chicago/Turabian Style

Vu, Tri, Feng Lin, Nabil Alshurafa, and Wenyao Xu. 2017. "Wearable Food Intake Monitoring Technologies: A Comprehensive Review" Computers 6, no. 1: 4. https://doi.org/10.3390/computers6010004

APA Style

Vu, T., Lin, F., Alshurafa, N., & Xu, W. (2017). Wearable Food Intake Monitoring Technologies: A Comprehensive Review. Computers, 6(1), 4. https://doi.org/10.3390/computers6010004

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop