Filipino Meal Recognition Scale with Food Nutrition Calculation and Smart Application ^†

Abstract

Nutritional awareness is considered prevalent in today’s society. In effect, more people have been inclined to perform food calculations in the food they eat to improve their physical fitness and balance the meals they eat. In this study, the Internet of Things was used through a mobile application with Filipino meal recognition that was integrated into a weighing scale to simplify meal recognition and food calculation without individually scaling and measuring the macronutrients of each food. An ESP32 was programmed to determine the weight of the food sample. Moreover, a TensorFlow Lite Model was created using Teachable Machine, whereas the dataset comprised three Filipino meal combinations of rice, pork adobo, and pork giniling; rice, ginataang kalabasa, and pork giniling; and rice, ginataang kalabasa, and pork adobo. The model identified the 15 samples of Filipino meals per combination. The precision was 91.26% for the first meal combination, 82.73% for the second meal combination, and 85.46% for the third meal combination. One-factor ANOVA was conducted to determine the similarities of the actual and predicted macronutrient contents of the food samples, whereas 10 weight values of successfully determined food meals for each combination were used. The model recognized each Filipino food combination with an overall accuracy of 93.33%. The predicted macronutrient contents were similar to the actual macronutrient contents of the meal based on the statistical analysis performed.

1. Introduction

The Internet of Things (IoT) is a revolutionary technology that goes beyond simple functionality and highlights how deeply it affects people as a whole. Apps demand databases of nutritional data and algorithms to provide users with customized advice for better eating practices. In today’s world, Apps are useful tools for people who want to maximize their nutrition and promote healthier living [1]. System security is essential because the IoT is used more and contributes significantly to the infrastructure of countries.

Security research on cyber-physical and IoT systems is guided by different principles. An “Internet Protocol” must be implemented on even the smallest devices. IoT devices’ ability to process data quickly and efficiently is constrained by their limited computer process unit (CPU), memory, and power [2]. Image processing is capable of solving real-world problems [3] in different applications such as medicine [4], plants [5], and animals [6]. The deep learning framework TensorFlow simulates, trains, and classifies data with an accuracy of up to 90% and yields promising results [7].

Accurate methods for measuring energy and food intake are crucial in treating obesity [8]. By providing users/patients with considerate and useful tools, helping them track their food intake, and compiling nutritional data, the long-term prevention of obesity and successful treatment programs are allowed. The ability to recognize food is crucial for food choices and consumption, both of which are vital for human health and well-being. For this reason, it is significant to facilitate various food-related vision and multimodal tasks such as cross-modal recipe generation and retrieval, food detection, and segmentation [9]. The recommended approach is available on mobile phones, which allow the user to snap a photo of the food and determine its calorie content instantaneously. Deep convolutional neural networks (DCNNs) are used to classify food pictures to accurately recognize the meal. The technology is used to recognize food portions with great accuracy. Through the use of technology, the user places the meal within an enclosure and snaps a single photo for food detection [10]. The technology allows for calculating the food item’s calorie content. After the technology is used to analyze the user’s meal photograph and calculate their caloric intake, the name of the food item and how many calories it contains are displayed [11].

A well-balanced diet is essential for optimal tissue function, illness prevention, and improved health. To provide individualized advice for healthy eating practices, the databases of nutritional data and algorithms are integrated with Apps. The Apps allow for healthier living and are practical tools to optimize their nutrition [12]. The connection between nutrition and health and the relationship between diet and health are becoming important for people [13]. To determine the application of the technology and Apps, we analyzed the efficacy of various detection techniques and instruments and compared their advantages and disadvantages. Food identification systems were also explored concerning food safety and dietary recommendations for different groups of people. We also reviewed the latest developments in smartphone use in the food business to provide a foundation for the expansion of food-related businesses including fast food identification, food source tracking, and customized diets [14].

Based on previous studies, we identified a key research gap in food recognition, traceability, and dietary customization. While prior research was performed to explore the effectiveness of detection techniques and the feasibility of applications, there is a need to study such elements in a single framework to cover the technical aspects of food recognition and the practicality of implementation, user-friendly design, and the provision of dietary advice. Convolutional neural networks (CNNs) were used to deliver the information successfully. However, for efficient training, CNNs need a lot of labeled data, time, and effort to be applied to different food products for food recognition.

Thus, it is necessary to analyze how emerging technologies benefit food-related industries and improve individuals’ dietary choices and overall health. Therefore, we integrated AI-based food recognition into a weighing scale with a mobile application to show the macronutrient content of the food. Using Teachable Machine, a food recognition scale with the integration of TensorFlow, a user-friendly system was developed for accurate food recognition with food nutrition calculation. We created the system to measure the weight of the food combinations using image recognition and developed an intuitive user interface with IoT connectivity to the device to determine the system’s capability by comparing the predicted and the actual macronutrient content of the food combination. The system represents a significant advancement in nutritional tracking technology by combining precise weight measurement using a load cell HX111 with AI. The system helps people adopt healthier eating habits by offering precise and up-to-date information about the nutritional value of the food they eat. By fusing IoT connectivity with user-centric design, the system provides nutrition tracking and becomes more convenient for users. The system can be applied to various industries. Enhanced quality control and traceability are allowed to ensure compliance with safety and nutritional standards for processed foods and agricultural products. Consumers can also access real-time nutritional data for educated dietary choices.

2. Materials and Methods

2.1. Conceptual Framework

In the system, images are captured as the dataset of the macronutrient content of food samples for providing calorie intake for the user. Foods and combinations are classified by utilizing the captured images in the dataset, and nutrition is calculated to predict the macronutrient content and calories. On the device and the mobile device, the weight, total calories, macronutrient content, total calorie intake, and needed calories are displayed (Figure 1).

2.2. Process Flowchart

In the system, the first process is logging in followed by the main menu to input the user profile including height, weight, age, gender, activity level, and target weight. Calculation is conducted using the food calculator menu based on foods captured. Macronutrient content is calculated based on the database of food samples and their weights. Foods and calories are deducted from the total calories needed for the day (Figure 2).

2.3. Hardware Development

In the system, a 12 V, 5 A, AC/DC power supply adapter was connected to the HW688 module to convert 12 to 5 V. Since the ring light and the ESP32 need 5 V, the HW688 supplies the right voltage needed. Components were connected to the ESP32 and the 16 × 2 I2C LCD to show the weight by utilizing the HX711 load amplifier (Figure 3, Figure 4 and Figure 5).

2.4. Software Development

The App was created using MIT App Inventor. The App includes a login page to create login credentials and the main menu for food calculation and a user profile where the user inputs their data for the computation of total calories needed per day. It also includes an interface for the weight output of the classified food combination (Figure 6).

2.5. Training and Food Nutrition Calculation

In the recognition model, the dataset or images of the sample are gathered. Figure 7 shows the different samples of food combinations to create the different food classification models.

Figure 8 shows the interface of the Teachable Machine with the different food combinations. The Teachable Machine was utilized to train a TensorFlow model with the dataset and identify food combinations to determine the macronutrient content of the food combination. To calculate the actual calorie content per food sample, the calories of each food sample are multiplied by its conversion factor which is as follows.

$${TC}_A={FV}_1\left({\displaystyle \frac{{TC}_1}{S1}}\right)+{FV}_2\left({\displaystyle \frac{{TC}_2}{S2}}\right)+R\left({\displaystyle \frac{{TC}_3}{S3}}\right)$$

(1)

where TC_A is the total actual calories of the food combination. FV₁ and FV₂ are the two Filipino viands on the plate, R is for Rice, TC₁ to TC₃ are the total calories in accordance with the food sample, and S₁ to S₃ are its serving size.

For predicting the total caloric and macronutrient content of the food combinations, the sample mean of each macronutrient was approximated. The total calories and macronutrient content for each parameter (AM_t) per serving size are equal to the sum of those of the listed food samples of the food combinations. The formula for the predicted caloric and macronutrient content per serving size is as follows.

$${AM}_T={\displaystyle \frac{{FV}_1+{FV}_2+R}{3}}$$

(2)

After gathering the total caloric content of the food combination per serving size AM_T, it is divided by the serving size (S) before multiplying by the total sum of each macronutrient content in grams of each retrieved food sample RF₁, RF₂, and RR in the food combination.

$${TC}_P={\displaystyle \frac{{AM}_T}{S}}({RFV}_1+{RFV}_2+RR)$$

(3)

The macronutrient count is presented in

Table 1. Nutritional content of chosen Filipino foods.

Sample Menu	Quantity	Food Group	Carbohydrates (G)	Protein (G)	Fat (G)	Energy (Kcal)
Pork Adobo
Pork Tenderloin	2 slices (70 g)	Low-Fat Meat	-	16	2	82
Potato	½ pc (85 g)	Rice B (Medium Protein)	11.5	1	-	50
Cooking oil	2 tsp (10 g)	Fat	-	-	10	90
		TOTAL	11.5 g	17 g	12 g	222 kcal
Pork Giniling
Cooking oil	2 tsp (10 g)	Fat	-	-	10	90
Lean ground pork	1 slice (35 g)	Low-Fat Meat	-	8	1	41
Chicken egg	1 pc medium (55 g)	Medium-Fat Meat	-	8	6	86
Potato	½ pc (85 g)	Rice B (Medium Protein)	11.5	1	-	50
Carrots	1 cup (90 g)	Vegetable	6	2	-	32
Green peas
Cooking oil	1 tsp (5 g)	Fat	-	-	5	45
		TOTAL	17.5 g	19 g	12 g	254 kcal
Ginataang Kalabasa
Pork tenderloin	1 slice (35 g)	Low-Fat Meat	-	8	1	41
Squash	1 cup (90 g)	Vegetable	6	2	-	32
String beans
Coconut cream	3 Tbsp (45 g)	Fat	-	-	15	135
Cooking oil	1 tsp (5 g)	Fat	-	-	5	45
		TOTAL	6 g	10 g	21 g	253 kcal
Rice
White Rice	1 cup (160 g)	Rice B (Medium Protein)	46	4	-	200
		TOTAL	46 g	4 g	0 g	200 kcal

(Computed based on the values provided by the DOST-FNRI Food Exchanges Lists for Meal Planning).

. The information on the macronutrients is based on the values provided by the DOST-NFRI Food Exchange List for Meal Planning.

3. Results and Discussion

Foods were classified and compared between the actual and predicted calories of the food combinations. FC1, FC2, and FC3 denote the food combinations of rice, ginataang kalabasa, and giniling. The average precision score for the food classification was calculated in the mobile application using the TensorFlow Lite model. For FC1, FC2, and FC3, the average precision score was 91.27, 82.73, and 85.47% (

Table 2. Average precision score of each food combination.

Number of Trials	Food Sample	Average Precision Score (%)
15	FC1	91.26666667
15	FC2	82.73333333
15	FC3	85.46666667

). The confusion matrix was constructed using 15 trials. For FC1, all were identified correctly, while for FC2 and FC3, one and two samples were misidentified. The computed overall accuracy was 93.33% (

Table 3. Confusion matrix.

	FC1	FC2	FC3	Classification Overall
FC1	15	0	0	15
FC2	0	13	2	15
FC3	0	1	14	15
Truth Overall	15	14	16	45

).

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\mathrm{T}\mathrm{P}/\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}$$

(4)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\mathrm{T}\mathrm{P}/\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}$$

(5)

$$\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N}}}\times 100\mathrm{\%}$$

(6)

$$\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=93.33\%$$

(7)

Table 4. Macronutrient content of FC1.

Food Combo	Actual					Predicted
Trial	Giniling (g)	Adobo (g)	Rice (g)	Sum (g)	Total Calories	Total Calories
1	37	47	125	209	254.29	246.35
2	54	56	88	198	236.15	233.39
3	84	52	123	259	302.74	305.29
4	84	63	78	225	261.29	265.21
5	95	55	106	256	295.87	301.76
6	60	50	93	203	239.97	239.28
7	57	62	105	224	268.29	264.04
8	94	80	67	241	279.82	284.07
9	81	69	87	237	277.79	279.36
10	86	57	99	242	281.34	285.25
	Average:				269.75	270.40

shows the different weights of FC1. The total calories of the sample were determined, too. For predicted total calories, the sum of the weights of each food sample was used and the computed macronutrient content per gram was used. The average actual total calories of FC1 was 269.75 kcal, and the average predicted total calories was 270.40 kcal. The same process was conducted for FC2 with different values because each food sample had a different macronutrient content. The average total calories was 319.95 kcal. The average predicted total calories was 323.70 kcal (

Table 5. Macronutrient content of FC2.

Food Combo 2	Actual					Predicted
Trial	Ginataang Kalabasa (g)	Adobo (g)	Rice (g)	Sum (g)	Total Calories	Total Calories
1	85	42	107	234	313.14	315.21
2	48	76	103	227	300.40	305.78
3	87	61	103	251	336.60	338.11
4	40	85	115	240	315.94	323.29
5	91	97	92	280	377.07	377.18
6	79	71	89	239	320.99	321.95
7	58	44	128	230	303.05	309.82
8	50	81	93	224	297.52	301.74
9	45	82	111	238	314.13	320.60
10	67	79	94	240	320.65	323.29
	Average:				319.95	323.70

). The average actual total calories of FC3 was 284.79 kcal, and the average predicted total calories was 280.98 kcal (

Table 6. Macronutrient content of FC3.

Food Combo 2	Actual					Predicted
Trial	Ginataang Kalabasa (g)	Giniling (g)	Rice (g)	Sum (g)	Total Calories	Total Calories
1	85	42	107	234	313.14	315.21
2	48	76	103	227	300.40	305.78
3	87	61	103	251	336.60	338.11
4	40	85	115	240	315.94	323.29
5	91	97	92	280	377.07	377.18
6	79	71	89	239	320.99	321.95
7	58	44	128	230	303.05	309.82
8	50	81	93	224	297.52	301.74
9	45	82	111	238	314.13	320.60
10	67	79	94	240	320.65	323.29
	Average:				319.95	323.70

).

4. Conclusions

We developed a system for food classification using image classification and machine learning. A working prototype was fabricated, and the mobile application was developed to determine nutritional information on foods for a healthier lifestyle. The potential of image processing was confirmed in determining the macronutrient content of selected foods and meals. Nutrition and health monitoring using mobile applications can be further developed with technological advancements.