Infield Apple Detection and Grading Based on Multi-Feature Fusion

Abstract

A field-based apple detection and grading device was developed and used to detect and grade apples in the field using a deep learning framework. Four features were selected for apple grading, namely, size, color, shape, and surface defects, and detection algorithms were designed to discriminate between the four features using machine vision and other methods. Then, the four apple features were fused, and a support vector machine (SVM) was used for infield apple grading into three grades: first-grade fruit, second-grade fruit, and other-grade fruit. The results showed that for a single index, the accuracy of detecting the apple size, the fruit shape, the color, and the surface defects, were 99.04%, 97.71%, 98%, and 95.85%. The grading accuracies for the first-grade fruit, second-grade fruit, other-grade fruit, and the average grading accuracy based on multiple features were 94.55%, 95.71%, 100%, and 95.49%, respectively. The field experiment showed that the average grading accuracy was 94.12% when the feeding interval of the apples was less than 1.5 s and the walking speed did not exceed 0.5 m/s, meeting the accuracy requirements of field-based apple grading.

1. Introduction

Fruit grading is crucial for fruit marketing and is directly related to the effects of fruit packaging, transportation, storage, and sales [1,2,3,4]. The external and internal quality of fruits are important factors affecting market price and customer satisfaction [5,6,7,8]. When people buy apples in the market, they typically evaluate the quality of apples based on exterior features related to fruit appearance, such as size, color, shape, and surface defects [6,9,10,11]. Therefore, grading apples according to their appearance is an important indicator to improve the market value of apples.

Apple field grading is essential to improve the economic value of the fruit and significantly affects the industrial chain of apple production [1]. Infield grading of apples provides farmers with fruit with a known status grade, and therefore improves the economic benefits [2,12,13,14]. In addition, the elimination of apples with surface defects can help to reduce the cross-infection between diseased apples during storage and transportation [15]. The downstream production enterprises can adopt targeted storage, processing, grading, and other processes according to the field grading results of the apples to increase the competitiveness of enterprises. Apple field grading refers to grading freshly harvested apples in orchards. The environment in apple field grading is more complex than in industrial grading and is easily affected by vibration and other factors [16].

Machine vision has achieved good results for grading agricultural products, such as apples, tomatoes, potatoes, and mangoes [17,18,19,20,21,22,23,24,25]. Pourdarbani et al. developed a jujube sorting system based on machine vision. The processing time was 0.34 s per image and 15.45 kg of jujube per hour [26]. Kumar et al. designed and developed a nondestructive grading method for pomegranate fruit based on wavelet transform and an artificial neural network [27]. The accuracy rate for distinguishing between unblemished and blemished pomegranates was 91.3%. Chiu et al. developed a system for the online detection of apple damage [13]. The system used a chlorophyll fluorescence image to detect the crush injury of apples and could process 92 apples per hour. Although the detection accuracy of crush injury was high, the detection speed was relatively low. In order to improve the accuracy and the detection speed for apple grading, Hang and Fei presented an online detection method of apple grading based on machine vision features [28]. The accuracy of apple grading was 95%, and the average grading rate was four apples per second. Zhang et al. designed an online grading system integrated with an apple-picking robot using machine vision to detect the apple size and rotten areas [2]. The grading accuracy was 89.71%, and the grading time was 2.89 s per apple. Sadegaonkar et al. established a roller conveyor belt to detect moving and rotating apples and sort apples automatically according to their color, size, weight, and maturity [29]. Vakilian et al. designed a system based on image texture feature extraction and used an artificial neural network to classify apples [30]. The detection accuracy of golden crown apples and snake fruits was 89% and 92%, respectively. Bhatt and Pant proposed an apple grading system based on machine vision and an artificial neural network (ANN), and developed specific hardware and software systems to automate apple grading [31]. The accuracy of automatic apple grading was 96%. Existing apple grading equipment required a stable environment and was relatively bulky and not suitable for grading fruits in the field.

Infield apple grading is mainly based on the appearance of apples, including the color, shape, size, and surface defects. A single characteristic does not adequately reflect the quality of apples. In this study, a field-based apple detection and grading device was developed, and Fuji apples were analyzed. The contribution of this study was: (1) the rapid detection of apple surface defects by the single-shot multibox detector (SSD); (2) a support vector machine (SVM) and multi-feature fusion of the apples’ features were used to detect various surface defects in fruits, so as to perform infield apple grading.

2. Materials and Methods

2.1. The Apple Field Grading Equipment

The apple field grading equipment consisted of a guidance mechanism, a conveying device, a detection device, an executing mechanism, and a collection device, as shown in Figure 1a. The guidance mechanism consisted of a guiding baffle and a direction-adjusting brush. The conveying device consisted of four parallel conveyor belts and motors. The detection device consisted of a detection box, industrial cameras, a photoelectric sensor, an illumination source, and a computer. The actuating device included a steering gear, actuating push plate, and other parts, and the collection device consisted of a collection pipeline and a collection box.

The detection device was the core part of the field grading equipment and was used to extract the apple diameter, shape, color, and surface defects, which were used to grade the apples. The detection device included a closed cuboid box with a size of 400 × 400 × 500 mm³ and industrial cameras (MV-UBS31GC, Medway Vision Technology Co., Ltd.) with a resolution of 720 × 480 pixels and a frame rate of 108 fps. The cameras were installed on the top and on both sides of the detection box, as shown in Figure 1b. The three cameras ensured that the two side and top images of the apple could be taken for feature detection, but the area of contact between the apple and the conveyor belt could not be captured by the camera. The industrial cameras were calibrated using Zhang Zhengyou’s calibration method [32]. Two LED strips were placed at the top of the box for stable and bright lighting.

The harvested apples entered one end of the guidance mechanism (Figure 1a) and were sent to the detection device by the conveyor belt for detection and grading. The results were sent to the actuating device, which pushed the apples of different grades into different collection pipelines to complete the field grading task. The workflow is depicted in Figure 2.

2.2. Apple Grading Criteria and Feature Extraction

We used the standard of fresh apples (GB/T 10651-2008, General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China) and conducted market research to determine the appropriate grading standard. The standard is listed in

Table 1. Standard for apple grading.

Grade	Size/mm	Color/%	Shape	Surface Defect
First grade	≥75	≥80	Good	None
Second grade	60~75	≥55	Slight shape distortion	Mild
Other grades	<60	<55	Strong shape distortion	Severe

. The final grade of the apples is based on the lowest grade of a single characteristic, such as the size, color, shape, and surface defect of the apples. Three levels were used: first-grade, second-grade, and other-grade fruit. The apple size was the maximum cross-sectional diameter of the apple. The apple color was characterized by the ratio of the red area to the total apple surface area.

2.2.1. Extraction of the Apple Size

In this research, the minimum circumscribed circle of the apple was used to calculate the maximum cross-sectional diameter of the apple in the image obtained from the top camera to extract the apple size, which was the same method used by some scholars [33]. First, the edge pixels of a binary apple image were detected, their coordinates were determined, and the centroid coordinates of the circumference of the apple were obtained based on the coordinates of the edge pixels. Then, the distance from the centroid to the edge of the apple was calculated, representing the diameter of the minimum circumscribed circle.

2.2.2. Extraction of the Fruit Shape

A regular shape is an important index in apple grading and also influences the consumer. Deformed apples usually sell at a lower price. In this study, the roundness and shape index were used to evaluate the apple shape.

The roundness describes the complexity of an object’s boundary. The range of the roundness is [0, 1], and the closer the value is to 1, the rounder the shape is. The roundness E₁ was calculated as follows:

$$E_1=\frac{4\pi S}{P^2}$$

(1)

where S is the apple area, i.e., the number of projected pixels on the apple’s surface; P is the perimeter of the apple, i.e., the number of pixels on the apple’s circumference.

The shape index E₂ is the ratio of the long axis to the short axis of the apple:

$$E_2=\frac{D_1}{D_2}$$

(2)

where D₁ is the long axis of the apple, i.e., the maximum length from the bottom of the calyx to the stem of the apple, mm; D₂ is the short axis of the apple, i.e., the maximum cross-sectional diameter of the apple, mm. When the shape index has a range of 0.6–0.8, the apple is oblate, 0.8–0.9 is round or nearly round, 0.9–1.0 is oval or conical, and 1.0 or more is oblong [6].

2.2.3. Extraction of the Apple Color

The apple’s color is a visual characteristic and an indicator of the apple’s quality and maturity. Therefore, the color of apples is crucial in apple grading. The hue, saturation, and value (HSV) color space model was adopted in this study since it is a robust color model.

The HSV color space model separates the hue, saturation, and value (brightness) of the apple’s color. The hue is least affected by illumination and most suitable for distinguishing different apple colors. Therefore, we selected the hue of the HSV image to extract the apple’s color. In this research, we used OpenCV to convert the collected apple image into the HSV space, and divided the value of hue by 2 to obtain the range of hue from 0 to 180. The red, yellow, and background components were extracted from the hue of the side view image of the apple for the statistical analysis, as shown in Figure 3.

The hue of the background was substantially different from that of the red parts of the apple. The hue ranges of the red parts were [0, 15] and [175, 180] (as shown in Figure 3a), the hue ranges of the background parts were [50, 125] (as shown in Figure 3c), and those of the yellow components were about [20, 25] (as shown in Figure 3b). Thus, the ratio of the red area was used to describe the color of the apple. The ratio of the red area is the ratio of the number of red pixels in the hue to the total number of pixels in the apple area:

$$H_r=\frac{{\displaystyle {\sum }_{i=1}^N{\mathrm{Mask}}_i}}{M}$$

(3)

where M is the total number of apple pixels in the camera field of view; Mask_i is set to 1 when the hue value of the ith pixel is within the red threshold range; otherwise, it is 0. The red threshold ranges from [0, 15] to [175, 180]. The red area ratios of the hue values of the apple images obtained by both cameras were calculated and averaged to describe the color of the apple. The value ranges from 0 to 1, and the closer it is to 1, the redder the apple is.

2.2.4. Extraction of the Fruit Surface Defects

Surface defects are the most critical factors in apple grading. During growth and harvest, the apples are affected by friction, squeezing, and insect pests, resulting in physical damage. Damaged apples are prone to mildew and decay, which affect the taste and commercial value of the apples and can reduce the storage capacity of entire batches of apples.

Apple surface defects include crush injury, stab injury, abrasion injury, sunburn, hail injury, cracks, splits, insect damage, and so on. Due to the low probability of some fruit surface defects, such as sunburn and hail injury, it is necessary to determine the proportion of different types of detected fruit surface defects. In order to obtain accurate results, we collected 1000 Fuji apples from the Huicheng Orchard (34.31° N, 108.02° E) in Yangling, Shaanxi Province and counted the surface defects. The statistical results are listed in

Table 2. Statistical results of apple surface defects (number of apples).

Total Number of Apples	No Defects	Bruising	Cracks and Splits	Damage Caused by Farm Chemicals	Fruit Rust	Sunburn	Other Defects
1000	878	52	21	15	9	5	20

.

It can be seen from the statistical results in Table 2 that the ratio of bruising and cracks is high, which was due to the physical damage caused by falling or colliding apples during picking. The bruises were divided into two categories: mild bruises, referred to as bruises, and severe bruises. Slight bruises were similar in color to the surrounding epidermis, without obvious discoloration. Severe bruising was caused by severe impact and pressure, and even juice flowing out. Due to the fact that the types of fruit surface defects described by cracks and splits were basically the same, the two types of cracks and splits were combined into cracks. Other surface defects were relatively few, among which damage caused by farm chemicals was the most, and other kinds of fruit surface defects were collectively referred to as skin defects.

In this study, the TensorFlow deep learning framework with the single-shot multibox detector (SSD) deep learning algorithm was used to identify the apple surface defects to ensure high accuracy of the surface defect detection of the field grading equipment. SSD is a target detection method [34,35]. The algorithm changes the two fully connected layers of the VGG-16 network structure into convolution layers, and adds four convolution layers to construct the network structure. The specific network structure is shown in Figure 4.

The overall objective loss function is a weighted sum of the localization loss (L_loc) and the confidence loss (L_conf):

$$L\left(x,c,l,g\right)=\frac{1}{N}\left(L_{conf}\left(x,c\right)+\alpha L_{loc}\left(x,l,g\right)\right)$$

(4)

where N is the number of matched default boxes, x indicates whether the prediction box matches the real label box, if so, it is 1, otherwise it is 0; c represents the confidence of softmax function for each classes; l represents the prediction box; g represents the real label box and the weight term α is set to 1 by cross-validation [35].

The images of the apples used in the training set were manually labeled using the LabelImg toolbox. The labels included no defect, bruising, severe bruising, cracks, and skin defects. Due to the small number of samples, transfer learning was used to optimize the parameters. We used the trained parameters of the MobileNetV2 classification network, removed the last classification layer, kept the parameters of the VGG5 layer unchanged, and randomly initialized the parameters of the other layers using a Gaussian distribution with 0 mean and 0.01 standard deviation. A batch random gradient descent algorithm was used for parameter optimization. The batch size was 64, the initial learning rate was 0.04, and the 800 images with a size of 3024 × 3024 pixels were scaled to 300 × 300 pixels. After training, the model was applied to the test set to detect surface defects in the apples. Rectangular boxes with scores exceeding 50% represented the areas with surface defects.

To verify the model accuracy, the apples in the image were checked, and the apple damage in the test set images was manually marked. An intersection over union (IOU) was performed to obtain the overlap between the predicted results obtained from the SSD deep learning algorithm and the actual results. According to the previous test, it can be considered that the IOU is 70% with high accuracy, that is, if the IOU is more than 70%, it is considered that the detection of the fruit surface defect type is correct, otherwise, it is considered that the detection is incorrect. The evaluation indices of detection accuracy are recognition precision (P), recall (R), and harmonic mean F₁, and the formula is as follows:

$$P=\frac{T_P}{T_P+F_P}\times 100\%$$

(5)

$$R=\frac{T_P}{T_P+F_N}\times 100\%$$

(6)

$$F_1=\frac{2PR}{P+R}\times 100\%$$

(7)

where T_P is the number of surface defects detected correctly; F_P is the number of non-surface defect areas mistakenly detected as surface defect areas or surface defect detection errors; F_N is the number of surface defect areas mistakenly detected as non-surface defect areas; F₁ is the harmonic mean of P and R, and the closer it is to 1, the better the model is.

2.3. Apple Grading Method Based on Feature Fusion

It is one-sided to only rely on a single feature to classify apples. When experienced fruit farmers grade apples, they comprehensively evaluate the advantages and disadvantages of apples according to their size, color, shape, and surface defects. Apple grading in the field requires fusing multiple related features to obtain a more accurate grade. Therefore, this study integrated four physical characteristics of apples, namely, size, color, shape, and surface defects. Five parameters of the four physical characteristics were used to evaluate and grade apples comprehensively. The five parameters include the fruit diameter for evaluating the fruit size, the red area ratio for evaluating the color, the roundness and fruit shape index for evaluating the fruit shape, and the surface defect types.

The apple diameter, red area ratio, roundness, and shape index were used directly as input features, and the degree of influence of the surface defects on apple grading was independently calibrated for the different categories. The extracted features were combined into a multi-feature high-dimensional vector X_i = [size, color, roundness, shape index, surface defects] of the apple. The actual grades of the apples were determined manually by experienced fruit farmers according to the grading standards listed in Table 1.

Obtaining the grade can be regarded as a process of self-learning using the training samples {(X_i, Y_i): i = 1, 2,…, n}, where X_i is the feature vector, and Y_i ∈ {±1} is the class label of the ith training sample. The optional boundary is defined by Equation (8):

$$f\left(x_i\right)=\omega x_i+b$$

(8)

The problem of defining ω and b values can be transformed into a convex optimization problem expressed by Equation (9):

$$\min (\frac{1}{2}{\left|\omega \right|}^2+\sigma {\displaystyle \sum {\epsilon }_i})$$

(9)

where σ is the penalty coefficient, ε_i is the error, and lab_i(ωX_i + b) ≥ 1 − ε_i.

A multi-class support vector machine (MSVM) was used to grade the apples to ensure high accuracy [17]. The MSVM is a specific application of the SVM that assigns one of many class labels to the input [22]. The multi-feature high-dimensional vector was used as the independent variable, and the real grade was used as the dependent variable to construct a classification model. In the MSVM classifier, the type of kernel function and the setting of the parameters have a crucial impact on the classifier performance. In this research, the radial basis function (RBF) was used as the kernel function of the SVM, and the optimal penalty coefficient and kernel function parameters of the grading model were obtained by a grid optimization algorithm and triple cross-validation.

2.4. Field Test

The equipment was transported to the Huicheng Orchard in November 2019 for a field experiment to verify the accuracy of the infield grading equipment and multi-feature fusion algorithm. A movable cable tray was used to supply power to the grading equipment in the field; the voltage was 220 V. The detection box was protected by a black cover to prevent light infiltration and improve the quality of image acquisition. The pickers placed the harvested apples onto the conveyor belt for grading. The speed of the conveyor belt was 0.4 m/s, and the grading equipment was manually controlled to operate in a straight line between the fruit trees, as shown in Figure 5.

3. Results and Discussion

3.1. Results of Apple Size Detection

The apple size obtained from the algorithm was compared with the manual measurements of 10 apple samples to determine the accuracy of the minimum circumscribed circle method for obtaining the apple diameter. The manual method consisted of measuring the maximum cross-sectional diameter of the apple with a vernier caliper. The comparison results are listed in

Table 3. Comparison of the measured and image-based results of the apple size.

Sample Number	Diameter Obtained from the Image/mm	Manually Measured Diameter/mm	Accuracy/%
1	75.6	76.3	99.37
2	79.3	78.8	99.51
3	82.0	82.4	99.42
4	68.9	69.3	99.19
5	73.6	74.2	98.97
6	88.4	87.5	98.13
7	73.4	74.8	98.84
8	78.5	77.6	98.77
9	80.0	81.0	99.16
10	83.7	83.0	99.04

.

The average accuracy of the algorithm was 99.04%, the maximum absolute error between the manual measurements and image measurements was 1.4 mm, the average absolute error was 0.75 mm, the maximum relative error was 1.8%, and the average relative error was 0.96%. The results indicate that this method can accurately extract the fruit size.

3.2. Results of Apple Shape Detection

The results of the apple edge extraction are shown in Figure 6. Figure 6c depicts the result of superimposing the fitting curve on the original image, indicating a good fit of the apple contour. The number of pixels in the apple area was 15,457, and that of the perimeter of the apple was 465. Therefore, the roundness of the sample was 0.90.

The side view image of the apple was transformed into the HSV color space, and the image was binarized, as shown in Figure 7. The distance between the top and bottom of the apple is the length of the long axis of the apple.

The shape index extraction results of 10 apple samples were compared with manual measurements, as listed in

Table 4. Comparison of the measured and image-based results of the shape index.

Sample Number	Shape Index Obtained from the Image	Shape Index Based on Measurements	Accuracy/%
1	0.95	0.93	97.85
2	0.83	0.83	100.00
3	0.99	0.95	95.79
4	0.87	0.90	96.67
5	0.85	0.86	98.84
6	0.90	0.89	98.88
7	1.01	0.97	95.88
8	0.94	0.92	97.83
9	0.92	0.89	96.63
10	0.79	0.80	98.75

. The average accuracy of the shape index obtained from the image was 97.71%, the maximum relative error was 3.3%, and the average relative error was 2.3%. The experimental results show that the image-based method of obtaining the apple shape index has a high accuracy, meeting the grading requirements.

3.3. Results of Apple Color Detection

The red area ratio is used as a single standard for grading to determine if the apple color obtained from the image can be used for grading. A red area ratio of more than 80% indicates first-grade fruit, more than 55% is second-grade fruit, and less than 55% is other-grade fruit. We selected 100 mature apples randomly from the orchard, and experienced fruit farmers were used as appraisers to grade the apples based only on the color. The accuracy of the manual grading method was compared with the grading results obtained from the red area ratio. The results are listed in

Table 5. Comparison of the manual grading and image-based grading results based on the apple color (number of apples).

Color Grade	Manual Grading	Correct Image-Based Grading	Errors
First grade	63	61	2
Second grade	26	26	0
Other grades	11	11	0

.

Out of 100 apples, 98 apples were graded correctly; thus, the grading accuracy was 98% based on using only the color index. The experiment shows that it is feasible to identify apple color using the red parts of the hue.

3.4. Results of Surface Defect Detection

We used 200 images in the test set to obtain the surface defects and evaluate the detection accuracy of the deep learning detection algorithm based on SSD. We inputted the 200 images of the test set into the trained network, marked the different types of surface defects with different color identification boxes, and labeled them. Some apples had multiple types of surface defects. The identification results are shown in Figure 8.

It can be seen in Figure 8 that SSD can accurately detect one type of surface defect of a single apple, and can also detect two types of surface defects of a single apple at the same time. The more fruit surface defects, the worse the quality of the fruit, which is conducive to improving the accuracy of apple infield grading.

The highest accuracy is obtained for an IOU threshold of 70%. Thus, the results are considered correct when the value exceeds the threshold and incorrect when the value is below the threshold. The surface defects of the tested apple images were counted. T_P was 208, F_P was 9, and F_N was 20; therefore, equal to the precision (P) was 95.85%, the recall (R) was 91.23%, and the harmonic mean F1 was 93.48%, indicating that the detection accuracy of surface defects was high.

3.5. Results of Multi-Feature Detection

In this study, 510 apple samples were collected and used for grading. The size, red area ratio, roundness, shape index, and surface defects were extracted from the top view and side view images of the apples, and the apples were classified into three grades using the SVM.

Table 6. Statistical results of the features of the apple samples.

Features	Minimum	Maximum	Mean ± SD
Fruit diameter (mm)	62.33	93.96	79.98 ± 5.15
Red area ratio	0.63	0.96	0.83 ± 0.04
Roundness	0.83	0.97	0.90 ± 0.02
Fruit shape index	0.68	1.01	0.86 ± 0.04

lists the statistical results of the apple diameter, red area ratio, roundness, and shape index.

The manual grading by the fruit farmers resulted in 202 first-grade fruits, 280 second-grade fruits, and 28 other-grade fruits from the 510 samples. The classification results of the SVM training model are listed in

Table 7. Manual and image-based grading (SVM) results of the apples using multiple features.

Grade of Apple	Correct Image-Based Grading (Number)	Manual Grading (Number)	Error (Number)	Accuracy (%)
First grade	191	202	11	94.55
Second grade	268	280	12	95.71
Other grades	28	28	0	100
Total	487	510	23	95.49

. The accuracy of the model was 95.49%, meeting the accuracy requirements of apple field grading. The algorithm’s performance was the highest for the other-grade fruit (100%), and there were a few classification errors for the first-grade fruit and second-grade fruit. Through the analysis of the fruits with errors, it is found that the reasons for grading error are that (1) the area of contact between the fruit and the conveyor belt could not be captured by the camera, and the fruit characteristics could not be extracted. The part of the fruit in contact with the conveyor belt had some defects, which were not easily detected by the image, leading to apple grading errors. (2) Slight bruises were not detected, resulting in the second grade apple being wrongly classified into the first grade, affecting the apple grading accuracy.

3.6. Field Test Results

When the feeding speed was high or the apples moved rapidly, the algorithm became unstable, resulting in a strong image blur, deviation of the apples from the central channel, and other problems that affect the accuracy of apple feature extraction. The apple detection and extraction device operated stably and accurately graded the apples when the feeding interval of the apples was less than 1.5 s, and the walking speed did not exceed 0.5 m/s. The field-based classification efficiency was about 40 apples/min. We used 170 apple samples to test the classification accuracy of the SVM model. The manual grading indicated 74 first-grade fruits, 87 second-grade fruits, and 9 other-grade fruits. The classification results of the SVM model in the field test are listed in

Table 8. Manual and image-based grading results (SVM) of the apples using multiple features in the field test.

Grade of Apple	Correct Image-Based Grading (Number)	Manual Grading (Number)	Error (Number)	Accuracy (%)
First grade	69	74	5	93.24
Second grade	82	87	5	94.25
Other grades	9	9	0	100.00
Total	160	170	10	94.12

.

In this study, the causes of the 10 wrongly grading apples were analyzed. It was found that the grading equipment will vibrate when driving on the uneven ground of the orchard. The irregular vibration of grading equipment leads to (1) a blurred image of the apple, and (2) the failure of fruit to stand upright on the conveyor belt. This leads to a decline in the accuracy of fruit size, roundness, and surface defect detection, which affects the grading accuracy of apples. With regard to the motion blur of the apple image caused by the driving of grading equipment on the uneven ground, we have begun to use the DeblurGAN method to deblur the apple images, so as to improve the accuracy of apple features extraction in the field.

In this research, grading equipment developed in different literature listed in

Table 9. Comparison of features and efficiency for performing apple grading.

Features	Accuracy of Grading	Grading Rate	Working Conditions	Reference
Size, defeat area, fruit shape, color degree, texture, and color distribution	95%	4 apples/s	-	[28]
Size, color, external defects, and weight	96%	-	Indoor	[31]
External defects, size, and soluble solids of apple	94%	0.71 s per apple	Indoor	[8]
Textural: image’s luminous intensity and contrast	92%	-	-	[30]
Size, weight, rot area	89.71%	2.89 s per apple	Infield	[2]
Size, color, shape, and surface defects	94.12%	1.5 s per apple	Infield	This study

Note: -, not specified.

are compared. The indoor environment has stable illumination and vibration. In a stable indoor environment, high grading accuracy and rate are achieved. The field grading is affected by complex factors such as vibration, which result in low grading accuracy and efficiency. Compared with reference [2], the average grading accuracy and rate of the grading equipment in this study are 94% and 1.5 s per apple respectively. Therefore, the SVM with the fused features provides good performance and meets the accuracy requirements for field grading apples.

4. Conclusions

In this research, field-based apple grading equipment was designed and tested. The accuracy of the method for detecting four apple features was tested. The performance of infield apple grading using fused features was evaluated. More specifically:

(1) Four physical characteristics (apple diameter, color, shape, and surface defects) were extracted for apple grading, and their detection accuracies were 99.04%, 97.71%, 98.00%, and 95.85%, respectively.

(2) The SVM algorithm was used to integrate these four features for apple grading. The accuracy of apple grading was 94.55%, 95.71%, 100%, and 95.49% for first-grade fruit, second-grade fruit, other-grade fruit, and average accuracy, respectively. The field experiment showed that the average grading accuracy was 94.12% when the feeding interval was less than 1.5 s and the walking speed was less than 0.5 m/s, meeting the accuracy requirements of apple field grading.

(3) The proposed apple grading device was not combined with a harvesting machine. In a future study, we plan to improve the speed of detection and grading and the mechanical structure of the grading system integrated with a harvesting machine. Future work also needs to improve the conveying device to make the apple rotate during conveying. Taking multiple pictures of a rotating apple for feature extraction would improve the grading accuracy.