The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot
Abstract
:1. Introduction
2. Solution Design
3. Picking Robot Identification Unit
3.1. Experimental Data
3.1.1. Data Acquisition
3.1.2. Dataset Labeling and Preprocessing
3.2. Yolov8 Model
3.3. Experimental Environment and Evaluation Index
3.3.1. Experimental Environment
3.3.2. Evaluation Index
3.4. Test Results and Analysis
4. Air Injection System
4.1. Structural Design of Air Injection System
4.2. Modeling of Air Injection System
4.2.1. The Workspace Modeling
4.2.2. Model Building of Nozzle
4.2.3. Numerical Calculation of Flow Field
4.2.4. Determination of Working Section
4.3. Sensing Pressure Experiment
4.3.1. Construction of Nozzle Test Bench
4.3.2. Test and Verify Methods
4.4. Results and Analysis
4.4.1. Effect of Different Parameters on Flat Tip Nozzles
4.4.2. Effect of Different Parameters on Circular Nozzles
4.4.3. A Comparison of Parameters in the Field of Work
4.4.4. Pressure Sensitive Experimental Results and Analysis
4.5. Fruit Stem Simulation and Experiment
4.5.1. Mechanical Analysis of Fruit Separation
4.5.2. Fruit Separation Simulation Analysis
4.5.3. Fruit Separation Experiment
5. Clamping System
5.1. Mechanical Claw Structure Design
5.2. The Principle and Establishment Process of the BP Neural Network
- (1)
- When dividing the training set and the test set, all samples were scrambled first, and then 75% of the samples were taken as the training set, and the remaining samples were taken as the test set.
- (2)
- When setting training parameters, the number of iterations should be set to 1000, the error threshold to 0.00001, and the learning rate to 0.1.
5.3. BP Neural Network Algorithm Design
- 2.
- The MSE can represent the effect of a predictive model, calculating the average of the squared differences between predicted results and actual results. The smaller the MSE, the higher the prediction accuracy. The MSE curves under optimal training conditions are shown in Figure 21, with all curves near the optimal curve, and the testing set’s MSE within 0.01 difference from the target value, meeting the prediction accuracy requirements.
- 3.
- After regression curve analysis, validation, testing, and overall portions of this experiment, the correlation analysis was carried out according to the process, and seven abnormal data are presented. After calculation, the correlation coefficients (R2), the sum of square errors (SSE), and the mean square deviation (MSD) of the four groups of regression analysis were obtained, the detailed data are in Table 10. The mean square deviation of the test set is lower than that of the training set. The correctness of the regression analysis is verified.
5.4. Results and Verification
6. Conclusions
- This study presents an intelligent flexible harvesting solution that addresses the difficulties in identifying and grabbing tightly clustered blueberries. It achieves low mis-picking and damage rates in automated harvesting.
- The YOLOv8n algorithm is proposed for fruit detection training, the mAP of the YOLOv8n algorithm can reach 84.62%, the precision is 94.49%, the recall rate is 83.85%, the F1 score is 88.85%, and the test time is 0.12 s. It still maintains effective recognition in complex scenarios, such as leaf obstruction, backlighting, and uneven lighting.
- According to the new ring model, the fruit separation device is designed, and can effectively realize the functional partition of the airflow. The 8-hole 40° round-head nozzle can well meet the needs of cluster blueberry blowing.
- Given the contradiction between the high picking efficiency and poor aging of complex algorithms, a simple BP network combined with YOLOv8n real-time images can quickly realize the adaptive gripping force to fruit parameters, meeting the requirements of both tight and unbroken blueberry fruit clamping.
- The adaptive method of fruit identification and holding force proposed in this study is also suitable for the automatic picking of other types of berries.
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Yu, H.-R.; Chen, B.-H. Analysis of Phenolic Acids and Flavonoids in Rabbiteye Blueberry Leaves by UPLC-MS/MS and Preparation of Nanoemulsions and Extracts for Improving Antiaging Effects in Mice. Foods 2023, 12, 1942. [Google Scholar] [CrossRef] [PubMed]
- Ji, M.; Ying, J.; Shao, X.; Tian, Y. An Empirical Examination of Aging’s Ramifications on Large-scale Agriculture: China’s Perspective. Economics 2024, 18, 20220094. [Google Scholar] [CrossRef]
- Changyi, L.; Daochun, X.; Jiale, C. Vibration Response of Walnuts under Vibration Harvesting. Agronomy 2023, 13, 461. [Google Scholar] [CrossRef]
- Zhao, J.; Tsuchikawa, S.; Ma, T.; Hu, G.; Chen, Y.; Wang, Z.; Chen, Q.; Gao, Z.; Chen, J. Modal Analysis and Experiment of a Lycium barbarum L. Shrub for Efficient Vibration Harvesting of Fruit. Agriculture 2021, 11, 519. [Google Scholar] [CrossRef]
- Du, X.; Chen, K.; Ma, Z.; Wu, C.; Zhang, G. Design, Construction, and Evaluation of a Three-Dimensional Vibratory Harvester for Tree Fruit. Appl. Eng. Agric. 2020, 36, 221–231. [Google Scholar] [CrossRef]
- Liu, S.S.; Liu, Y.B.; Simmons, G.S. Oxygenated Phosphine Fumigation for Control of Light Brown Apple Moth (Lepidoptera: Tortricidae) Eggs on Cut-Flowers. J. Econ. Entomol. 2015, 108, 1630–1636. [Google Scholar] [CrossRef]
- Tan, K.; Lee, W.S.; Gan, H.; Wang, S. Recognising blueberry fruit of different maturity using histogram oriented gradients and colour features in outdoor scenes. Biosyst. Eng. 2018, 176, 59–72. [Google Scholar] [CrossRef]
- Hiroko, K. Application of artificial intelligence in quality test of vibratory forest fruit harvesting machinery. Comput. Informatiz. Mech. Syst. 2019, 2, 012205. [Google Scholar]
- Takeda, F.; Krewer, G.; Li, C.; MacLean, D.; Olmstead, J.W. Techniques for increasing machine harvest efficiency in highbush blueberry. HortTechnology 2013, 23, 430–436. [Google Scholar] [CrossRef]
- Li, P.; Lee, S.; Hsu, H. Review on fruit harvesting method for potential use of automatic fruit harvesting systems. Intelligent Information Technology Application Association. In Proceedings of the 2011 International Conference on Power Electronics and Engineering Application (PEEA 2011), Shenzhen, China, 24–25 December 2011; Division of ITEE, School of AMME, University of South Australia: Mawson Lakes, SA, Australia, 2011; pp. 359–374. [Google Scholar]
- De Preter, A.; Anthonis, J.; De Baerdemaeker, J. Development of a Robot for Harvesting Straw-berries. IFAC-PapesOnLine 2018, 51, 14–19. [Google Scholar] [CrossRef]
- Zhu, Y.; Feng, K.; Hua, C.; Wang, X.; Hu, Z.; Wang, H.; Su, H. Model Analysis and Experimental Investigation of Soft Pneumatic Manipulator for Fruit Grasping. Sensors 2022, 22, 4532. [Google Scholar] [CrossRef] [PubMed]
- Han, K.S.; Kim, S.C.; Lee, Y.B.; Kim, S.C.; Im, D.H.; Choi, H.K.; Hwang, H. Strawberry Harvesting Robot for Bench-type Cultivation. J. Biosyst. Eng. 2012, 37, 65–74. [Google Scholar] [CrossRef]
- Ji, H.; Song, Q.; Liu, Z. Micro-milling machinability prediction for crystalline materials via numerical-analytical hybrid modelling and strain rate-dependent grain-scale simulation. J. Manuf. Process. 2024, 124, 972–984. [Google Scholar] [CrossRef]
- Peta, K.; Żurek, J. Prediction of air leakage in heat exchangers for automotive applications using artificial neural networks. In Proceedings of the 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), New York, NY, USA, 8–10 November 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 721–725. [Google Scholar]
- Kang, M.; Chen, Q.; Fan, Z.; Yu, C.; Wang, Y.; Yu, X. A RRT based path planning scheme for multi-DOF robots in unstructured environments. Comput. Electron. Agric. 2024, 218, 108707. [Google Scholar] [CrossRef]
- Bao, Y.; Yuan, N.; Zhao, Y.; Wu, L. Recent Patents for Collection Device of Fruit Harvesting Machine. Recent Pat. Eng. 2022, 16, 96–108. [Google Scholar] [CrossRef]
- Ye, R.; Gao, Q.; Qian, Y.; Sun, J.; Li, T. Improved YOLOv8 and SAHI Model for the Collaborative Detection of Small Targets at the Micro Scale: A Case Study of Pest Detection in Tea. Agronomy 2024, 14, 1034. [Google Scholar] [CrossRef]
- Reyad, M.; Sarhan, A.M.; Arafa, M. A modified Adam algorithm for deep neural network optimization. Neural Comput. Appl. 2023, 35, 17095–17112. [Google Scholar] [CrossRef]
- Sokolova, M.; Lapalme, G. A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 2009, 45, 427–437. [Google Scholar] [CrossRef]
- Pedraza-Peñalosa, P. Themistoclesia diminuta (Ericaceae: Vaccinieae), a new mortiño and blueberry relative from Colombia. Phytotaxa 2022, 556, 291–295. [Google Scholar] [CrossRef]
- Chen, L.; Feng, Z.-H.; Dong, T.-Z.; Wang, W.-H.; Liu, S. Numerical simulation of the internal flow field of a new main nozzle in an air-jet loom based on Fluent. Text. Res. J. 2015, 85, 1590–1601. [Google Scholar] [CrossRef]
- Zhu, H.; Lin, Y.; Xie, L. Fluent12 fluid analysis and engineering simulation. In Tamron Technology; Tsinghua University Press: Beijing, China, 2011. [Google Scholar]
- Guo, X.; Li, T.; Chen, R.; Huang, S.; Zhou, X.; Wang, N.; Li, S. Effects of the nozzle design parameters on turbulent jet development of active pre-chamber. Energy 2024, 306, 132568. [Google Scholar] [CrossRef]
- Kumar, R.; Mirikar, D.; Agrawal, A.; Yadav, H. Insights into the flow and heat transfer aspects of single and multi-orifice synthetic jets. Int. J. Heat Mass Transf. 2024, 231, 125897. [Google Scholar] [CrossRef]
- Liu, J.; Peng, Y.; Faheem, M. Experimental and theoretical analysis of fruit plucking patterns for robotic tomato harvesting. Comput. Electron. Agric. 2020, 173, 105330. [Google Scholar] [CrossRef]
- Nnadi, S.N.; Ajadalu, I.; Rahmani, A.; Aliyu, A.; Elgeneidy, K.; Montazeri, A.; Sohani, B. Development, Experimental, and Numerical Characterisation of Novel Flexible Strain Sensors for Soft Robotics Applications. Robotics 2024, 13, 103. [Google Scholar] [CrossRef]
- Xu, B.; Zhang, X.; Yang, Z.; Wang, J.; Yan, S.; Ding, H. Dual flexible contact material removal model for robotic disk grinding. J. Manuf. Process. 2024, 124, 867. [Google Scholar] [CrossRef]
- Kai, X. Research on the Improvement of BP Neural Network Algorithm and its Application. Adv. Mater. Res. 2014, 926–930, 3216–3219. [Google Scholar] [CrossRef]
- Cho, G.; Kim, J.; Oh, H. Vision-Based Obstacle Avoidance Strategies for MAVs Using Optical Flows in 3-D Textured Environments. Sensors 2019, 19, 2523. [Google Scholar] [CrossRef]
- Naeem, P.; Kamaledin, G. Improving the backpropagation algorithm with consequentialism weight updates over mini-batches. Neurocomputing 2021, 461, 86–98. [Google Scholar]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning: Adaptive Computation and Machine Learning Series; The MIT Press: Cambridge, MA, USA, 2016; pp. 163–266. [Google Scholar]
- Wang, Z.; Yin, H.; Li, W.; Li, Y.; Liu, J. Research on the Design Method of Blueberry Automatic Harvesting Clamp Force Based on Neural Networks. Appl. Eng. Agric. 2024, 40, 327–338. [Google Scholar] [CrossRef]
- Wang, H.; Shao, W.; Hu, Y.; Cao, W.; Zhang, Y. Assessment of Six Machine Learning Methods for Predicting Gross Primary Productivity in Grassland. Remote Sens. 2023, 15, 3475. [Google Scholar] [CrossRef]
- Di Nardo, F.; Morbidoni, C.; Cucchiarelli, A.; Fioretti, S. Recognition of Gait Phases with a Single Knee Electrogoniometer: A Deep Learning Approach. Electronics 2020, 9, 355. [Google Scholar] [CrossRef]
- Samik, B.; Suman, C.; Debashree, G. Support Vector Regression-Based Monte Carlo Simulation of Flexible Water Clusters. ACS Omega 2020, 5, 7065–7073. [Google Scholar]
Dataset | Total Number of Images | Loose Fruit | Compact Fruit | Back Light |
---|---|---|---|---|
Training set | 4108 | 1441 | 1594 | 1073 |
Validation set | 586 | 197 | 248 | 141 |
Test set | 1175 | 412 | 496 | 267 |
Total | 5869 | 2050 | 2338 | 1481 |
Related Configuration | Configuration Parameter |
---|---|
Operating system | Windows10 Professional |
Processor | Intel(R) Core(TM) i7-9700 CPU @ 3.00 GHz 3.00 GHz |
Internal memory | 32.0 GB |
Graphics card | 32.0 GB |
Programming language | Python3.9.10 |
Deep learning framework | Pytorch |
GPU computing platform | CUDA 12.2 |
Argument | Symbol | Stats |
---|---|---|
Inlet diameter | D | 11 mm |
Nozzle length | L | 23 mm |
Hole number | N | 6, 8, 10, 12 |
aperture | d | 1.0 mm, 1.25 mm, 1.5 mm, 1.75 mm, 2.0 mm |
Hole inclination | α | 30°, 35°, 40°, 45° |
Parameters | Stats |
---|---|
1.44 | |
1.92 | |
0.09 | |
1.00 | |
1.30 |
Parameters | Stats |
---|---|
17.24839 | |
61.26441 | |
11.39066 | |
2.4368 | |
Reduced Chi-Sqr | 0.08925 |
R square (COD) | 0.99925 |
The adjusted R square | 0.9991 |
Parameters | Stats |
---|---|
diameter of fruit | 25 mm |
thickness of fruit | 15 mm |
density of fruit | 1.16 g/cm3 |
fruit Poisson ratio | 0.35 |
elastic modulus of fruit | 0.225 MPa |
length of stem | 15 mm |
diameter of frustum | 1.5 mm |
density of fruit stems | 38 g/cm3 |
fruit pedicel Poisson ratio | 0.38 |
elastic modulus of fruit stem | 14.2 Mpa |
the attachment of the stem to the ripe fruit | 0.17~0.83 N |
the attachment of the stem to immature fruit | 1.64~3.67 N |
(a) | ||||
---|---|---|---|---|
Value | Training Set | |||
Diameter (mm) | High (mm) | Weight (g) | Fruit Firmness (N) | |
Mean value | 16.9620 | 11.4955 | 2.4857 | 9.6581 |
Variance | 9.2816 | 2.0901 | 0.9548 | 1.8554 |
Standard Deviation | 3.0466 | 1.4457 | 0.9772 | 1.36213 |
(b) | ||||
Value | Test Set | |||
Diameter (mm) | High (mm) | Weight (g) | Fruit Firmness (N) | |
Mean value | 17.4629 | 11.6871 | 2.6192 | 9.8479 |
Variance | 9.3647 | 2.2664 | 0.9555 | 1.9713 |
Standard Deviation | 3.060 | 1.5054 | 0.9775 | 1.4040 |
Training Parameters | Iterations (Times) | Error Threshold | Learning Rate |
---|---|---|---|
Data | 1000 | 0.00001 | 0.1 |
Index | R2 | MAE | MBE |
---|---|---|---|
training set | 0.6992 | 0.6454 | −0.0360 |
testing set | 0.7679 | 0.6079 | −0.0557 |
Index | Training Set | Validation Set | Test Set | Test Set |
---|---|---|---|---|
R2 | 0.7795 | 0.70601 | 0.82219 | 0.77456 |
SSE | 54.5507 | 15.4288 | 10.2979 | 79.5620 |
MSD | 0.3897 | 0.5320 | 0.3551 | 0.3998 |
Number | Diameter (mm) | High (mm) | Weight (g) | Fruit Firmness (N) |
---|---|---|---|---|
1 | 12 | 7 | 1 | 8.9002 |
2 | 13 | 10 | 3 | 9.1340 |
3 | 15 | 13 | 2 | 9.4552 |
…… | ||||
13 | 16 | 8 | 4 | 7.8371 |
14 | 18 | 11 | 3 | 9.7081 |
15 | 20 | 14 | 3 | 9.7295 |
…… | ||||
25 | 21 | 11 | 4 | 11.1960 |
26 | 23 | 13 | 4 | 11.6039 |
27 | 25 | 15 | 5 | 11.8470 |
Value | Test Set | Orthogonal Test |
---|---|---|
Mean value | 9.8479 | 9.9346 |
Variance | 1.9713 | 1.6076 |
Standard Deviation | 1.4040 | 1.2679 |
Number | Measured Blueberry Diameter (mm) | Measured Blueberry Height (mm) | Actual Blueberry Weight (g) | Orthogonal Table Data (N) | Actual Data (N) | Absolute Error (N) | Relative Error |
---|---|---|---|---|---|---|---|
1 | 12.19 | 7.37 | 1.1 | 8.9002 | 8.5 | 0.4002 | 4.71% |
2 | 12.92 | 10.10 | 2.5 | 9.1340 | 8.9 | 0.2340 | 2.63% |
3 | 15.01 | 12.18 | 1.7 | 9.4552 | 9.8 | −0.3448 | 3.52% |
…… | |||||||
13 | 16.03 | 8.94 | 3.2 | 7.8371 | 7.4 | 0.4371 | 5.91% |
14 | 18.4 | 12.11 | 2.7 | 9.7081 | 9.3 | 0.4081 | 5.20% |
15 | 19.92 | 14.02 | 3.6 | 9.7295 | 9.5 | 0.2295 | 2.41% |
…… | |||||||
25 | 20.95 | 12.20 | 3.9 | 11.1960 | 11.3 | −0.1040 | 0.92% |
26 | 22.26 | 12.85 | 4.4 | 11.6039 | 12.6 | −0.9961 | 7.91% |
27 | 24.77 | 14.16 | 5.4 | 11.8470 | 12.5 | −0.653 | 5.22% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, X.; Li, P.; Hu, B.; Yin, H.; Wang, Z.; Li, W.; Xu, Y.; Li, B. The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot. Processes 2024, 12, 2591. https://doi.org/10.3390/pr12112591
Liu X, Li P, Hu B, Yin H, Wang Z, Li W, Xu Y, Li B. The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot. Processes. 2024; 12(11):2591. https://doi.org/10.3390/pr12112591
Chicago/Turabian StyleLiu, Xiaohong, Peifu Li, Bo Hu, Hao Yin, Zexian Wang, Wenxin Li, Yanxia Xu, and Baogang Li. 2024. "The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot" Processes 12, no. 11: 2591. https://doi.org/10.3390/pr12112591
APA StyleLiu, X., Li, P., Hu, B., Yin, H., Wang, Z., Li, W., Xu, Y., & Li, B. (2024). The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot. Processes, 12(11), 2591. https://doi.org/10.3390/pr12112591