Next Article in Journal
The Influence of Mg, Na, and Li Oxides on the CO2 Sorption Properties of Natural Zeolite
Previous Article in Journal
An Intelligent Kick Detection Model for Large-Hole Ultra-Deep Wells in the Sichuan Basin
Previous Article in Special Issue
Application of Machine Learning for the Prediction of Absorption, Distribution, Metabolism and Excretion (ADME) Properties from Cichorium intybus Plant Phytomolecules
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot

1
College of Mechanical and Electrical Engineering, Qingdao Agricultural University, Qingdao 266109, China
2
College of Mechanical and Automotive Engineering, Qingdao Technology University, Qingdao 266520, China
3
Qingdao Wallen Blueberry Industry Co., Ltd., Qingdao 266000, China
*
Author to whom correspondence should be addressed.
Processes 2024, 12(11), 2591; https://doi.org/10.3390/pr12112591
Submission received: 9 October 2024 / Revised: 6 November 2024 / Accepted: 13 November 2024 / Published: 18 November 2024
(This article belongs to the Special Issue Transfer Learning Methods in Equipment Reliability Management)

Abstract

:
Identifying fruit maturity accurately and achieving damage-free harvesting are challenges in designing blueberry-picking robots. This paper presents an intelligent flexible picking system. First, we trained a deep learning-based YOLOv8n network to locate the position of the fruit and determine fruit ripeness. We used a neural network to establish the relationship between fruit hardness and shape parameters, achieving an adaptive gripping force for different fruits. To address the issue of dense clusters in some blueberry varieties, we designed a fruit separation subsystem using a combination of flow field analysis and pressure-sensitive experiments. The results show that the mean average precision can reach 84.62%, the precision is 94.49%, the recall is 83.85%, the F1 score is 88.85%, and the test time is 0.12 s, which can meet the requirements for blueberry fruit recognition accuracy and speed. The spacing between closely packed fruits can increase by 4 mm, and the damage-free picking rate exceeds 92%, achieving stable, damage-free harvesting.

1. Introduction

Blueberries are rich in nutrients, help prevent the aging of brain cells, and enhance the immune system [1]. The traditional picking process is labor-intensive and requires significant human and material resources. However, the issue of an aging farming population seriously limits the development of modern orchards [2]. Given the trend toward smart orchards, the digitization of the harvesting process is very important.
The simplest harvesting method in a mechanical picking system is using a vibration harvester. This method aims to solve the problem of low yield and high consumption in artificial harvesting. It is used to harvest walnuts [3] and collect goji berries [4]. The principle of common vibration harvesters, such as inertial harvesters, is to apply mechanical vibrations to blueberries, causing the fruits to detach from their stems [5]. However, even with harder-skinned fruits like lychee, it is important to reduce impact damage during the harvesting process. This method is particularly “brutal” for delicate blueberries, easily causing harm to the fruit. To solve this problem, we propose a digital picking method. Digital picking mainly involves the precise identification of target fruits, and thus, quick harvesting. Traditional image recognition techniques for fruit detection take a long time and are easily affected by environmental factors like lighting. Liu et al. introduced an apple detection algorithm based on color and shape features [6], achieving over 85% recall, but this method lacks robustness. Tan et al. explored a method for identifying and counting blueberries at different maturity levels using gradient histograms and color features [7]. However, this method struggles with effectively identifying occluded fruits, and it is time-consuming. For quick harvesting, common vibration harvesters use mechanical vibrations to make blueberries fall from their stems [8]. However, this method can easily damage the fruits during harvest. The American company BEI produces a self-propelled rotary blueberry harvester using mechanical vibrations [9], which is widely used in production. While it has high harvesting efficiency, it can damage branches and fruits, leading to softening, making it suitable only for harder-skinned fruits. Schertz and Brown first proposed using semi-automatic mechanical or pneumatic shaking for fruit harvesting [10], developing a suction and breaking end effector. They initially sucked citrus fruits into a rubber tube, then twisted the tube to break the stems. However, this end effector struggles with different fruit sizes and can cause significant damage to both the fruit and the tree. In 2018, the agricultural R&D company Octinion developed a strawberry-picking robot that achieved autonomous harvesting [11]; however, only 82% of the harvested fruits are intact and undamaged. Additionally, because blueberries vary in size, it is challenging to set a fixed size threshold for smooth harvesting, complicating communication with the robot’s end effector. Peter P. Ling and others at Ohio State University developed a four-finger mechanical hand for tomato harvesting [12], using vacuum suction with an air pump. It can grasp fruits of various shapes and sizes, but controlling it is complex. Blueberries, growing in clusters, pose further challenges for mechanical harvesting due to small gaps between fruits. In the early 2000s, K. Naoshi created a prototype robot for strawberry harvesting [13], completing a pick in about seven seconds using suction, which is not suitable for small, clustered blueberries. Liming X and Chiu Y C developed a fruit-picking robot with a suction end effector, using a vacuum to separate individual fruits from clusters and a rotating cutter to slice stems [14]. This method addresses the challenges of picking clustered fruits but faces issues with collisions between the picking mechanism and neighboring fruits.
To address the above issues, this paper designs a new intelligent picking system based on deep learning methods and analyzes the fruit airflow design. Common adaptive algorithms and optimization methods in deep learning include support vector regression, random forests, radial basis functions, convolutional neural networks, and standard backpropagation neural networks. First, we selected a convolutional neural network (CNN), which is widely used in image classification and air leak prediction [15]. The CNN extracts image features through convolutional layers, introduces non-linearity with activation functions, and reduces feature dimensions with pooling layers to improve computational efficiency. We used the YOLOv8n object detection algorithm for the real-time identification and localization of fruit positions. Next, when designing the gripping force of the mechanical claw, we noted that support vector regression is suitable only for small-scale calculations, random forests require binary decisions on feature values and have lower detection accuracy, and radial basis functions have longer training cycles. Therefore, we choose the backpropagation neural network (BP), which has fewer data requirements, strong adaptability, and good generalization performance, to optimize the gripping force for the fruit. Finally, we designed a fruit separation device using airflow analysis and pressure-sensitive experiments.

2. Solution Design

The hardware of the picking robot includes a binocular depth camera, a robotic arm, and a bionic gripper. The chassis of the picking robot supports the entire body. It has 12 wheel motors for driving power and 12 servos for steering. The robot can move forward, backward, and sideways, and it can turn 360 degrees in place. The chassis is an all-terrain design, and the robot’s maximum load is 200 kg (Supplementary File Figure S1).
The robot-picking process is shown in Figure 1. The robot uses binocular cameras to recognize road conditions and navigate. First, it moves into the picking area and takes pictures for recognition. When it detects blueberry clusters, the depth camera sends the cluster coordinates (x1, y1, and z1) to the mobile chassis, which positions itself appropriately. Once the camera identifies a ripe fruit, it sends the target fruit’s coordinates (x2, y2, and z2) to the robotic arm. The arm, equipped with a gripper, moves toward the target fruit. The depth camera checks the distance between the target fruit and nearby fruits. If the distance is less than 4 mm, the nozzle sprays air at the fruit. When the picking conditions are met, the gripper uses the optimal force calculated by a BP neural network to pick the fruit, which is placed in the collection bin before returning to the initial position. If the target location exceeds the set limit, the picking action does not occur. The research on the mobile chassis and fruit collection method is well established [16,17], so this paper does not elaborate further.

3. Picking Robot Identification Unit

3.1. Experimental Data

3.1.1. Data Acquisition

The visual algorithm distinguishes blueberries based on their appearance features, such as color and saturation. We classify blueberries into three categories based on their appearance: ripe, semi-ripe, and unripe (Supplementary File Figure S2). Ripe blueberries are deep purple, large, and uniformly dark. Semi-ripe blueberries are red or light purple, small, and vibrant. Unripe blueberries are green, small, and very bright. The blueberry images were collected from the Qingdao Wallen Blueberry Industry Co. We also gathered some photos online to ensure dataset diversity. In total, we obtained a dataset of 1425 raw images. The dataset contains different types of blueberry pictures on sunny days, cloudy days, backlit conditions, blade obstruction conditions, etc.

3.1.2. Dataset Labeling and Preprocessing

First, to test the differences between the original images and the images annotated by the labeling tool LabelImg, we used a transfer learning method to test the blueberry dataset. We used a public strawberry dataset with 5724 images as the auxiliary domain to train the YOLOv8n model (Supplementary File Figure S3). Then, we used the blueberry dataset as the target domain and continued training the YOLOv8n model to develop the blueberry maturity recognition model. The results show that the differences between the original images and the annotated images are minimal. Therefore, we decided to use the LabelImg tool to annotate the images.
We labeled ripe blueberries as “blueberry-R”, semi-ripe blueberries were labeled as “blueberry-S”, and immature were labeled as “blueberry-U”, after which the annotation file of XML type was generated. In this study, flipping, scaling, translation, rotation, increasing noise, and other methods were randomly combined to enhance the data of the acquired images. In addition, the corresponding annotation files of each image were synchronously transformed. Some examples of image enhancement are shown in Figure 2. We performed data augmentation on the original images, resulting in a total of 5869 images. The dataset was divided into training (4108 images), validation (586 images), and test (1175 images) sets at a ratio of 7:1:2. Through the training of the model and the evaluation of the generalization ability in the training process, the accuracy of the model was finally verified. The distribution of this dataset is shown in Table 1.

3.2. Yolov8 Model

YOLOv8 was divided into five models based on different use cases. The network depth increased with each model, leading to improved detection accuracy. YOLOv8n has the fastest detection speed. Because harvesters are not particularly good at complex algorithms, our study selected the YOLOv8n version. The YOLOv8n network consists of four parts: Input, Backbone, Neck, and Head [18] (Supplementary File Figure S4).

3.3. Experimental Environment and Evaluation Index

3.3.1. Experimental Environment

The experimental environment in this study was built on the deep learning framework Pytorch 1.10, with relevant configurations shown in Table 2. To speed up model training convergence, we used the Adam optimization algorithm [19]. This adaptive stochastic optimization method helps prevent gradient vanishing in the initial training phase, achieving stable training and faster convergence. The momentum was set to 0.9, and the weight decay was 0. To avoid memory issues during training, we set the batch sizes to 16 for the frozen phase and 8 for the thawed phase. The training ran for 300 epochs, with the first 100 epochs as the frozen phase and the last 200 as the thawed phase. This two-phase training design accelerated network convergence and improved model performance.

3.3.2. Evaluation Index

The precision ( P ), recall ( R ), mean average precision ( m A P ), F1 score ( F 1 ) , and test time were used as evaluation indicators to quantitatively compare model recognition performance.
P = T P T P + F P
R = T P T P + F N
F 1 = 2 × P × R P + R
m A P = 1 M K = 1 M A P ( K )
In the formula, T P represents the number of positive samples that the model correctly predicts. F P indicates the number of negative samples incorrectly predicted as positive by the model. F N represents the number of positive samples incorrectly predicted as negative. M is the total number of classes, and A P ( K ) is the average precision value for the K-th class.

3.4. Test Results and Analysis

After training, it can be seen that the m A P of the YOLOv8n algorithm can reach 84.62%, the precision is 94.49%, the recall rate is 83.85%, the F1 score is 88.85%, and the test time is 0.12 s [20]. This algorithm can recognize the blueberry fruit and judge whether the fruit is ripe by color images, selecting several blueberry pictures in the test set to display the visual results, and to verify the practicability of the algorithm, as shown in Figure 3. The prediction results show that the YOLOv8n algorithm can efficiently and accurately identify the maturity of blueberries in complex scenes with blade shading, backlighting, and illumination inequality. Therefore, YOLOv8n can be fully deployed on agricultural-embedded mobile devices. The fast and accurate identification of blueberry fruit is facilitated.

4. Air Injection System

4.1. Structural Design of Air Injection System

Blueberries belong to the Ericaceae family and the Vaccinium genus, and they grow in deciduous shrubs [21]. Their fruits cluster closely together, similar to grapes. During mechanical harvesting, it is important for the mechanical claw to fit between the fruit accurately and to avoid damaging them. Therefore, the design of this blueberry picking robot included an air injection system. The air injection system consists of an air compressor, a connecting tube, and a nozzle (Supplementary File Figure S5).
The air jet system primarily works by using a depth camera to identify target blueberries and assess their spacing. When the distance between blueberries is less than 4 mm (the diameter of the claw), the air jet system activates and releases high-pressure air through the nozzle to scatter the blueberries. This ensures the fruit spacing meets the requirements, allowing the mechanical claw to grab the ripe blueberries through the gaps accurately. The specific air jet process is shown in Figure 4. Since the pressure needed to disperse the blueberries is only a few kPa, a small, oil-free dry air compressor was selected. It was installed within the tracked chassis and connected by a PU rubber hose. The nozzle was mounted on the claw assembly, with a secure connection between the nozzle and hose within the claw’s housing.

4.2. Modeling of Air Injection System

4.2.1. The Workspace Modeling

The northern blueberry is nearly spherical or slightly flat, with a plump shape. The operation of the air jet device requires the separation of target blueberries from the surrounding ones. The output characteristics of the nozzles need to encompass all sizes of blueberry. Therefore, we measured the length and thickness of 300 blueberries, as illustrated in Figure 5. The diameter of the circular surface is less than or equal to 25 mm, the thickness is less than or equal to 25 mm, and the difference in diameter between fruits does not exceed 15 mm. The diameter of the blueberry clusters was determined to be no more than 40 mm. Accordingly, we created an ideal ring model with small forces at the center and large forces at the edges, as depicted in Figure 6a., the center of the weak force zone in the working area of the nozzle ring, has a diameter of 25 mm, while the diameter of the strong force zone is not less than 40 mm, with a working distance of 40 mm.

4.2.2. Model Building of Nozzle

Common flat tip and round tip nozzles were utilized (Supplementary File Figure S6), and their structural model is shown in Figure 6b. The number of nozzle holes, the tilt angle of the hole, and the diameter of the hole were designed. The hole inclination angle is the angle between the center line of the nozzle shell and the center line of the hole. The data are shown in Table 3.

4.2.3. Numerical Calculation of Flow Field

The Reynolds number of various nozzles was calculated using numerical simulation based on the following Equation.
R e = ρ v d μ
where R e represents the Reynolds number, ρ represents the air density, ν represents the average airflow speed, d represents the inner diameter of the nozzle, and μ represents the viscosity coefficient of the air. The minimum Reynolds number is no less than 10,000, so the standard k ε model was chosen, and thus the perfect turbulent flow and molecular viscosity are negligible. Since the Mach number is equal to 0.286, the fluid can be considered incompressible and the energy equation can be ignored [22].
ε is defined in the model as follows:
ε = μ ρ U i ˙ x k U i x k ¯
The turbulent viscosity coefficient is as follows:
μ t = ρ C μ k 2 ε
The standard k ε turbulence model is expressed as follows:
ρ k t + ρ k μ i X i = X j μ + μ t σ k k X j + G k G b ρ ε Y M + S k
ρ ε t + ρ ε i μ i x i = x j μ + μ t σ ε ε x j sin C 1 ε ε k G κ + C 3 G b ) C 2 ε ρ ε 2 k + s k
G k = μ i ( μ i X j + u X i ) μ i X j G b = β g i μ i P r t T X i β = 1 ρ ρ T Y M = 2 ρ ε M t 2 M t = k a 2 a = γ R T
where G k is generated by turbulent kinetic energy k under the average velocity gradient effect, G b is generated by turbulent kinetic energy k under the average velocity buoyancy effect, Y M is the ripple expansion effect of the total dissipation rate ε , and C 1 ε , C 2 ε , C 3 ε , σ k , and σ ε are empirical constants [23]; the relevant parameters are listed in Table 4.

4.2.4. Determination of Working Section

After calculating the movement stroke of the mechanical claws, it was determined that the working distance set in Figure 6a is 4 cm. As the nozzle is a multi-hole structure, the annular region in the working plane represents the coupling action distance of multi-hole airflow. Therefore, a 2D model of the coupling airflow output characteristics was established, as shown in Figure 7, comprising three stages: the beginning stage, the basic stage, and the dissipated stage. The evaluation indices of the output characteristics include the effective width of the nozzle ejecting the fluid and the velocity of the fluid cross section obtained in the foundation stage [24]. The cross section at the working distance within the foundation stage is indicated by the blue dashed line, with its length representing the effective width. Additionally, the velocity rings in the strong and weak force regions should be as round as possible, with the fluid velocity distribution avoiding divergence. The wind speed in the strong zone of the working area should not only facilitate the effective separation of the fruit, but also prevent the damage of the fruit. Therefore, Equation (11) is required:
V w V e   a n d   V w m a x V b = min V b 1 , V b 2
V b is the initial wind speed of damaged blueberries; V b 1 is the minimum wind speed of the blueberry peel; and V b 2 is the minimum wind speed of a ripe blueberry when connected with fruit stem. V w is velocity in strong zone; V w m a x is the maximum flow; and V e is the effective wind speed, that is, the initial wind speed at which the fruit falls. The effective wind speed and the initial wind speed of damaged fruit can be measured experimentally by an anemometer.
The areas delimited by lines in Figure 7 represent the airflow characteristics of the two symmetrical holes. The characteristics of the jet flow emitted from a porous nozzle are influenced by the synergistic effects of the two holes. The diagram shows that the cross-sectional diameter D c (the region marked by the lines) of the airflow intersection increases as it moves away from the nozzle. When the L c is small, there exists a basically linear relationship between D c and L c . As L c exceeds 6 D , D c increases at a slow rate; when it surpasses 8.5 D , D c remains essentially unchanged. In comparison to single-hole injection, the dotted line exhibits conspicuous shrinkage. When the L c value is different, the flow velocity V first increases and then decreases, and its peak value appears at 2.5 D < L c < 8.5 D , which is the preferred range in the design of the annular model. Therefore, L c = 40 mm is adopted in the example. It is worth noting that the central flow velocity in the superimposed part of the airflow is weaker than that in the center of the hole. In the actual blowing process, this part faces the target fruit, and the pressure increases from the center to the edges. As the distance increases, the airflow in the region where the two airflows intersect also increases. The airflow velocity within the intersection area ranges from 0 to a value similar to that at the boundary of the intersection range. The gas velocity gradually increases in the multi-hole nozzle and eventually equals the velocity at the boundary. Here, V represents the flow rate at the nozzle, and V m a x represents the maximum flow rate. For the airflow of a single hole, the jet expands according to the diffusion angle, the jet radius increases linearly with distance [25], and the variation law of the radius of the jet in the basic stage is shown in Equation (12):
R = 3.4 a ( x 0 + s )
In the formula, R represents the jet radius, a represents the turbulence coefficient, x 0 represents the pole depth, and S represents the jet range. The Formula (13) can be obtained by transforming the following formula:
R r 0 = x 0 + s x 0 = 1 + s r 0 t g β = 1 + 3.4 a s r 0
In the formula, r 0 represents the nozzle radius and β represents the jet diffusion angle. This leads to the following Equation (14):
R r 0 = λ ( a s r 0 + 0.294 )
In the formula, λ represents the exit section coefficient, and the change in the intersecting jets is usually represented by φ , as shown in Equation (15). The φ value begins to change after the jet intersects in the initial stage. The φ value is determined by the distance between the jet holes, the size of the hole dip angle α , and the size of the diffusion angle β at the outer boundary of each jet. When the main deformation rate φ in the dissipation stage is equal to a constant, the main deformation rate on any section at this time is equal, and the confluent flow is like a single free jet. At this time, the jet deformation caused by intersecting jets disappears.
φ = R X R r 0
where R X represents the jet radius after the intersection. In addition, the experimental results of the flow field analysis can be compared with the planar model.

4.3. Sensing Pressure Experiment

4.3.1. Construction of Nozzle Test Bench

The main components of the nozzle test bench include a compressor, a pressure regulator, an electromagnetic valve, a field-programmable gate array (FPGA) control board, a nozzle, and a sensor tester control board (Supplementary File Figure S7). The compressor provides high-pressure air for the test bench, while the pressure regulator controls the air pressure. The FPGA control board manages the opening and closing of the electromagnetic valve to enable air jetting. The test platform features uniformly arranged film sensors of the same size on its panel. After air jetting, the film sensors transmit signals through the sensor control board to a computer for collection and processing.

4.3.2. Test and Verify Methods

In this study, the working cross-section was set at a plane 4 cm from the nozzle. The compressor worked with the pressure regulator to maintain an output pressure of 0.5 MPa, and the nozzle connected to the compressor had a diameter of 1/4 inch, resulting in an inlet speed of 9.73 m/s for the pressure sensitivity test. Additionally, the support and the position of the sensor panel remained fixed during the tests; only the nozzles with different parameters were replaced. Each nozzle was tested three times, and the computer collected pressure data through the sensor controller to calculate the average value.

4.4. Results and Analysis

4.4.1. Effect of Different Parameters on Flat Tip Nozzles

For flat tip nozzles, the main influence parameters are the hole size and quantity. Figure 8 shows the airflow velocity (m/s) distribution of the flat nozzle under different hole numbers and diameters, with the outlet conditions kept the same, under the same outlet conditions. From the graph, for a flathead nozzle, increasing the number of holes results in a greater decrease in output velocity characteristics, while the larger the pore size, the greater the decrease in the gas output velocity characteristics. For flat heads, the best choice is a 6-hole configuration with a 1 mm hole size for optimal output characteristics.

4.4.2. Effect of Different Parameters on Circular Nozzles

Figure 9 displays the airflow velocity distribution of round tip nozzles with different numbers of holes and different inclination angles, under the same outlet conditions. From the graph, it is evident that under the same hole diameter and inclination angle conditions, the convergence of the gas velocity output characteristics of round tip nozzles improves with increasing numbers of holes. Observing the effect of the inclination angle on round tip nozzles under a constant hole diameter and number of holes, the velocity output characteristics of the nozzle gradually diverge as the tilt angle increases. Within the range of a 30° to 45° inclination angle, the velocity output characteristics of the 6-hole nozzle change rapidly, and the decreasing trend in velocity characteristics is very pronounced for the 8-hole nozzle. At the 40° inclination angle, the gas velocity characteristics of the 8-hole nozzle start to diverge. However, for the 10-hole and 12-hole round tip nozzles, the decrease in velocity characteristics is slower. In the context of dispersing blueberry clusters during the harvesting process, the 6-hole 35°nozzle, the 8-hole 40°nozzle, and the 10-hole 45°nozzle demonstrate favorable velocity output characteristics.

4.4.3. A Comparison of Parameters in the Field of Work

A comparison was made of the working area by selecting a 4 cm × 4 cm square cross-section at a distance of 4 cm from the nozzle. The experimental results of the working area for several nozzles with favorable output characteristics are illustrated in Figure 10.
From the simulated working area, it is observed that the velocity output characteristics of the 6-hole flat tip nozzle are favorable, with a slow decrease in velocity and good circularity of the plane. However, excessive convergence is not advantageous for the requirements of this study. Meanwhile, at 4 cm from the nozzle, the central velocity exceeds 15 m/s, forming a circular area with a diameter of approximately 1 cm. The 6-hole 35° round tip nozzle exhibits a noticeable difference between the weak force zone and strong force zone at the center of the 4 cm working plane, which aligns with our demand. While the output characteristics of the 6-hole are dispersed, their interaction in the intersection area of the working plane is relatively minor, although the cohesion between holes is slightly weaker. The 8-hole 40° round tip nozzle exhibits a circular distribution of weak and strong zones on the 4 cm field of work. The cohesion between holes is relatively good, enabling effective separation of surrounding fruits and foliage. On the other hand, the 10-hole 45° round tip nozzle experiences a decline in velocity output characteristics and an earlier onset of diffusion due to its larger inclination angle and greater number of holes. On the 4 cm working plane, there is significant mutual influence between the holes, resulting in mostly uniform airflow with a relatively weak velocity intensity.
Combined with the above analysis of different parameters of nozzles and working section, it is concluded that the circular nozzle has more advantages, and the nozzle with an 8-hole circular head and 40° inclination angle will have the best effect.

4.4.4. Pressure Sensitive Experimental Results and Analysis

The experiment involved pressure sensitivity testing and comparison among several relatively favorable nozzles: A. 6-hole flat tip, B. 6-hole 35° round tip, C. 8-hole 40° round tip, and D. 10-hole 45° round tip. The experimental data were transmitted to the server through the sensor for recording and analysis.
Under the same 0.5 MPa air compressor output pressure, the lateral pressure variation for the four types of nozzles is depicted in Figure 11. The flat tip nozzle exhibits the highest axial pressure output, reaching a maximum pressure of 106 kPa, which then rapidly decreases with increasing axial distance. The other three nozzles all maintain axial output pressures within 30 kPa, which are moderate enough to not damage the fruit or disrupt the fruit clusters. The round tip nozzles, both the 6-hole 35° and the 8-hole 40°, exhibit similar pressure outputs, with the lowest axial pressure point occurring around 3.5 cm from the nozzle, close to the 4 cm working plane. The 10-hole 45° circular nozzle, on the other hand, begins to show the lowest pressure point at 2.5 cm, with pressure values gradually increasing thereafter. Considering the pressure test results for the 4 cm working plane in Figure 12, it is evident that the pressure output characteristics of the flat tip nozzle and the 10-hole 45° circular nozzle do not meet the requirements. Both the 6-hole 35° and the 8-hole 40° circular nozzles exhibit maximum pressure points followed by a decrease as the axial distance increases, enabling the nozzle to achieve a ring-shaped airflow. In terms of the working plane area and peak pressure magnitude, the 8-hole 40° circular nozzle surpasses the 6-hole 35° circular nozzle, with the pressure variation trend being more pronounced. Figure 13 presents a comparison between experimental tests and numerical simulations.
In summary, according to the stress test results, the order of effectiveness for the nozzles is as follows: 8-hole 40° round > 10-hole 45° round > 6-hole 35° round > 6-hole flat.

4.5. Fruit Stem Simulation and Experiment

4.5.1. Mechanical Analysis of Fruit Separation

During the air jet process, the fruit is mainly subjected to the pressure effect from the nozzle, represented as the distributed load Q in Figure 14, as well as its weight G . In the experiment, we assumed that the weight of the fruit stem can be ignored. Meanwhile, we assumed that there is a connecting force, F S , between the fruit and the stem. We defined the angle between the fruit’s stem and the main branch as θ . In the experiment, the stem was simplified as a cantilever beam [26]. We then performed a mechanical analysis and derived the following deflection formula:
E I w = M l
where E is the elastic modulus of the material, I is the moment of inertia of the cross section of the stem, w is the deflection, and the bending moment M is the relation between the distance l from the end point to the fruit. Through the boundary conditions, Formula (17) can be determined.
E I w = 0 L Q sin θ l 2 l d l d l d 1 6 G L 3 cos θ
where L is the length of a blueberry stem, and load Q is set by the logistic model according to the output characteristics of the nozzle axis distance l ; the detailed values are shown in Table 5, and the established equation is shown in (18).
Q = A 2 + A 1 A 2 1 + ( x x 0 ) P
The relationship between l and x is as follows:
x = l s i n θ
The parameter l is a direct indicator of the blueberry’s separation from the stem. We provide an example of the design for the largest blueberry: When the blueberry stalk is in a vertical direction and subjected to a weak gravitational force, the deformation caused by the full load is approximately 10.87 mm. When the fruit stalk is in a horizontal position and influenced by the dominant force of gravity, the sagging deformation is approximately 1.56 mm. When the length of the blueberry stalk is 15 mm and the blueberry is tightly positioned with an angle of 53°, the deformation is approximately 7.25 mm. Considering the sagging deformation of 1.66 mm for the horizontally positioned fruit stalk, there remains approximately 5.59 mm of space, which completely satisfies the extension of the mechanical claw into the remaining gap.

4.5.2. Fruit Separation Simulation Analysis

We conducted a fluid–structure interaction simulation of blueberry fruits using an 8-hole 40° nozzle. Taking the largest fruit as an example, the fruit-stalk model and its structure and material specific parameters are detailed in Table 6 below.
Under the output conditions of the 8-hole nozzle at 40°, the blueberry is placed in a 4 cm working area. The pressure and strain on the vertical fruit when the stem length is 15 mm are shown in Figure 15a,b. The pressure on the vertical fruit concentrates on the lower left side facing the nozzle, while the pressure on the overall fruit and the reverse of the stem is smaller. Thus, vertical fruit deformation occurs mainly on the stem, tilting away from the nozzle. The deformation is about 9.68 mm. After the fruit bends, the load decreases, and the effects of gravity and the pressure difference in the flow rates above and below the fruit cause a little error with the design. The pressure and strain on horizontally placed fruit are shown in Figure 15c,d. The pressure on the horizontal fruit mainly focuses on the outer edges, where there are connected maximum pressure points. The pressure on the overall fruit is substantial, but remains below 20 kPa, resulting in very small deformations that do not affect harvesting, consumption, or sales. The airflow from the 8-hole nozzle is symmetrical, creating balanced pressure around the fruit. However, due to the effects of gravity, the deformation of the stem leans downward. This leads to a total deformation of about 1.74 mm, which slightly deviates from the earlier theoretical calculations. The pressure and strain when the fruit is positioned at a 53° angle are shown in Figure 15e,f. The pressure is mainly on the top of the blueberry, while other areas show more balanced pressure. The stem deforms upward, with a total deformation of about 7.12 mm. This upward deformation increases the load, resulting in greater pressure differences between the upper and lower sides, leading to larger deformations than the theoretical calculations. Considering the 1.74 mm deformation in the horizontal position, the smallest gap in the lower fruit cluster is approximately 5.38 mm, which differs by 0.2 mm from the ideal calculated gap. However, all results meet the requirements for the mechanical gripper’s insertion.

4.5.3. Fruit Separation Experiment

To further verify the accuracy of the simulation and theoretical analysis, we conducted separation experiments on blueberry clusters. The air jet device used the previously selected 8-hole, 40° round nozzle, and the results of the separation experiment are shown in Figure 16. The figure displays how fruits at different ripeness levels (green, red, and ripe) interfere with the target fruit. To prevent the robotic claw from piercing the fruit, the gap for fruit separation after the air jet should be greater than 4 mm. In Figure 16a, the posture of the blueberries shows that the target fruit leans against the interfering fruit. When the target fruit is larger than the interfering fruit, good grabbing conditions are achieved after the air jet. Figure 16b illustrates that ripe fruits with a larger diameter are in a weak force zone and remain upright against the airflow, making them less likely to deform. The surrounding unripe red fruits are influenced by the strong force zone and disperse, providing favorable conditions for the harvesting robot. Figure 16c presents the separation results between ripe green and red fruits. The green fruit is harder and affected by a smaller airflow range, but its stem is softer than the red fruit, resulting in a deformation amount similar to that of the red fruit. In Figure 16d, the gap between the fruits measures approximately 6.53 mm using a caliper. Figure 16e shows the interference of ripe fruits with the target ripe fruit, while Figure 16f presents the separation results between ripe fruits. With the airflow directed at the target fruit, the distance between the side fruits is about 5.08 mm. Due to differences between real blueberry clusters and theoretical calculations, there is an error of 10.0% compared to the theory, and a simulation error of 5.9%. All measurements meet the robotic claw’s requirements. In addition to the specific cases in the figures, we also conducted 30 sets of air jet experiments (Supplementary File Figure S8). For Type I, the maximum is 5.81 mm, the minimum is 5.08 mm, and the average is 5.41 mm, with a small deviation. Type II has a maximum of 6.42 mm, a minimum of 5.62 mm, and an average of 5.95, also with a small deviation. Type III shows a maximum of 5.43 mm, a minimum of 4.76 mm, and an average of 5.2 mm, with a larger deviation. Although the green fruit has a short stem and a poor bending effect, its minimum distance of 4.71 mm is still greater than the claw diameter, meeting design requirements. At the same time, to calibrate the effective wind speed mentioned earlier, we placed a wind speed tester under the fruit cluster during the air jet separation process. Under the requirement of a 3 mm separation gap, the nozzle output speed must be at least 9.2 m/s to be effective. We directed the air jet device at the back of the fruit and gradually increased the output pressure while placing the wind speed tester behind the fruit. The results show that at an airflow of about 23.5 m/s, the fruit completely separates from the stem and flies out. When we fixed the blueberry fruit in a fixture and blew air directlyat the fruit at a speed of 23.5 m/s, the skin of the fruit did not break. Therefore, we set the maximum wind speed that the fruit can withstand at 23.5 m/s.

5. Clamping System

5.1. Mechanical Claw Structure Design

First, because blueberries are delicate, flexible harvesting is essential, requiring both flexible materials and structures [27]. For materials, elastic options like rubber, silicone, or flexible fibers are suitable [28]. In designing the flexible structure, a multi-joint design simulates hand movement. This design used a “petal-style” wrapping method, with thin claws that can fit into small gaps between blueberries. The claws of the mechanical grip are designed to be thin so they can fit into small gaps between blueberries. The claws also curve at a specific angle to ensure that even if blueberries fall between them, they can still be caught without damage.
Next, the structure of the robotic arm, shown in Figure 17, mainly consists of a servo motor, hydraulic cylinder, stop ring, and harvesting hand. The design used a “twist” and “pull” method to pick blueberries. The “twist” is achieved through servo motor 2. When the jet nozzle blows away clustered blueberries, the robotic arm grabs them, and then servo motor 2 starts to rotate. Because of the detachable connection at servo motor 2, it also turns the hydraulic cylinder and robotic arm to separate the blueberries from the plant. The “pull” is handled by servo motor 1. When the blueberries are tightly attached to the plant, twisting alone is not enough, so the robotic arm pulls them. Servo motor 1 slightly rotates the entire arm to facilitate this pulling action. After the robotic arm wraps around and picks the blueberries, it rotates to the collection box for gathering.

5.2. The Principle and Establishment Process of the BP Neural Network

A neural network using an error backpropagation algorithm is called a BP neural network. The network structure consists of an input layer, several intermediate hidden layers, and an output layer. Each level is fully interconnected, but there are no connections between nodes within the same level. The working principle of the BP neural network is mainly based on two core processes: the forward propagation of the signal and the backpropagation of the error [29]. Because the Sigmoid function has a limited output range, it prevents data from easily diverging during propagation and can represent probability values at the output layer [30]. Therefore, we used the Sigmoid function as an activation function. The value of x can be any real number, meaning the range of x is from to + . The Sigmoid function formula is as follows:
φ x = 1 1 + e x
In the forward propagation stage, the input data start from the input layer, are processed by each hidden layer, and are finally passed to the output layer. At this point, the output is calculated based on pre-set weights and bias values. If there is a large difference between the actual output and the expected output, the network enters the error backpropagation phase. At this stage, the errors of the output layer will be returned layer by layer to the input layer, and distributed to the nodes of each layer to calculate the error signal of each layer. These error signals are then used to adjust the weights of the nodes. The model converges to the optimal solution faster during optimization by using the Nesterov accelerated gradient method [31]. By repeating this process, the weight of the network is gradually adjusted until the error of the network output is reduced to an acceptable preset value or reaches a predetermined number of iterations. At the end of the iteration, the optimal parameter is obtained, that is, the parameter with the minimum value of the performance function, including the final weight matrix and bias, which can be used for prediction. The established process is shown in Figure 18.
What should be added is the following:
(1)
When dividing the training set and the test set, all samples were scrambled first, and then 75% of the samples were taken as the training set, and the remaining samples were taken as the test set.
(2)
When setting training parameters, the number of iterations should be set to 1000, the error threshold to 0.00001, and the learning rate to 0.1.

5.3. BP Neural Network Algorithm Design

According to “DEEP LEARNING” [32], the BP neural network model was established: A grouping scheme with randomly selected fruits was used for the training and test sets. After grouping, they were measured and calculated, and the mean and standard deviation of parameters such as diameter are shown in Table 7a,b.
As can be seen from Table 7a,b, the range of each parameter of the test set fruits covered the training set. It proves that the data distribution of the two samples is reasonable under this grouping scheme.
The training parameters were set as shown in Table 8. After that, the training of the grid was started.
After the simulation test, the prediction results of the original data were output.
After repeated training, the best fitting results were obtained, and the root mean square error (RMSE) of both the training set and test set is less than 1. The comparison of the predicted results of the obtained training set and test set is shown in Figure 19 and Figure 20 [33]: Where the horizontal coordinate is the predicted sample, the vertical coordinate is the predicted result, the red line represents the true value, and the blue line represents the predicted value.
Through the root-mean-square error calculation Formula (21), the root-mean-square error of the training set and test set is less than 1, and the fitting effect is good.
R M S E = M S E = 1 n i = 1 n ( y i y i ^ ) 2
The test set represents the final simulation results. After excluding some larger errors, the deviation between the actual results and projected results of most predicted samples is very small. The model can accurately predict the hardness of the blueberry fruit based on the input diameter, height, and weight.
The code was input into the picking machine claw. After the experiment, the deviation value of blueberry weight is very small, and the influence weight is very small. Therefore, the upper and lower limits of the blueberry weight can be brought into the algorithm to calculate the upper and lower limits of the mechanical claw-holding force at this time. When picking blueberries, the recognition camera of the mechanical claw can identify the diameter and height of the blueberries, and then input the upper and lower limits of the weight of the blueberries, respectively. After BP neural network training, the output can obtain the interval value of the hardness of the blueberries to control the size of the clamping force of the mechanical claw.
The specific parameters are as follows:
  • The coefficient of determination [34] (R2), the mean absolute error [35] (MAE), and mean bias error [36] (MBE), The R2, MAE, and MBE of the training and testing datasets are displayed in Table 9.
The closer the coefficient of determination R2 is to 1, the better, as it indicates the degree to which the model explains the data. It reflects the extent to which the model accounts for the variability of the dependent variable. An R2 value of around 0.7 is close to 1, indicating that it has a relatively good fit. The R2 values for the testing and training sets are close, with a deviation from the optimal value of 0.7 of less than 10%.
The MAE refers to the average absolute difference between the predicted and actual values. The mean absolute error of the test set results is less than 1, meeting the expected requirement. The MBE refers to the average relative difference between the predicted and actual values. Based on the test set results, the MBE is −5.6%. Since the absolute value of the MBE is within 10%, the model is considered reasonable.
2.
The MSE can represent the effect of a predictive model, calculating the average of the squared differences between predicted results and actual results. The smaller the MSE, the higher the prediction accuracy. The MSE curves under optimal training conditions are shown in Figure 21, with all curves near the optimal curve, and the testing set’s MSE within 0.01 difference from the target value, meeting the prediction accuracy requirements.
3.
After regression curve analysis, validation, testing, and overall portions of this experiment, the correlation analysis was carried out according to the process, and seven abnormal data are presented. After calculation, the correlation coefficients (R2), the sum of square errors (SSE), and the mean square deviation (MSD) of the four groups of regression analysis were obtained, the detailed data are in Table 10. The mean square deviation of the test set is lower than that of the training set. The correctness of the regression analysis is verified.
Figure 22 shows the standardized regression curve. As seen in the figure, the data points fit the linear curve well, confirming that the regression curve follows a linear relationship. This also indicates a linear correlation between the independent and dependent variables, which helps speed up parameter decoupling and training.

5.4. Results and Verification

The model uses the BP neural network method, which takes the following fruit parameters as input: blueberry diameter, height, and weight. It outputs the maximum pressure the blueberry can withstand. A range of values are selected for blueberry diameter (12–25 mm), height (6–15 mm), and weight (1–5 g). These values form an orthogonal experiment with a total of 27 data points, which are fed into the model for training. The final result is the blueberry hardness, rounded to four decimal places. Table 11 shows the input and output results.
In the orthogonal test, the blueberry fruit hardness values need to remain within the range of the test set. The specific data are shown in Table 12.
As shown in Table 10, the blueberry hardness values from the orthogonal experiment are close to those from the test set. However, since the variance from the orthogonal experiment is smaller than that from the test set, we can consider the test set to contain the experimental set, which validates the correctness of the orthogonal test.
The verification method was as follows:
Three types of blueberries (large, medium, and small) were selected and divided into three groups. The diameter, height, and weight of the blueberries were measured. Using the rounding method, the approximate values from the orthogonal table were input. The maximum pressure the blueberries can withstand was calculated using the orthogonal table and was set as the preset gripping force for the mechanical claw. A force gauge was then used to measure the maximum pressure the blueberries can withstand, recorded as the blueberry rupture pressure. The rupture pressure was compared to the preset gripping force to calculate the absolute and relative errors (rounded to four decimal places). The specific data are shown in Table 13.
When the absolute error is positive within the range (0, + ), it means the predicted maximum pressure the fruit can withstand exceeds the actual value. In this case, the gripping force of the mechanical claw exceeds the blueberry’s maximum pressure. If the difference is too large, the blueberry will rupture. When the absolute error is negative within the range ( , 0), it means the predicted maximum pressure the fruit can withstand is lower than the actual value. In this case, the gripping force of the mechanical claw is less than the maximum pressure the fruit can handle. If the difference is too large, the claw’s grip may be too weak, making it difficult to successfully pick the fruit.
To avoid the problems caused by positive and negative errors, a flexible material can be used to design the main claw structure for better fruit gripping. Comparing the experimental results with the orthogonal table values, the relative error is below 8%. The deviation is within a reasonable range, which indicates that the model is valid and feasible.

6. Conclusions

  • This study presents an intelligent flexible harvesting solution that addresses the difficulties in identifying and grabbing tightly clustered blueberries. It achieves low mis-picking and damage rates in automated harvesting.
  • The YOLOv8n algorithm is proposed for fruit detection training, the mAP of the YOLOv8n algorithm can reach 84.62%, the precision is 94.49%, the recall rate is 83.85%, the F1 score is 88.85%, and the test time is 0.12 s. It still maintains effective recognition in complex scenarios, such as leaf obstruction, backlighting, and uneven lighting.
  • According to the new ring model, the fruit separation device is designed, and can effectively realize the functional partition of the airflow. The 8-hole 40° round-head nozzle can well meet the needs of cluster blueberry blowing.
  • Given the contradiction between the high picking efficiency and poor aging of complex algorithms, a simple BP network combined with YOLOv8n real-time images can quickly realize the adaptive gripping force to fruit parameters, meeting the requirements of both tight and unbroken blueberry fruit clamping.
  • The adaptive method of fruit identification and holding force proposed in this study is also suitable for the automatic picking of other types of berries.

Supplementary Materials

The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/pr12112591/s1, Figure S1: Blueberry picking robot structure diagram; Figure S2: Schematic diagram of blueberry maturity grade; Figure S3: Strawberry dataset labeling diagram; Figure S4: YOLOv8 model structure; Figure S5: Air injection device; Figure S6: Part of the nozzle prototype; Figure S7: Test bench diagram; Figure S8: Part of the nozzle prototype.

Author Contributions

Conceptualization, X.L. and H.Y.; methodology, H.Y.; software, H.Y.; validation, W.L., Z.W. and Y.X.; resources, B.H.; data curation, W.L.; writing—original draft preparation, P.L.; writing—review and editing, X.L.; supervision, H.Y.; project administration, B.L.; funding acquisition, B.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation (grant number: 52202508) and the Shandong Provincial Postdoctoral Science Foundation (grant number: SDCX-ZG-202400208).

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Materials, further inquiries can be directed to the corresponding author.

Acknowledgments

This study was supported by the Doctoral Startup Fund of Qingdao Agricultural University. Special thanks go to the Laboratory of Acoustics and Intelligent Control at Qingdao University of Technology for providing experimental equipment, as well as to Wolin Agriculture (Qingdao) for providing the experimental site.

Conflicts of Interest

Author Bo Hu is employed by the company Qingdao Wallen Blueberry Industry. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.

References

  1. Yu, H.-R.; Chen, B.-H. Analysis of Phenolic Acids and Flavonoids in Rabbiteye Blueberry Leaves by UPLC-MS/MS and Preparation of Nanoemulsions and Extracts for Improving Antiaging Effects in Mice. Foods 2023, 12, 1942. [Google Scholar] [CrossRef] [PubMed]
  2. Ji, M.; Ying, J.; Shao, X.; Tian, Y. An Empirical Examination of Aging’s Ramifications on Large-scale Agriculture: China’s Perspective. Economics 2024, 18, 20220094. [Google Scholar] [CrossRef]
  3. Changyi, L.; Daochun, X.; Jiale, C. Vibration Response of Walnuts under Vibration Harvesting. Agronomy 2023, 13, 461. [Google Scholar] [CrossRef]
  4. Zhao, J.; Tsuchikawa, S.; Ma, T.; Hu, G.; Chen, Y.; Wang, Z.; Chen, Q.; Gao, Z.; Chen, J. Modal Analysis and Experiment of a Lycium barbarum L. Shrub for Efficient Vibration Harvesting of Fruit. Agriculture 2021, 11, 519. [Google Scholar] [CrossRef]
  5. Du, X.; Chen, K.; Ma, Z.; Wu, C.; Zhang, G. Design, Construction, and Evaluation of a Three-Dimensional Vibratory Harvester for Tree Fruit. Appl. Eng. Agric. 2020, 36, 221–231. [Google Scholar] [CrossRef]
  6. Liu, S.S.; Liu, Y.B.; Simmons, G.S. Oxygenated Phosphine Fumigation for Control of Light Brown Apple Moth (Lepidoptera: Tortricidae) Eggs on Cut-Flowers. J. Econ. Entomol. 2015, 108, 1630–1636. [Google Scholar] [CrossRef]
  7. Tan, K.; Lee, W.S.; Gan, H.; Wang, S. Recognising blueberry fruit of different maturity using histogram oriented gradients and colour features in outdoor scenes. Biosyst. Eng. 2018, 176, 59–72. [Google Scholar] [CrossRef]
  8. Hiroko, K. Application of artificial intelligence in quality test of vibratory forest fruit harvesting machinery. Comput. Informatiz. Mech. Syst. 2019, 2, 012205. [Google Scholar]
  9. Takeda, F.; Krewer, G.; Li, C.; MacLean, D.; Olmstead, J.W. Techniques for increasing machine harvest efficiency in highbush blueberry. HortTechnology 2013, 23, 430–436. [Google Scholar] [CrossRef]
  10. Li, P.; Lee, S.; Hsu, H. Review on fruit harvesting method for potential use of automatic fruit harvesting systems. Intelligent Information Technology Application Association. In Proceedings of the 2011 International Conference on Power Electronics and Engineering Application (PEEA 2011), Shenzhen, China, 24–25 December 2011; Division of ITEE, School of AMME, University of South Australia: Mawson Lakes, SA, Australia, 2011; pp. 359–374. [Google Scholar]
  11. De Preter, A.; Anthonis, J.; De Baerdemaeker, J. Development of a Robot for Harvesting Straw-berries. IFAC-PapesOnLine 2018, 51, 14–19. [Google Scholar] [CrossRef]
  12. Zhu, Y.; Feng, K.; Hua, C.; Wang, X.; Hu, Z.; Wang, H.; Su, H. Model Analysis and Experimental Investigation of Soft Pneumatic Manipulator for Fruit Grasping. Sensors 2022, 22, 4532. [Google Scholar] [CrossRef] [PubMed]
  13. Han, K.S.; Kim, S.C.; Lee, Y.B.; Kim, S.C.; Im, D.H.; Choi, H.K.; Hwang, H. Strawberry Harvesting Robot for Bench-type Cultivation. J. Biosyst. Eng. 2012, 37, 65–74. [Google Scholar] [CrossRef]
  14. Ji, H.; Song, Q.; Liu, Z. Micro-milling machinability prediction for crystalline materials via numerical-analytical hybrid modelling and strain rate-dependent grain-scale simulation. J. Manuf. Process. 2024, 124, 972–984. [Google Scholar] [CrossRef]
  15. Peta, K.; Żurek, J. Prediction of air leakage in heat exchangers for automotive applications using artificial neural networks. In Proceedings of the 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), New York, NY, USA, 8–10 November 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 721–725. [Google Scholar]
  16. Kang, M.; Chen, Q.; Fan, Z.; Yu, C.; Wang, Y.; Yu, X. A RRT based path planning scheme for multi-DOF robots in unstructured environments. Comput. Electron. Agric. 2024, 218, 108707. [Google Scholar] [CrossRef]
  17. Bao, Y.; Yuan, N.; Zhao, Y.; Wu, L. Recent Patents for Collection Device of Fruit Harvesting Machine. Recent Pat. Eng. 2022, 16, 96–108. [Google Scholar] [CrossRef]
  18. Ye, R.; Gao, Q.; Qian, Y.; Sun, J.; Li, T. Improved YOLOv8 and SAHI Model for the Collaborative Detection of Small Targets at the Micro Scale: A Case Study of Pest Detection in Tea. Agronomy 2024, 14, 1034. [Google Scholar] [CrossRef]
  19. Reyad, M.; Sarhan, A.M.; Arafa, M. A modified Adam algorithm for deep neural network optimization. Neural Comput. Appl. 2023, 35, 17095–17112. [Google Scholar] [CrossRef]
  20. Sokolova, M.; Lapalme, G. A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 2009, 45, 427–437. [Google Scholar] [CrossRef]
  21. Pedraza-Peñalosa, P. Themistoclesia diminuta (Ericaceae: Vaccinieae), a new mortiño and blueberry relative from Colombia. Phytotaxa 2022, 556, 291–295. [Google Scholar] [CrossRef]
  22. Chen, L.; Feng, Z.-H.; Dong, T.-Z.; Wang, W.-H.; Liu, S. Numerical simulation of the internal flow field of a new main nozzle in an air-jet loom based on Fluent. Text. Res. J. 2015, 85, 1590–1601. [Google Scholar] [CrossRef]
  23. Zhu, H.; Lin, Y.; Xie, L. Fluent12 fluid analysis and engineering simulation. In Tamron Technology; Tsinghua University Press: Beijing, China, 2011. [Google Scholar]
  24. Guo, X.; Li, T.; Chen, R.; Huang, S.; Zhou, X.; Wang, N.; Li, S. Effects of the nozzle design parameters on turbulent jet development of active pre-chamber. Energy 2024, 306, 132568. [Google Scholar] [CrossRef]
  25. Kumar, R.; Mirikar, D.; Agrawal, A.; Yadav, H. Insights into the flow and heat transfer aspects of single and multi-orifice synthetic jets. Int. J. Heat Mass Transf. 2024, 231, 125897. [Google Scholar] [CrossRef]
  26. Liu, J.; Peng, Y.; Faheem, M. Experimental and theoretical analysis of fruit plucking patterns for robotic tomato harvesting. Comput. Electron. Agric. 2020, 173, 105330. [Google Scholar] [CrossRef]
  27. Nnadi, S.N.; Ajadalu, I.; Rahmani, A.; Aliyu, A.; Elgeneidy, K.; Montazeri, A.; Sohani, B. Development, Experimental, and Numerical Characterisation of Novel Flexible Strain Sensors for Soft Robotics Applications. Robotics 2024, 13, 103. [Google Scholar] [CrossRef]
  28. Xu, B.; Zhang, X.; Yang, Z.; Wang, J.; Yan, S.; Ding, H. Dual flexible contact material removal model for robotic disk grinding. J. Manuf. Process. 2024, 124, 867. [Google Scholar] [CrossRef]
  29. Kai, X. Research on the Improvement of BP Neural Network Algorithm and its Application. Adv. Mater. Res. 2014, 926–930, 3216–3219. [Google Scholar] [CrossRef]
  30. Cho, G.; Kim, J.; Oh, H. Vision-Based Obstacle Avoidance Strategies for MAVs Using Optical Flows in 3-D Textured Environments. Sensors 2019, 19, 2523. [Google Scholar] [CrossRef]
  31. Naeem, P.; Kamaledin, G. Improving the backpropagation algorithm with consequentialism weight updates over mini-batches. Neurocomputing 2021, 461, 86–98. [Google Scholar]
  32. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning: Adaptive Computation and Machine Learning Series; The MIT Press: Cambridge, MA, USA, 2016; pp. 163–266. [Google Scholar]
  33. Wang, Z.; Yin, H.; Li, W.; Li, Y.; Liu, J. Research on the Design Method of Blueberry Automatic Harvesting Clamp Force Based on Neural Networks. Appl. Eng. Agric. 2024, 40, 327–338. [Google Scholar] [CrossRef]
  34. Wang, H.; Shao, W.; Hu, Y.; Cao, W.; Zhang, Y. Assessment of Six Machine Learning Methods for Predicting Gross Primary Productivity in Grassland. Remote Sens. 2023, 15, 3475. [Google Scholar] [CrossRef]
  35. Di Nardo, F.; Morbidoni, C.; Cucchiarelli, A.; Fioretti, S. Recognition of Gait Phases with a Single Knee Electrogoniometer: A Deep Learning Approach. Electronics 2020, 9, 355. [Google Scholar] [CrossRef]
  36. Samik, B.; Suman, C.; Debashree, G. Support Vector Regression-Based Monte Carlo Simulation of Flexible Water Clusters. ACS Omega 2020, 5, 7065–7073. [Google Scholar]
Figure 1. Flow chart of robot picking.
Figure 1. Flow chart of robot picking.
Processes 12 02591 g001
Figure 2. Example of image enhancement.
Figure 2. Example of image enhancement.
Processes 12 02591 g002
Figure 3. Detection effect of blueberry fruit.
Figure 3. Detection effect of blueberry fruit.
Processes 12 02591 g003aProcesses 12 02591 g003b
Figure 4. A flow chart of the air injection device.
Figure 4. A flow chart of the air injection device.
Processes 12 02591 g004
Figure 5. Size scale of blueberries: (a) The length of the blueberry; (b) Thickness of blueberries.
Figure 5. Size scale of blueberries: (a) The length of the blueberry; (b) Thickness of blueberries.
Processes 12 02591 g005
Figure 6. Nozzle diagram: (a) nozzle working diagram; (b) nozzle structure diagram.
Figure 6. Nozzle diagram: (a) nozzle working diagram; (b) nozzle structure diagram.
Processes 12 02591 g006
Figure 7. The output characteristics of a porous nozzle.
Figure 7. The output characteristics of a porous nozzle.
Processes 12 02591 g007
Figure 8. A flow velocity distribution diagram of a flat nozzle. (a) A flat nozzle with a 1 mm diameter and different hole numbers; (b) a flat nozzle with 6 holes of different diameters.
Figure 8. A flow velocity distribution diagram of a flat nozzle. (a) A flat nozzle with a 1 mm diameter and different hole numbers; (b) a flat nozzle with 6 holes of different diameters.
Processes 12 02591 g008
Figure 9. Simulation results of circular nozzles with different angles and number of holes.
Figure 9. Simulation results of circular nozzles with different angles and number of holes.
Processes 12 02591 g009
Figure 10. Simulation results of the work area: (a) Flat head with 6 holes; (b) Round head 6 holes 35°; (c) Round head 8 holes 40°; (d) Round head 10 holes 45°.
Figure 10. Simulation results of the work area: (a) Flat head with 6 holes; (b) Round head 6 holes 35°; (c) Round head 8 holes 40°; (d) Round head 10 holes 45°.
Processes 12 02591 g010
Figure 11. Pressure changes of the axis of the four nozzles.
Figure 11. Pressure changes of the axis of the four nozzles.
Processes 12 02591 g011
Figure 12. The variation of diameter pressure in the working area of the four nozzles.
Figure 12. The variation of diameter pressure in the working area of the four nozzles.
Processes 12 02591 g012
Figure 13. Comparison between test pressure and simulation speed of four nozzles.
Figure 13. Comparison between test pressure and simulation speed of four nozzles.
Processes 12 02591 g013
Figure 14. The force analysis of different fruit stems under the nozzle.
Figure 14. The force analysis of different fruit stems under the nozzle.
Processes 12 02591 g014
Figure 15. ((a) Pressure diagram (b) Strain diagram) Simulation results of fruit in horizontal state; ((c) Pressure diagram (d) Strain diagram) Simulation results of vertical fruit; ((e) Pressure diagram (f) Strain diagram) Simulation results of fruit with an inclination of 53°.
Figure 15. ((a) Pressure diagram (b) Strain diagram) Simulation results of fruit in horizontal state; ((c) Pressure diagram (d) Strain diagram) Simulation results of vertical fruit; ((e) Pressure diagram (f) Strain diagram) Simulation results of fruit with an inclination of 53°.
Processes 12 02591 g015
Figure 16. Fruit separation experiment results.
Figure 16. Fruit separation experiment results.
Processes 12 02591 g016
Figure 17. Manipulator structure. 1, 2—servo motor; 3—hydraulic pump; 4—clamp ring; 5—manipulator.
Figure 17. Manipulator structure. 1, 2—servo motor; 3—hydraulic pump; 4—clamp ring; 5—manipulator.
Processes 12 02591 g017
Figure 18. The building process of a BP neural network model.
Figure 18. The building process of a BP neural network model.
Processes 12 02591 g018
Figure 19. Comparison of training set prediction results.
Figure 19. Comparison of training set prediction results.
Processes 12 02591 g019
Figure 20. Comparison of testing set prediction results.
Figure 20. Comparison of testing set prediction results.
Processes 12 02591 g020
Figure 21. Mean squared error curve and optimal point.
Figure 21. Mean squared error curve and optimal point.
Processes 12 02591 g021
Figure 22. Regression curve.
Figure 22. Regression curve.
Processes 12 02591 g022
Table 1. The distribution of the blueberry dataset.
Table 1. The distribution of the blueberry dataset.
DatasetTotal Number of ImagesLoose FruitCompact FruitBack Light
Training set4108144115941073
Validation set586197248141
Test set1175412496267
Total5869205023381481
Table 2. Experimental environment configuration.
Table 2. Experimental environment configuration.
Related ConfigurationConfiguration Parameter
Operating systemWindows10 Professional
ProcessorIntel(R) Core(TM) i7-9700 CPU @ 3.00 GHz 3.00 GHz
Internal memory32.0 GB
Graphics card32.0 GB
Programming languagePython3.9.10
Deep learning frameworkPytorch
GPU computing platformCUDA 12.2
Table 3. Nozzle design parameters.
Table 3. Nozzle design parameters.
ArgumentSymbolStats
Inlet diameterD11 mm
Nozzle lengthL23 mm
Hole numberN6, 8, 10, 12
apertured1.0 mm, 1.25 mm, 1.5 mm, 1.75 mm, 2.0 mm
Hole inclinationα30°, 35°, 40°, 45°
Table 4. Parameters of the standard model in the ANSYS Fluent 2022 R1 software.
Table 4. Parameters of the standard model in the ANSYS Fluent 2022 R1 software.
ParametersStats
C 1 ε 1.44
C 2 ε 1.92
C 3 ε 0.09
σ k 1.00
σ ε 1.30
Table 5. Specific parameters of load model fitting.
Table 5. Specific parameters of load model fitting.
ParametersStats
A 1 17.24839
A 2 61.26441
x 0 11.39066
P 2.4368
Reduced Chi-Sqr0.08925
R square (COD)0.99925
The adjusted R square0.9991
Table 6. Material parameters of fruit stem.
Table 6. Material parameters of fruit stem.
ParametersStats
diameter of fruit25 mm
thickness of fruit15 mm
density of fruit1.16 g/cm3
fruit Poisson ratio0.35
elastic modulus of fruit0.225 MPa
length of stem15 mm
diameter of frustum1.5 mm
density of fruit stems38 g/cm3
fruit pedicel Poisson ratio0.38
elastic modulus of fruit stem14.2 Mpa
the attachment of the stem to the ripe fruit0.17~0.83 N
the attachment of the stem to immature fruit1.64~3.67 N
Table 7. (a) Evaluation metrics for the fruit in the training set. (b) Evaluation metrics for the fruit in the test set.
Table 7. (a) Evaluation metrics for the fruit in the training set. (b) Evaluation metrics for the fruit in the test set.
(a)
ValueTraining Set
Diameter
(mm)
High
(mm)
Weight
(g)
Fruit
Firmness (N)
Mean value16.962011.49552.48579.6581
Variance9.28162.09010.95481.8554
Standard Deviation3.04661.44570.97721.36213
(b)
ValueTest Set
Diameter
(mm)
High
(mm)
Weight
(g)
Fruit
Firmness (N)
Mean value17.462911.68712.61929.8479
Variance9.36472.26640.95551.9713
Standard Deviation3.0601.50540.97751.4040
Table 8. Training parameter.
Table 8. Training parameter.
Training ParametersIterations (Times)Error ThresholdLearning Rate
Data10000.000010.1
Table 9. R2, MAE, and MBE of training set and test set.
Table 9. R2, MAE, and MBE of training set and test set.
IndexR2MAEMBE
training set0.69920.6454−0.0360
testing set0.76790.6079−0.0557
Table 10. Evaluation index of regression analysis.
Table 10. Evaluation index of regression analysis.
IndexTraining SetValidation SetTest SetTest Set
R20.77950.706010.822190.77456
SSE54.550715.428810.297979.5620
MSD0.38970.53200.35510.3998
Table 11. Orthogonal experimental.
Table 11. Orthogonal experimental.
NumberDiameter (mm)High (mm)Weight (g)Fruit Firmness (N)
112718.9002
2131039.1340
3151329.4552
……
1316847.8371
14181139.7081
15201439.7295
……
252111411.1960
262313411.6039
272515511.8470
Table 12. Fruit firmness in test set and orthogonal test.
Table 12. Fruit firmness in test set and orthogonal test.
ValueTest SetOrthogonal Test
Mean value9.84799.9346
Variance1.97131.6076
Standard Deviation1.40401.2679
Table 13. Experimental validation table.
Table 13. Experimental validation table.
NumberMeasured Blueberry
Diameter (mm)
Measured Blueberry Height (mm)Actual Blueberry Weight (g)Orthogonal Table Data (N)Actual Data (N)Absolute Error (N)Relative Error
112.197.371.18.90028.50.40024.71%
212.9210.102.59.13408.90.23402.63%
315.0112.181.79.45529.8−0.34483.52%
……
1316.038.943.27.83717.40.43715.91%
1418.412.112.79.70819.30.40815.20%
1519.9214.023.69.72959.50.22952.41%
……
2520.9512.203.911.196011.3−0.10400.92%
2622.2612.854.411.603912.6−0.99617.91%
2724.7714.165.411.847012.5−0.6535.22%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Liu, X.; Li, P.; Hu, B.; Yin, H.; Wang, Z.; Li, W.; Xu, Y.; Li, B. The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot. Processes 2024, 12, 2591. https://doi.org/10.3390/pr12112591

AMA Style

Liu X, Li P, Hu B, Yin H, Wang Z, Li W, Xu Y, Li B. The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot. Processes. 2024; 12(11):2591. https://doi.org/10.3390/pr12112591

Chicago/Turabian Style

Liu, Xiaohong, Peifu Li, Bo Hu, Hao Yin, Zexian Wang, Wenxin Li, Yanxia Xu, and Baogang Li. 2024. "The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot" Processes 12, no. 11: 2591. https://doi.org/10.3390/pr12112591

APA Style

Liu, X., Li, P., Hu, B., Yin, H., Wang, Z., Li, W., Xu, Y., & Li, B. (2024). The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot. Processes, 12(11), 2591. https://doi.org/10.3390/pr12112591

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop