Design, Development, Integration, and Field Evaluation of a Ridge-Planting Strawberry Harvesting Robot

Abstract

Due to the complex unstructured environmental factors in ridge-planting strawberry cultivation, automated harvesting remains a significant challenge. This paper presents an oriented-ridge double-arm cooperative harvesting robot designed for this cultivation. The robot is equipped with a novel non-destructive harvesting end-effector and two self-developed specialized manipulators, integrated with the strawberry picking point visual perception system based on the lightweight Mask R-CNN and a CAN bus-based machine control system. The greenhouse harvesting experiments show that the robot achieved an average harvesting success rate of 49.30% in natural environments after flower and fruit thinning, while only a 30.23% success rate was achieved in untrimmed natural environments. This indicates that the agronomic practice of flower and fruit thinning can significantly simplify the automated harvesting environment and improve harvesting performance. Automated harvesting efficiency test results show that the single-arm average harvesting speed is 7 s per fruit, while double-arm cooperative harvesting can achieve 4 s per fruit. Future expansion by increasing the number of robotic arms could significantly improve harvesting efficiency. However, the study conducted for this paper was poor for those strawberries whose body or stem was severely blocked, which should be further improved upon in follow-up studies.

1. Introduction

Ripe strawberry harvesting requires a lot of artificial labor. According to a survey report, the cost of artificial harvesting accounts for more than 50% of the total production cost, and this number is still rising year by year [1]. Therefore, using machines to replace people can significantly reduce labor costs and effectively promote the increasing production income of the strawberry industry.

There are great differences in strawberry cultivation patterns in different regions around the world, and the standardization of the harvesting operation environment is uneven, which brings about many challenges for automatic and intelligent harvesting [2]. In addition, the end-effector cannot directly touch the strawberry fruit due to its brittle and vulnerable surface, so it needs to harvest the fruit by non-contact clipping or by burning off the fruit stem [3]. As a major producer and seller of strawberries, Japan is one of the first countries in the world to develop strawberry harvesting robots [4]. Kondo et al. [5] and Hayashi et al. [6] have developed several generations of strawberry harvesting robots by focusing on a variety of strawberry cultivation modes, such as ridge-planting and elevated-planting. However, the early strawberry harvesting robots were not suitable for complex natural environments. Many adjacent fruits have knotted stalks and similar colors to the background branches and leaves, making it difficult to accurately identify them [7,8]. In order to improve the robustness of visual perception systems in identifying strawberries in complex natural environments, many scholars currently use fruit target detection methods based on deep neural networks [9,10,11,12,13,14,15,16]. In the past two decades, with the rapid development of machine vision, automatic control, and artificial intelligence, many scholars have developed more advanced strawberry harvesting robots. For example, Xiong Ya et al. have achieved excellent results in the field of strawberry harvesting robots in recent years. During their time working at the Norwegian University of Life Sciences, they developed a strawberry harvesting robot for elevated cultivation [17]. This machine is mainly composed of four core modules: the target detection vision module, the Cartesian dual-track manipulator, two “swallow” cable-driven end-effectors with a novel design [18], and the chassis mobile platform. In particular, the newly developed “swallow” cable-driven end-effector can “swallow” the strawberry targets from the bottom to the top and achieve the continuous harvesting of multiple fruits, greatly improving the harvesting efficiency of “pick one then release one” in the existing studies. Additionally, they proposed an obstacle separation control method that can grasp the ripe target while avoiding interference from leaves and unripe fruits when harvesting from naturally clustered fruits in unstructured environments [19]. To improve the accuracy and real-time performance of the aforementioned method, Xiong proposed an enhanced active obstacle separation method [20] that combines two neural networks and color thresholding for real-time strawberry detection, tracking, and localization. This method achieved a 36.8% improvement in the obstacle avoidance success rate compared to previous studies. The latest results show that Xiong’s team has developed a new six-degrees-of-freedom robotic arm [21], which has two sliders capable of moving independently along a single rail, acting as two feet. This unique design allows the robotic arm to avoid common obstacles in greenhouses, such as pipes, tables, and beams. However, the above-mentioned harvesting robots focused on standardized elevated cultivation, while research on harvesting robots that aims to improve traditional ridge-planting is lacking. Moreover, ridge-planting is the most mainstream strawberry planting pattern around the world and has the largest planting scale and the most urgent demand for automatic harvesting. Ying’s team developed a greenhouse mobile robotic platform that can monitor the growth status of the strawberry plants and fruits through its sensing system and harvest the ripe strawberries non-destructively using a pneumatic soft end-effector [22]. However, the hardware subsystems mentioned above are all commercially available products, resulting in high costs and redundant degrees of freedom in motion. Therefore, the design of a non-destructive harvesting robot, the precise localization of the fruit picking points, and the decision-making of the picking behavior have become major bottlenecks in the research of automatic harvesting.

Therefore, with a focus on the ridge-planting mode of strawberries in the greenhouse, this paper developed a novel double-arm collaboration harvesting robot. The main innovations include: (1) an oriented-ridge double-arm collaborative harvesting prototype was designed. The core components include the end-effector, which has the advantage of the fast and lossless cutting of the fruit stalk, and the self-developed manipulator that is suitable for the ridge-planting mode; (2) integrated software modules such as a strawberry target visual perception system, a double-arm collaborative control system and a four-legged walking control system, which realized the automatic harvesting of ripe strawberries in an unstructured environment; and (3) multiple rounds of field trials and rich testing data were carried out to evaluate the performance of the robot designed in this paper and confirmed the next optimization direction.

2. Materials and Methods

2.1. Composition of the Ridge-Planting Strawberry Harvesting Robot

To adapt to the low and narrow environment of ridge-planting and greatly improve the operation efficiency of the harvesting robot, an oriented-ridge double-arm cooperative harvesting robot was designed. To avoid a collision between the machine and the ridge surface, as well as to adapt to the difference in the ridge size in various strawberry fields, the working width and height of the machine were made to be adjustable. The mechanical structure of the newly developed machine is shown in Figure 1. The core components of the harvesting actuator are the end-effector and the self-developed manipulator.

2.2. The Novel End-Effector

A novel non-destructive harvesting end-effector for strawberries is proposed in this paper, which is mainly used to cut and hold the stem of the strawberry quickly. The mechanical structure of the end-effector is shown in Figure 2. The main bearing parts of the end-effector (base, motor bracket, etc.) are made of an aluminum alloy, and the fingertips, the space cam, and the connector to the manipulator are made of nylon by 3D printing. The end-effector has three innovations: (1) “hand-eye” servo control approaching and cutting. The “palm” of the end-effector is equipped with a USB camera (Jie rui wei tong DF500, China, resolution: 640 × 480) to detect and align with the strawberry targets and servo guide the end-effector closer to the target. A pair of laser-to-laser sensors are mounted on the fingertips, with one side firing the laser and the other receiving it. The fingertip is fitted with a surgical blade that cuts the stem when it is closed; (2) mechanical energy storage drives the opening and closing of the fingertips. The micro-DC brush motor (Lingkong, China) drives the space cam to rotate, and a pair of metal firing pins and a pair of metal contacts are adopted to build a simple and efficient start–stop signal marker for the maximum opening range. When the space cam rotates to the maximum range, the GPIO of the controller will receive a high-level signal and stop rotating. The spring steel sheet on both sides will be in a deformed state of energy storage. When the stem triggers the laser sensor of the fingertips, the space cam rotates, with the contour decreasing. The spring steel will drive the fingertips to close rapidly; and (3) the collision protection for the fingertips. When the end-effector reaches forward to grip the fruit, it may collide with the ridge wall due to the errors of visual perception and joint control. So, a collision protection system was designed to avoid the impact damage of the fingertips. When the force is applied in front of the fingertips, the slider moves back to trigger the collision signal so that the instantaneous impact force can receive timely feedback.

2.3. The Self-Developed 6-DOF Manipulator

According to the growth characteristics of the ridge-planting strawberry, a 6-DOF “P-R-R-P-R-R” manipulator was designed in this paper, as exhibited in Figure 3a. The manipulator is composed of two prismatic joints and four rotating joints. Joint 1 is responsible for translating the whole arm left and right along the moving direction of the robot, which is used for the searching and targeting of the fruit target. Joint 2 is used to realize the overall rotation of the mechanical arm. After harvesting fruit, the end-effector is controlled to rotate and place the fruit backwards through the rotation action of joint 2. Joint 3 is connected with two rods, forming a rotation angle to adjust the end-effector’s forward height. Joint 4 drives the end-effector through the rod to realize the forward-grabbing action. Joint 5 is the wrist swing joint of the end-effector, which can cooperate with the hand–eye vision module to search for the target fruit. Joint 6 is a rotating joint at the wrist of the end-effector, which is used to adjust the end-effector’s forward posture to pick fruit targets with different inclines. Each joint of the manipulator is driven by a planetary deceleration DC motor, and a magnetic switch is set for the joint calibration.

To solve the kinematics formula of the manipulator, a 3D coordinate system of the 6-DOF manipulator is established (Figure 3b). The motion direction of the robot is the X-axis, parallel to the ground facing the ridge is the Y-axis, and perpendicular to the ground downward is the Z-axis. The forward kinematics solution Formula (1) and the inverse kinematics solution Formula (2) for the specialized manipulator proposed in this paper are shown as follows:

$$\begin{array}{cc}\hfill X=& J_1-Z_3\times {\mathit{cosJ}}_2+(J_4+Z_4)\times {\mathit{sinJ}}_2\times {\mathit{cosJ}}_3\hfill \\ \hfill & + Z_5\times {\mathit{sinJ}}_2\times \mathit{cos}(J_3+J_5)\hfill \\ \hfill Y=& Z_3\times {\mathit{sinJ}}_2+(J_4+Z_4)\times {\mathit{cosJ}}_2\times {\mathit{cosJ}}_3\hfill \\ \hfill & + Z_5\times {\mathit{cosJ}}_2\times \mathit{cos}(J_3+J_5)\hfill \\ \hfill Z=& Z_2-(J_4+Z_4)\times {\mathit{sinJ}}_3-Z_5\times \mathit{sin}(J_3+J_5)\hfill \\ \hfill A=& J_2\hfill \\ \hfill B=& J_3+J_5\hfill \\ \hfill C=& J_6\hfill \end{array}$$

(1)

$$\begin{array}{l}{J}_1=(X\times \mathit{cos}A+Z_3-Y\times \mathit{sin}A)/\mathit{cos}A\\ J_2=A\\ J_3=\mathit{arctan}{\displaystyle \frac{(Z_2-Z-Z_5\times \mathit{sin}B)\times \mathit{cos}A}{Y-Z_3\times \mathit{sin}A-Z_5\times \mathit{cos}B\times \mathit{cos}A}}\\ J_4={\displaystyle \frac{Z_2-Z_5\times \mathit{sin}B-Z}{\mathit{cos}(\frac{(Z_2 - Z - Z_5 \times \mathit{sin}B) \times \mathit{cos}A}{Y - Z_3 \times \mathit{sin}A - Z_5 \times \mathit{cos}B \times \mathit{cos}A})}}-Z_4\\ J_5=B-\mathit{arctan}{\displaystyle \frac{(Z_2-Z-Z_5\times \mathit{sin}B)\times \mathit{cos}A}{Y-Z_3\times \mathit{sin}A-Z_5\times \mathit{cos}B\times \mathit{cos}A}}\\ J_6=C\end{array}$$

(2)

where X, Y, and Z represent the 3D coordinate values of the fingertips and A, B, and C represent the rotation angles of the fingertips around the X, Y, and Z axes, respectively. J₁–J₆ represent the rotation angles of joints 1–6 and Z₂–Z₅ represent the sizes of the robotic arms. Figure 3c displays the motion range of the manipulator. The radius of the two hemispheres is 60 cm, and the length of the cylinder in the middle is 50 cm. The hardware parameters of each joint are shown in

Table 1. Motor parameters of each joint on the 6-DOF manipulator.

Joint Serial Number	Rated Voltage/V	Rated Torque/N·m	Maximum Allowable Torque/N·m	Output Power/w	Rated Power/w	DC Motor Model (Lingkong, China)
Joint 1	24	3	10	10	12.5	M36GXR
Joint 2	24	30	90	9	12.5	M36GXR
Joint 3	24	30	90	9	12.5	M36GXR
Joint 4	24	3	10	6.3	7.2	M28GXR
Joint 5	24	25	75	5.5	7.2	M28GXR
Joint 6	6	1.5	7.5	0.35	0.78	JGA12-N20B

.

2.4. Vision Perception System

A strawberry detection method based on the lightweight Mask R-CNN [23] that has been described in detail in a previous study [3] has good performance and effect on the fruit detection of multi-fruit adhesion, overlap, and occlusion under different light-intensity environments. The instance segmentation of strawberry images is shown in Figure 4. The average accuracy rate, recall rate, and MIoU are 95.78%, 95.41%, and 89.85%, respectively. It can process 12 images per second, which can meet the real-time demand of the embedded control terminal.

A localization method for the picking point based on the visual attention in the ROI was proposed to detect the picking points and locate them in a 3D space using an Intel RealSense D435i depth camera. First, the ROIs of the picking points were screened out through the strawberry contour, and then the alternative lines of the fruit stem were obtained through Hough line detection for the binary ROI image. Second, the line of the fruit stem was determined by combining the slope difference between the fruit axis line and the alternative lines as the constraint condition. Finally, the center point of the stem line segment was selected as the fruit picking point. The main causes of localization error are stem bending and fruit malformation. The detection and localization process of the picking point are shown in Figure 5.

After several image calibrations, the adaptive sizes of the stem ROI were set as the following: length Roi_L = 0.8 W, height Roi_H = 0.4 h, and regional center (x_i, y_i-0.7 h), where x_b, y_b, w, and h, respectively, represent the abscissa of the center point, the ordinate of the center point, the width of the ROI, and the height of the ROI. The coordinate values P_ia (x_ia, y_ia) and P_ib (x_ib, y_ib) of the two endpoints of all the detected line segments were recorded, and the extension line with the shortest “point-line” distance was selected as the fruit stem line. Finally, the center point, P, of the selected line segment was set as the fruit picking point.

2.5. Integrated Architecture of the Harvesting Control System

The control system architecture of the ridge-planting strawberry harvesting robot developed in this paper is presented in Figure 6, which mainly includes three core modules: the central control system module, the visual perception system module, and the communication control system module. The hardware electronic components of the control system include the executive controllers, the embedded artificial intelligence controllers, RGB-D depth cameras, DC servo motors, mechanical touch switches, and laser sensors.

The control system of the ridge-planting strawberry harvesting robot adopts a multi-threaded parallel computing architecture, divided into sub-thread modules including “fruit picking point visual perception”, “manipulator path planning control”, and “chassis automatic walking” [24]. Each sub-thread maintains communication with the main controller through the CAN bus. To adapt to the narrow and complex operating environment of ridge planting, our team’s previous research [25] established a 6-DOF manipulator dynamic simulation model constrained by harvesting environment models. The motion stroke and power consumption information of each joint were analyzed during harvesting and simulated paths and motion cycles for the harvesting process were set up. Test results showed that compared to the S-curve motion mode, under the uniform acceleration-constant speed-uniform deceleration motion mode, each joint of the robotic arm moved more smoothly with lower power consumption.

During a harvesting action cycle, the machine control system obtains the world coordinates of the target fruit and calculates the amount of motion of each joint through the kinematic analysis. Finally, each joint motor is controlled by CAN commands to operate. Compared with the commercial-grade manipulator products, the special manipulator designed in this paper is more suitable for the strawberry-harvesting environment. It not only has a significant advantage in cost but also is more convenient regarding the control interface, which can be iteratively upgraded according to future research and development needs.

3. Results and Discussion

3.1. Experimental Setup

A field evaluation of the newly developed strawberry harvesting robot in several ridge-planting strawberry greenhouses in Changping District, Beijing (Greenhouse A) and Xuzhou City, Jiangsu Province (Greenhouse B), respectively, was carried out from 13–15 January and 13–16 March 2024. Among them, greenhouse A has been thinning the flowers and fruits of strawberries, and greenhouse B has maintained the original growth state. The main growth states are shown in Figure 7.

As shown in Figure 7, after the thinning of flowers and fruits, the growing environment of strawberries is greatly simplified, reducing the mutual occlusion of multi-fruits or stems. The thinning of flowers and fruits allows the ripe strawberries to obtain more nutrients, and grow larger, and the stems are longer and stronger.

3.2. Field Harvesting Process

In each round of the harvesting experiment, two groups of manipulators operate simultaneously, and their visual perception modules are responsible for searching for the ripe strawberries. When the target fruit is found, the visual perception system sends the 3D coordinates to the central control system, and the end-effector is controlled to approach the target continuously through harvesting priority judgment. The hand–eye vision system detects the optimal picking point of the target fruit in real time. When the laser sensor is triggered by the target fruit stem, the fingertips are closed immediately, and the stem is cut. Finally, the fruit is transferred to the fruit storage container. The harvesting action sequence is shown in Figure 8.

3.3. Experimental Results and Discussion

3.3.1. Harvesting Success Rates Under Different Natural Conditions

One successful harvesting is defined as when the stem is cut off successfully, then clamped and transported to the fruit container without any surface damage. If the stem of the target fruit is not cut successfully in the first picking, the end-effector will pick it two or three times. If the fruit cannot be picked successfully after three times, the fruit will be abandoned and the next target fruit will be targeted. The harvesting process of each fruit is mainly divided into three stages: the location of the picking point, cutting-grabbing, and transporting-releasing the fruit. In order to discuss the influence of the different stages on the harvesting success rate, the experiment recorded the harvesting success rates in two greenhouses, respectively, and the results are shown in

Table 2. Harvesting success rates in two greenhouses.

Growth State	Fruit Number	Success Rate (Success Number of Fruit) in Each Harvesting Process			Harvesting Evaluation
		Detection	Cutting- Clamping	Releasing	Damage Rate (Fruit Number)	Harvesting Success Rate (Fruit Number)
A	213	80.28% (171)	66.20% (141)	61.97% (132)	12.68% (27)	49.30% (105)
B	215	70.70% (152)	57.21% (123)	49.77% (107)	19.53% (42)	30.23% (65)

.

As shown in Table 2, the success rates in each harvesting process of Greenhouse A were higher than those of B. The success rate of non-destructive harvesting in Greenhouse A was 19.07% higher than in B, indicating that thinning the flowers and fruits could greatly reduce the harvesting difficulty and provide great convenience for automatic harvesting.

Table Note: The main failure reasons in harvesting: (1) the target fruit is not successfully recognized by the visual perception system; (2) the localization error of the picking point; (3) the stem is thick and cannot be cut successfully; (4) the stem is completely covered and cannot be cut; (5) the infrared sensor on the fingertips is accidentally triggered; (6) the unripe fruits adjacent to the target fruit are harvested together; (7) the clamping force of the end-effector is insufficient; and (8) the cut stem is too long and wobbles during transport.

Through the harvesting experiments in Greenhouses A and B with different planting states, this study analyzed the failure causes in each harvesting process. A comparison of the effects of thinning the flowers and fruits on the harvesting performance was conducted, which is shown in

Table 3. Failure reasons in each harvesting process.

Growth State	Each Harvesting Process	Failure Reasons	Probability Ratio of Failure Causes/%
A	Detection	(1), (2)	42.6; 57.4
	Cutting-clamping	(3), (4), (5), (6)	41.2; 10.6; 32.5; 15.7
	Releasing	(7), (8)	34.7; 65.3
B	Detection	(1), (2)	21.6; 78.4
	Cutting-clamping	(3), (4), (5), (6)	7.5; 35.3; 31.7; 25.5
	Releasing	(7), (8)	70.2; 29.8

. There are two main reasons for the identification errors of the picking points: (1) the visual perception system does not recognize the covered ripe strawberries (Figure 9a), so it cannot identify and locate the picking points and (2) complex natural factors, such as large curvature of the stem (Figure 9b), will cause the location errors. And the other leaves or stems around the target stem also interfere with the effect of Hough linear fitting. After thinning the flowers and fruits, the complex environmental interference near the target fruit stem is reduced, and the predicted picking point is more accurate.

In Greenhouse A, 141 of 171 visually detected strawberries were successfully clipped, with a two-stage success rate of 66.2%, while Greenhouse B had a success rate of only 57.21%. There are two main reasons for the failure of cutting-clamping of the stem. Firstly, the fruit stem is thicker after thinning the flowers and fruits, resulting in the end-effector not cutting the stem smoothly. This problem can be effectively solved by improving the mechanical structure of the fingertips. Secondly, because the infrared signal is triggered for judging the stem entering the fingertips, if there is interference from other stems or leaves near the picking point of the target stem, it may trigger in advance and the fingertips will not reach the target stem (Figure 9c). In addition, the fingertips cannot approach the target stem due to the occlusion of leaves and immature fruits. The above failure occurred more frequently in Greenhouse B, indicating that thinning the flowers and fruits has a positive significance for automatic strawberry harvesting.

During the transport and release stage after cutting, the fruit may fall off. The main reason is that multiple fruits grow next to each other, and the end-effector may pick the target fruit and the adjacent fruit together, resulting in increased weight on the fingertips. Shaking of the fruit during transportation means that the fruit is prone to falling. Even if the above three harvesting processes can be successfully completed, skin damage to the target fruit may be caused during the entire operation. The main cause of damage is that the two ripe fruits are relatively close together (Figure 9d), and the opening size of the fingertips exceeds the distance between the two fruits, which causes abrasions on the adjacent fruit.

Therefore, the ridge-planting strawberry harvesting robot designed in this paper possesses a low success rate in harvesting strawberries whose stems are severely blocked. By artificially removing some of the excessive flowers, young fruits, stems, and leaves, the growing environment can be effectively simplified, and the probability of the fruit being blocked is greatly reduced, which provides great convenience for automatic harvesting.

3.3.2. Harvesting Speeds with the Single-Arm and Double-Arms

To compare and evaluate the harvesting efficiency with the single-arm and double-arms, the harvesting speeds were calculated according to the videos from multiple cameras. The average harvesting speed is equal to the total time it takes to harvest several fruits divided by the number of fruits. Four independent strawberry rows (R1–R4) were selected in the same greenhouse. Each row was 10 m in length but had a different number of fruits and growth densities. In addition, each ridge was equally divided into two zones referred to as a and b (R1a–R4a, R1b–R4b), respectively, for the single-arm and double-arm comparison experiments. When evaluating the performance of the single-arm harvesting, the strawberry fruits on the left side of the ridge were emptied, and the whole harvesting process by the right arm of the robot was filmed by a monitoring camera. The harvesting speed of the harvesting robot was calculated by counting the total time to complete the harvesting of each ridge and the total number of fruits successfully harvested. The detailed harvesting results are shown in

Table 4. Harvesting time at different fruit growth densities.

Number of Arms	Ridge No.	Number of Fruits	Total Harvesting Time/s	Single Fruit Harvesting Time/s	Average Harvesting Time/s
Single-arm	R1a	75	307.5	4.1	7.3
	R2a	58	307.4	5.3
	R3a	32	230.4	7.2
	R4a	15	189.0	12.6
Double-arms	R1b	142	383.4	2.7	4.7
	R2b	118	365.8	3.1
	R3b	74	347.8	4.7
	R4b	26	210.6	8.1

.

It can be seen from Table 3 that the machine takes different amounts of time to harvest the fruits with the same ridge length and different growth densities. The larger the fruit growth distribution density is, the shorter the harvesting time of a single fruit and the faster the average harvesting speed. The spatial distance between fruits with high growth distribution density was shorter, and the searching time of the visual perception system and the movement time of the manipulator were relatively short, so the average harvesting time of a single fruit was shorter. The results show that the average harvesting time of a single fruit with the single arm was 7.3 s and that with the double arms was 4.7 s, though it did not decrease proportionally. If the density of the fruit on either side of the row is different, the robot chassis will move forward to the next harvesting area only after the arm on the denser side has picked all the strawberry targets in the current vision field.

The above experimental results show that the robot designed in this paper can achieve a good harvesting effect under the condition that the stem is not completely blocked. The average harvesting time of a single fruit with one arm and double arms was 7.3 s and 4.7 s, respectively, which is slightly slower than the strawberry harvesting robot designed by Xiong et al. (6.1 s and 4.6 s) [17]. However, the robot designed by Xiong et al. is oriented to elevated strawberry cultivation, whose growing environment is highly structured and the difficulty of harvesting is lower than in ridge-planting cultivation. Moreover, the harvesting experiments in this paper verified that the multi-fruit harvesting priority decision-making method and the manipulator path planning algorithm could effectively improve the harvesting success rate, shorten the harvesting operation time, and reduce the risk of fruit damage.

4. Conclusions

By combining the agronomic characteristics of the ridge-planting strawberry, this study developed an oriented-ridge double-arm cooperative harvesting robot. The robot is equipped with an innovative non-destructive harvesting end-effector, two self-developed 6-DOF manipulators, and a strawberry picking point detection system based on the lightweight Mask R-CNN, which has good performance and effect on the fruit detection of multi-fruit adhesion, overlap, and occlusion under different light intensity environments. Through the integration of the above core components and a vision system under the robot control architecture, cooperative double-arm harvesting of the ridge-planting strawberry can be achieved automatically. The greenhouse harvesting experiments showed that the strawberry harvesting robot developed in this paper achieved a success rate of 49.3% in strawberry growing environments after flower and fruit thinning, while only a 30.23% success rate was achieved in natural strawberry growing states. The agronomic practice of flower and fruit thinning significantly simplifies the operating environment for the harvesting robot and improves the automatic harvesting success rate. The causes of harvesting failures under different harvesting environments were also analyzed. Additionally, harvesting experiments measured that the average harvesting efficiency for single-arm and double-arm cooperative harvesting were 7.3 s and 4.7 s per fruit, respectively, indicating that expanding to multi-arm cooperative harvesting in the future could significantly improve the harvesting efficiency.

It should be noted that there are still some defects and deficiencies in the present study. Firstly, the fruit target visual perception system can only recognize ripe and unripe fruits and lacks the fine-grained classification of fruit ripeness and variety quality. Secondly, temporary storage space for fruits will be added to the robot structure, and a novel end-effector for continuous harvesting will be designed. Furthermore, the operation mode of “pick one then release one” will be upgraded to “pick multiple fruits and release once”, further improving the harvesting efficiency.