Advances in Agriculture Robotics: A State-of-the-Art Review and Challenges Ahead

Abstract

The constant advances in agricultural robotics aim to overcome the challenges imposed by population growth, accelerated urbanization, high competitiveness of high-quality products, environmental preservation and a lack of qualified labor. In this sense, this review paper surveys the main existing applications of agricultural robotic systems for the execution of land preparation before planting, sowing, planting, plant treatment, harvesting, yield estimation and phenotyping. In general, all robots were evaluated according to the following criteria: its locomotion system, what is the final application, if it has sensors, robotic arm and/or computer vision algorithm, what is its development stage and which country and continent they belong. After evaluating all similar characteristics, to expose the research trends, common pitfalls and the characteristics that hinder commercial development, and discover which countries are investing into Research and Development (R&D) in these technologies for the future, four major areas that need future research work for enhancing the state of the art in smart agriculture were highlighted: locomotion systems, sensors, computer vision algorithms and communication technologies. The results of this research suggest that the investment in agricultural robotic systems allows to achieve short—harvest monitoring—and long-term objectives—yield estimation.

1. Introduction

There are currently 193 countries that have officially joined the United Nations (UN), each with its particular problems. However, according to the UN, all of these countries must pay attention to a common problem, global population growth. The planet Earth has about 7.6 billion people and it is estimated that by 2050 the world population will increase to 9.8 billion people, an increase of 28.94%, with half of the world’s population growth being concentrated in just nine countries: India, Nigeria, the Democratic Republic of the Congo, Pakistan, Ethiopia, the United Republic of Tanzania, the United States of America, Uganda and Indonesia [1]. The population growth challenges farmers to make changes in the form of control, monitoring and management of their farms to meet the growing demand for food (requiring double the current food production capacity for 2050 [2]) of high quality because more and more people are looking for healthier foods, without the addition of herbicides and pesticides [3]. On the other hand, the global urbanization process is transforming rural landscapes in urban, causing 68% of the population to reside in urban environments by 2050 [4]. As a result, rural producers are looking for new ways to produce their food in increasingly smaller environments, since in 1991 the percentage of world arable land was around 39.47% and in 2013 it was around 37.7%, that is, a 1.77% reduction in the availability of arable land [5].

1.1. Global Socio-Economic Challenges in Times of the Pandemic

Agricultural activities still rely heavily on human labor, which in turn is subject to health problems such as the worldwide public health crisis generated by the Coronavirus pandemic (COVID-19), which in addition to causing a large number of deaths around the world (about 2,527,891 confirmed deaths until 01/03/2021, according to the World Health Organization (WHO) [6]), imposed several forms of restriction of social and economic activities. In that sense, poor countries that are heavily dependent on food provided by small farmers, livestock farmers and artisanal fishermen will be most impacted by the pandemic. According to the Food and Agriculture Organization (FAO), the social isolation measures imposed by COVID-19 prevent farmers from accessing input and product markets by increasing post-harvest losses [7]. On the other hand, in rich countries, such as the United States, because agricultural tasks are considered “hard work” and low profitability, youngsters are looking for job opportunities in urban regions, so farmers are looking for new ways to automate their farms and thus recover any losses [8].

1.2. Precision Agriculture

Fortunately, to overcome the challenges imposed by population growth, accelerated urbanization, high competitiveness of high-quality products, lack of qualified labor and the vulnerability of human labor to situations of health risk, scientific advances in different areas of human knowledge are transforming the way of managing agricultural activities, increasingly reducing human intervention. According to [9], the term Precision Agriculture (PA) can be defined as “that kind of agriculture that increases the number of (correct) decisions per unit area of land per unit time with associated net benefits”. This definition is more generic so that decision-making can be done either by electronic devices or humans. On the other hand, in [10] the PA is described as being “a management strategy that uses electronic information and other technologies to gather, process, and analyze spatial and temporal data for the purpose of guiding targeted actions that improve efficiency, productivity, and sustainability of agricultural operations”. This definition clearly describes the use of technologies as a means of improving agricultural operations. In this review work, these technologies were grouped into three main areas: robotics, Artificial Intelligence (AI) and Internet of Things (IoT). These technologies can be used either alone, or together, according to Figure 1.

PA has gained prominence with the use of robotic systems and electronic devices in agricultural tasks such as: land preparation, sowing, planting, pest control and harvesting [11]. The market value of Precision Agriculture in 2016 was estimated at $3.67 billion and is estimated to be growing at a rate of 14.7%, to reach $7.29 billion by 2021 [12].

As seen in Figure 1, although this review will address the use of robots in agriculture, often the technological areas of AI and IoT are part of the subsystems of an application involving the use of robots for the execution of agricultural activities. According to Zha, who conducted a review of the use of AI in agriculture, AI can be used in soil management, weed management and in cooperation with IoT technologies [13]. He describes that computer vision algorithms, such as Deep Belief Networks (DBN) and Convolution Neural Network (CNN), have promising results in fruit classification and weed detection in complex environments, that is, with varying ambient lighting, the complexities in the background, the angle when capturing and the variation of the shapes and colors of the fruits/weeds. Elijah et al. carried out a review work on the benefits of using IoT and data analytics in agriculture [14]. According to them, IoT technologies make it possible to monitor farms through sensors of different types, such as optical, mechanical, electrochemical, dielectric soil moisture and location. Due to the existence of short/long-range communication technologies, these sensors work as a data source for prediction, storage management, decision, farm management and precise application algorithms. The advantages of using IoT in agriculture include: safety control and fraud prevention, competitive advantages, wealth creation and distributions, cost reduction and wastage, operational efficiency, awareness and asset management. As open challenges, the authors highlight the need for technological innovations, realization of applications in real large-scale scenarios (pilot project) and cost reduction, standardization and regulation of IoT technologies that facilitate their use in agriculture. Another review work on IoT-based smart agriculture, which also describes the use of several electronic sensors in the improvement of agricultural control and monitoring tasks, is described in [3]. One aspect in common between the survey works mentioned above is that all report the use of robots as tools of technological improvement of agricultural activities. The performance of robots in agriculture depends not only on the type of crop, but also on the type of task that the robot is intended to perform. In this sense, there are review works that address the use of robots in agriculture to perform general tasks, as in [15,16,17], or to improve the performance of specific tasks, such as harvesting high-value crops [18] and solving wheeled mobile robots’ navigation problems [19].

Bac et al. analyzed harvesting robots in the period between 1984 and 2014 [18], Oliveira et al. studied about 21 robots [15] and Fountas et al. carried out a systematic review on various agricultural tasks [16]. Therefore, this review article contributes to an updated analysis of the works carried out until 2021, expanding the analysis made by Bac et al. [18], explores new robotic systems, expanding the results obtained by Oliveira et al. [15], and suggests new challenges for future research work, complementing the conclusions obtained by Fountas et al. [16].

Bearing in mind these ideas, the rest of the work is organized as follows: Section 2 describes the materials and methods adopted as a source of research in this review work. The main robotics applications in agriculture are described in Section 3, presenting a critical analysis of each of the tasks performed by robots in such environments. In Section 4 it is discussed which are the locomotion, vision and navigation systems most used in the reviewed works, unresolved problems and proposals for new research work. Finally, Section 5 presents the conclusions reached by its authors in this review paper.

2. Materials and Methods

The materials used as a source of data and scientific information for this review article were: Google Scholar, Scopus, IEEE Xplorer, Wiley, SpringerLink and Web of Science. For the case of robotic systems in the commercial stage, the search was made through the analysis of their datasheets and commercial websites. A total of about 62 works/projects of robotic systems applied in the agricultural area were counted at the end of the information search process.

In general, all robots were evaluated according to the following criteria: its locomotion system, what is the final intended application, if it has sensors, robotic arm and/or computer vision algorithm, what is its development stage and to which country and continent they belong. These evaluation criteria were chosen in order to highlight the main technologies used to perform each agricultural task and to highlight which are the countries and continents that have the largest number of agricultural robotic systems, in order to correlate the use of such systems with the real needs of the agricultural market in each region.

3. Robotic Applications in Agriculture

The various agricultural activities were grouped into subsections: land preparation before planting, sowing/planting, plant treatment, harvesting and yield estimation and phenotyping. Therefore, the following subsections will address the various types of robotic systems applications in individual agricultural environments.

3.1. Robotic Applications in Agriculture for Land Preparation before Planting

Preparing the land before planting is one of the first agricultural tasks to be carried out, such as plowing the land and applying fertilizers. The practice of plowing the land (inversion of soil layers) allows a greater introduction of oxygen and expulsion of carbon dioxide; however, depending on the local climate conditions, it can harm future crops by considerably reducing the soil’s carbon stocks. On the other hand, soil fertilization serves to replace the nutrients necessary for the development of crops. The review article [20], published in 1987, refers that one of the main challenges in the development of robots that operate in rough terrain, as in a plowed field, is the creation of a robotic system that is precisely controlled. To create a robot capable of assisting rural workers not only in soil fertilization but also for pest control, soil management, harvesting and transportation, the German company Raussendorf developed, in 2014, the robot entitled Cäsar, as shown in Figure 2a. The commercially available Cäsar robot [21] can fertilize the soil in two ways: via remote control or autonomously. To perform tasks automatically, it uses Real-Time Kinematic (RTK) technology for the Global Navigation Satellite System (GNSS) (GNSS is the navigation device, however, it can use different services, such as Global Positioning Systems (GPS) (North American), GLONASS (Russian) or GALILEO (European) [22,23]), resulting in a location accuracy of up to 3 cm. Designed to work on the farm together with human workers, the Cäsar robot has a collision detection system based on ultrasonic sensors that guarantee its immediate stop, with a maximum detection distance of 5 m.

The Greenbot robot (shown in Figure 2b), also available commercially [24], is capable of carrying out the tasks of fertilizing, plowing or seeding throughout the day. It has a Four-Wheel Steering (4WS) system and, through its 100 HP diesel engine, it is capable of transporting up to 750 kg and 1500 kg in its front and rear compartments, respectively. Like the Cäsar robot, the Greenbot also has collision detector sensors to detect objects ahead and make emergency stops [24].

Unlike the terrestrial robots Cäsar and Greenbot, the Chinese company DJI developed an Unmanned Aerial Vehicle (UAV) to carry out agricultural activities. The ability to fly over farms eliminates the interference of any rocks, holes, elevations and branches during the course of the UAV, as these are terrestrial obstacles. However, UAV can collide with branches of high vegetation or high voltage wires (power lines), have flight time limited by battery power, in addition to its trajectories being strongly affected by rain and winds. In this sense, the improvement of the energy consumption of UAV and, therefore, its flight time, enabled its greater efficiency in carrying out agricultural tasks without contact with the soil. In general, the greater the number of rotors in a UAV, the greater its load and control capacity. Thus, in 2016 DJI developed the octocopter AGRAS MG-1P to carry out the precise application of liquid fertilizers, pesticides and herbicides. With the capacity to transport up to 10 l of payload over a maximum distance of up to 3 km, it has a spraying capacity of 6 ha/h and allows to control up to 5 UAV with a single remote control. To avoid collisions with high voltage wires or high vegetation, it has an anti-collision system using an omnidirectional radar with a maximum detection distance of up to 15 m. To carry out the spraying precisely, it presents the integration between the RTK GPS and an Inertial Measurement Unit (IMU) (gyroscope, accelerometer and compass), guaranteeing an accuracy of 1 cm + 1 ppm. The UAV can remain flying steadily even if there is a malfunction in one of its rotors, because it has propeller rotor redundancy [25].

Unlike the robots mentioned above, the AgBot robot (seen in Figure 2c) is still in the research stage. With a Two-Wheel Drive (2WD) system, the robot was developed to apply fertilizer and herbicide on a corn farm. The AgBot robot has four distinct reservoirs, allowing for an exclusive reservoir for each type of herbicide and/or fertilizer. Its control and navigation system consists of components and platforms for the development of low-cost embedded systems (Arduino and Raspberry Pi). Through a low-cost Red Green Blue (RGB) camera and using the Haar feature-based cascade classifiers machine learning algorithm, the robot is able to detect three different types of weeds commonly found in corn farms: Giant ragweed, Redroot pigweed and Cocklebur. The authors conclude that, although the weed detection algorithm was able to identify three different species, the low-cost RGB camera used proved to be unsuitable for external use, requiring future research works [23].

Table 1. Comparison between the revised robotic applications for land preparation.

Robots	Locomotion System	Final Application	Navigation Sensors	Obstacle Detection Sensors	Development Stage	Year
Cäsar [21]	4WD	Orchard or vineyard	RTK GNSS	Ultrasonic sensor	Commercial	2014
Greenbot [24]	4WS	Horticulture, fruit and arable farming	RTK GPS	Bump sensor	Commercial	2015
AGRAS MG-1P [25]	UAV Octocopter	Rice, soy and corn	RTK GPS, RGB camera, gyroscope, accelerometer and compass	Omnidirectional radar	Commercial	2016
AgBot [23]	2WD	Corn	RTK GPS, RGB camera, compass and accelerometer	–	Research	2017

summarizes the main information from the aforementioned works.

As it is considered a semi-structured environment, to navigate the entire farm with precision in its positioning data, all the robots previously discussed present the integration between the RTK system and the GNSS. Therefore, in the aspect of control of robots in real agricultural environments, it is noted that, through the use of RTK/GNSS technologies, there has been a great improvement in the control of location and positioning of robots, compared to the review article published in 1987 [20], since RTK technology emerged in the mid-1990s. Figure 3 exemplifies the difference between the obstacle detection systems of the robots listed in Table 1.

3.2. Robotic Applications in Agriculture for Sowing and Planting

Traditionally, sowing and planting tasks are performed by specialized planting equipment connected behind the tractor. However, tractors are characterized by being heavy machinery and, therefore, their constant locomotion throughout the farm intensifies soil compaction. The soil compaction activity has several damaging effects for the agricultural environments, such as increasing the apparent density and soil resistance, reducing porosity, the rate of water infiltration and aeration, in addition to affecting the chemical properties and biogeochemical cycles, affecting the development of plants and soil biodiversity [26]. In 1996, Sakaue was already developing robotic systems to automate the sowing and planting process in Japan [27], with the elaboration of four variations of a wheeled robot. Despite having a rudimentary structure, prototype number four was capable of planting cabbage, lettuce, broccoli, cauliflower or celery of up to 2200 plants/h. However, the system was not able to move autonomously.

Motivated to produce a small, high precision and capable of agile and easily moving along Chinese wheat farms, Haibo et al. developed the Lumai-5 robot, as shown in Figure 4a. For this type of task, the robot must ensure that the sowing process remains constant, even for different speeds of detachment. The Lumai-5 robot has a 4WS, a closed-loop control system and speed, angle and pressure sensors to accurately carry out wheat sowing. The size of the planting tray, the vacuum chamber pressure and the planting speed were the main factors that directly affected the seeding quality [28].

The Australian Center for Field Robotics (ACFR) at the University of Sydney started, in 2015, the development of the Digital Farmhand project. With the clear objectives of developing a robotic system with the concept of off-the-shelf, both digital and physical, the Australian researchers created the Di-Wheel robot (depicted in Figure 4b). The Di-Wheel concept consists of a 2WD robot that supports and moves only under two wheels, reducing the size, weight and mechanical complexity of the robot, in addition to simplifying its transport and assembly (about 15 min). The robot, whose distance between the wheels can vary to adjust to different crop types, was designed to perform the tasks of precision seeding (depicted in Figure 5), spraying and weeding, with all electronic devices located in its central part. Since the financial cost is the biggest barrier that prevents the use of robotic systems by small producers, the Di-Wheel has support to fix smartphones at a height and angle that enables the use of the smartphone’s internal sensors, such as temperature, light and humidity sensors, RGB cameras, as well as gyroscopes, accelerometers and GNSS devices. The author concludes that, due to constant advances in the field of computing, the use of open-source machine learning algorithms (OpenAI, TensorFlow and PaddlePaddle) can contribute to better crop control and management [29].

Unlike tractors (heavy vehicles) that are used to work on large-scale farms, small and low-cost robots can be used on small farms. As shown in Figure 4c, a 4WD seeding robot under development in Pakistan was used to sow corn using an individual seed selector, capable of distributing the number of seeds suitable for planting. According to the authors, the prototype covers sowing 5 times faster than the conventional way of sowing, at a sowing rate of 90 seeds/min, totaling a cycle of 0.66 acres/h [30]. To maximize the robot’s weight versus soil compaction ratio, Indian researchers presented in 2016 a prototype of a small-sized seed drill robot with the capacity to transport a reservoir with up to 17 kg of payload, using the locomotion system with tracked drives to carry heavy loads on non-uniform soils. As it is a prototype in the initial research phase, the authors propose the use of a photovoltaic panel to recharge the robot’s electrical system, as well as the implementation of a Kalman Filter to improve the estimation of the robot’s position and location along with the crop [31].

Table 2. Comparison between the revised robotic applications for sowing and planting.

Robots	Locomotion System	Final Application	Guidance Sensors	Seeding Mechanism	Development Stage	Year
Lumai-5 [28]	4WS	Wheat	Angle and speed	Seeding motor and vacuum fan	Research	2010
Di-Wheel [29]	2WD	Horticulture in general	Smartphone embedded sensors	Roll type seeder	Research	2015
Sowing robot 1 [30]	4WD	Corn	Ultrasonic	Linear actuator and vacuum motor	Research	2016
Sowing robot 2 [31]	Track	Seeds in general	Ultrasonic and magnetometer	Solenoid actuator	Research	2016

summarizes the works previously mentioned.

According to Table 2, all robots studied are still in the prototype/research phase and have different locomotion systems. Although the proposals bet on low-cost devices, only the Di-Wheel robot has a modular physical and digital structure, using the off-the-shelf concept to eliminate the cost of using unnecessary sensors in the robot, since the robot makes use of the embedded sensors of smartphones.

3.3. Robotic Applications in Agriculture for Plant Treatment

After finishing the sowing stage, maintaining the planting growth properly, that is, free of diseases and pests, requires constant monitoring by the farmer. According to FAO data, about 20 to 40% of world crop production is lost due to pests and diseases [32]. The good growth of crops invaded by weeds is strongly compromised and can even destroy crops completely. Weeds can also attract pests in addition to small animals like snakes and mice. Thus, the sooner it is extracted, the lower the financial losses. As an example of these costs, in Australia weed management costs are close to $4 billion a year [33]. Plant treatment is commonly done by applying herbicides and pesticides (insecticides and fungicides). The searches for the automation of the diseases’ identification process in plants and detection of weeds are not recent—there have already been works in this area since 1998. In [34] a robotic weed control system for tomatoes is described, based on the Bayesian classifier algorithm, that correctly identified 73.1% of tomatoes and 68.8% of weeds in the validation set of field images. In [35], researchers Lee and Slaughter chose to develop the hardware-based neural network, rather than the Bayesian classifier, for increasing the plant identification. With this new classification method, the robotic system correctly identified 38.9% of tomato cotyledons and 85.7% of weeds. These were just the first robotic systems applied to weed control, from a series of robots that have been developed over the years. Therefore, this subsection will address several works related to these agricultural tasks, focusing on its latest developments.

Using a 6 Degrees of Freedom (DoF) manipulator arm, an RGB camera and a laser distance sensor (DT35, SICK), all mounted on a fixed platform, Schor et al. developed a robotic system to perform the detection of powdery mildew and tomato spotted wilt virus diseases in greenhouses. The RGB camera and the laser sensor were placed on the final actuator of the manipulator (shown in Figure 6a) to extract the images from different angles and avoid collisions with the plant. The images were used in the disease detection process, using Principal Component Analysis (PCA) and Coefficient of Variation (CV). The system obtained an accuracy rate of 64% for the process of classification of plants with powdery mildew disease in an early stage of evolution and up to 90% for the case of tomato spotted wilt virus, thus enabling accurate detection of diseases in its initial stage [36]. For the same purpose, the mobile robot eAGROBOT (depicted in Figure 6b) was used to identify pests in cotton and groundnut crops. Through the application of artificial intelligence algorithms, such as artificial neural networks and K-means, in the images acquired by an RGB camera of crops in the initial sowing stage (period of the emergence of diseases such as leaf spot and anthracnose), the robot achieved a precision of 83–96% for disease identification in normal images and 89% for wide images [37].

Concerning weed control, the robots can perform the detection and its subsequent removal, through the application of herbicides in the weeds and/or through mechanical tools. A study focused on the detection of weeds in broccoli and lettuce crops was carried out using an RGB-Depth (RGB-D) Kinect v2 camera. Image processing was divided into four stages: using the Random Sample Consensus (RANSAC) technique, plant extraction (2D connected-component method), resource extraction (length, width and height of leaves, an arrangement of ribs and area) and classification of plants (based on characteristics). The authors report that through the proposed method, tested in a real environment, the robotic system (shown in Figure 6c) achieved a detection rate of 91.7% for broccoli and 90.8% for lettuce [38].

According to Jorgensen et al., weed control can be done in two ways: mechanical combined with the use of herbicides or fully mechanical, covering 90% and 10%, respectively, of the total outdoor gardening area in Denmark [39]. In this review, this concept was generalized, dividing the weed control into two ways: the use of mechanical tools and the use of chemical products (herbicides). Following this concept, in addition to detecting weeds, the Australian AgBot II robot mechanically performs the removal of weeds from crops, through the action of three types of tools: an arrow-shaped hoe, a toothed tool and a cutting tool. The AgBot II uses techniques such as Local Binary Pattern (LBP) and Covariance Feature in its image processing stage, collected by the RGB camera, to identify weeds [40].

As examples of the use of autonomous robots in weed control at a commercial-stage, one can mention the French robots Oz (Figure 6d), Dino (Figure 6e) and Ted (Figure 6f), all from the company Naio Technologies and designed for the publics of market farmers (vegetables, nurseries, horticulture), for large scale vegetable farms (vegetables in a row and on beds) and wine growers (vines–row width > 150 cm/60 inches), respectively [41,42,43]. All of these robots use mechanical tools to eliminate weeds and are powered entirely by Lithium batteries with an autonomy of 8 h (depending on the type of tool used and the soil). According to the Naio Technologies company, 70 Oz robots were sold in 2018 alone, being 80% of sales to the French internal market, 15% to European countries and 5% to the rest of the world. As they are high-tech robots (containing RTK/GPS sensors, RGB cameras and Light Detection and Ranging (LiDAR)), operating autonomously in large crops (avoiding the need for human supervision), they are all tracked and have a communication protocol for sending telephone messages (Short Message Service (SMS)), for possible theft situations.

Through mechanical cutting tools, the VITIROVER and Tertill robots (depicted in Figure 6g,h, respectively) are light, small, have photovoltaic panels in their mechanical structures, operate under conditions of sun and rain and perform the removal of weeds. Both allow the control and monitoring of robot information through a mobile application [44,45], exploring the IoT concepts. In addition to having a cutting tool, the Tertill (the first robot designed to remove weeds from residential gardens) has wheels designed to assist in weed removal.

Small robots with mechanical weeding were also used in paddy fields. Rice is grown within a field flooded with arable land, in which case both the rice seed and the weed seed grow underwater. The 4WS K-Weedbot robot (shown in Figure 6i) was developed to perform the removal of weeds while moving, guided by a high precision image processing system (uses gray image, median filter, Otsu method, noise elimination, image segmentation and K-means clustering) so that the robot does not collide with the plants. To improve the extraction of weeds, the K-Wheedbot has gears instead of wheels. Through a common RGB camera and a row detection algorithm, the robot moves around the rice field, with a maximum deviation of 1° in its trajectory [46].

The AIGAMO-ROBOT (seen in Figure 6j) was developed to be compact, powered by batteries (to prevent oil leakage and emissions of polluting gases into the atmosphere) and easy to use. It uses its tracked locomotion system to remove weeds. Thus the robot reduces the emergence of weeds both inter ranks as intra ranks [47].

Japanese researchers Sori et al. analyzed the development of paddy fields with weeds, removing weeds with herbicide application and removal through the robot (shown in Figure 6k). As its results indicate in Figure 7, the use of the robotic system together with more spaced rice crops allows several improvements for the growth and crop yield, such as bigger roots, stems, leaves, height and weight of the rice [48].

Unlike the K-Weedbot (4WS) and AIGAMO-ROBOT (track) robots, a robotic system (shown in Figure 6l) capable of floating on the surface of paddy fields was used to cause disturbances in the water, leaving it turbid to reduce the incidence of sunlight and reduce weed photosynthesis. To carry out the disturbances in the water, the researchers used chains distributed evenly and fixed on the rear of the robot. During the dragging of these chains on the surface of the soaked soil, the weed species detach themselves from the soil surface and head towards the water surface. Although the robot used is not autonomous, it was also able to reduce the emergence of weeds [49].

In addition to being a high-value product, pesticide products are harmful to human health. For a rural worker to apply the pesticide on the plants, he must wear several personal protective equipment. To relocate the rural worker who performs the application of pesticides to a safe environment, through human–machine interaction, researchers Adamides et al. developed two robots, AgriRobot and SAVSAR (Figure 6m,n), to remotely spray pesticides on vineyards using a Human Machine Interface (HMI) [50]. Another mobile robot (shown in Figure 6o) was used to reduce the amount of material sprayed and, in this case, the robot has a RGB camera and distance sensors to automatically open the pesticide spray valve based on the machine vision Foliage Detection Algorithm (FDA) and Grape clusters Detection Algorithms (GDA), resulting in a 45% reduction in pesticide material [51].

Another way to perform weed control is through the use of herbicides. The RIPPA [52] and Ladybird [53] robots were also developed to remove weeds, as shown in Figure 6p,q, respectively. RIPPA is smaller than Ladybird, but uses several of its technology. The RIPPA is a prototype for the commercial version. Unlike the AgBot II, the RIPPA and Ladybird robots, in addition to having a photovoltaic panel in their mechanical structures, perform the elimination of weeds through spraying of herbicide in the appropriate place. The RIPPA and Ladybird robots also capture hyperspectral images. Spectral information can indicate plant health (through the use of machine learning algorithms). Thus, if there is a plant labeled with a low health value, the same system will spray an appropriate amount of fertilizer. It is noted, therefore, that robotic systems that have a liquid spraying system can be used not only to remove weeds, but also to strengthen crops by spraying fertilizers.

The BoniRob robot (as shown in Figure 6r) is more complete, as it performs the tasks of detecting (through cameras and ultrasonic sensors) and removing weeds, not only using mechanical tools, but also spraying herbicide [54].

Like the RIPPA and Ladybird robots, the Swagbot robot (seen in Figure 6s) was developed by Australian researchers from ACFR. The robot was built for general purposes, to perform activities such as autonomous weed identification and spraying, soil and pasture analysis, biomass estimation and livestock monitoring [55]. The objective of developing robots with a wide variety of applications is to be able to establish a process of standardization and modularization of robotic systems.

To navigate the farm, a UAV, using the GPS service and an IMU, was used to monitor the soil and the efficiency in the management of the irrigation system [56]. To determine the need to activate the irrigation system, the UAV has a multispectral camera to calculate the wine-growing vegetation indices, based on the Normalized Difference Vegetation Index (NDVI). In addition to avoiding the use of airplanes and satellites, crop monitoring by UAV is carried out at a low altitude, without the interference of clouds. A UAV can contribute to reducing the use of pesticides and maximizing the efficiency of crop management [3,57]. Proposing to assist in monitoring the planting and identification of weeds, Sánchez et al. carried out a behavioral study of the vegetation in wheat fields, before and after sowing through the application of herbicide. In this case they performed multi-temporal mapping of a fraction of vegetation at the beginning of the season through various indexes, such as CIVE, ExG, ExGR, Woebbecke Index, NGRDI and VEG [58,59].

Unlike the vineyards commonly found around the world, situated on flat terrain, the vineyards in the Ribeira Sacra region of Galicia in Spain, the Douro region in Portugal, Banyuls and Rhône–Alpes in France and the Rhine and Moselle in Germany are located on a steep slope terrain. This characteristic imposes strong difficulties in the mobility of robots. To carry out the controlled application of pesticides in such crops, Italian researchers developed the UAV Bly-c-agri (depicted in Figure 6t) with the capacity to load a tank of up to 10 l of pesticide and, thus, eliminate all problems related to land locomotion [60]. Another UAV was used to spray Urea (organic compound) within predefined regions. With a maximum load capacity of 5 L, this type of application also allows reducing expenses with the generalized application of herbicides [61].

The kiwi cultivation, among other treatments, is dependent on pollination. In [62], the authors perform this type of task using a machine vision system based on CNN and control the spray time of a mechanical system composed of 20 nozzles. The robot (shown in Figure 6u) was able to pollinate about 79.5% of the kiwi flowers at a speed of 3.5 km/h [62].

Although it is complex for a robot to perform, pruning the plants is an important task. According to Verbiest et al., who carried out research activities in pome orchards, the main challenges of this type of task consist of scanning and measuring the structures of the plants to know the exact place to prune [63]. Furthermore, one way to improve pruning performance is to adapt the crop geometric characteristics to the technical requirements of robotic systems [64]. As described in [65], which describes a robot system for the automatic pruning of grape vines, to inhibit the interference of variations in natural lighting, as well as the interference of the background landscape, researchers Botterill et al. developed a mobile platform (as shown in Figure 6v) that has a 6-DoF manipulator, Light-Emitting Diode (LED) and three RGB cameras. Pruning is carried out through a saw connected to the end of the robotic arm and the classification of branches to be cut is based on the Support Vector Machine (SVM) learning algorithm and Rapidly Exploring Random Tree (RRT), RRT-Connect. The system proved to be able to cut the branches of vines [65].

In [66] a platform composed of a 3-DoF prismatic manipulator equipped with low-cost RGB-D cameras was used to perform automated green shoot thinning in vineyards. In this case, the system design is composed of a Faster R-CNN-based approach to extract the cordon trajectories and a control system (6th order polynomial-based) to operate the thinning end-effector. The robotic platform obtained a Root Mean Square Error (RMSE) of 1.47 cm of thinning end-effector position at a forward speed of 6.6 cm/s.

Although the robots in Figure 6 are from different regions of the world and used in different crops, many have wheels, use 4WS and have sensors such as GNSS and RGB cameras. To standardize the components commonly used in agricultural robotic systems, the company SAGA Robotics developed a modular robotic system, the Thorvald II. Figure 6w shows the different possible configurations of the robot, namely the following versions: models with two possibilities for a differential motor drive with caster wheels for support, models with different track widths, models with/without suspension modules, versions with different heights and models containing three to six wheels [67]. Similarly, Clearpath [68] also has several general-purpose robotic systems (as shown in Figure 6x), being used in some of the robots described in the following sections. Finally, the Russian company Avrora Robotics designed its AgroBot universal control system that can be installed on any special equipment or tractor [69].

Table 3. Comparison between the analyzed robotic applications for plant treatment.

Task	Robots	Locomotion System	Final Application	Location Sensors	Sensors Used to Perform the Task	Computer Vision Algorithms
Disease identification	Disease robot [36]	Not included	Bell pepper	–	RGB camera and laser	PCA and CV
	eAGROBOT [37]	4WD	Cotton and groundnut	–	RGB camera	K-means and Neural Networks
Mechanical weeding	Weeding robot 1 [38]	4WD	Broccoli and lettuce	–	RGB-D camera	RANSAC
	AgBot II [40]	4WS	Cotton, sow thistle, feather top rhodes grass and wild oats	–	RGB camera	LBP
	Oz [41]	4WD	Vegetables, nurseries, and horticulture	LiDAR	RGB camera	–
	Dino [42]	4WS	Vegetables in row and on beds	RTK/GPS	RGB camera	–
	Ted [43]	4WS	Grape	RTK/GPS	RGB camera	–
	VITIROVER [44]	4WD	Soil grass	RTK/GNSS	–	–
	Tertill [45]	4WD	Residential gardens	–	Capacitive sensors	–
	K-Weedbot [46]	4WS	Paddy field	RGB camera	–	Hough transform
	AIGAMO-ROBOT [47]	Track	Paddy field	–	–	–
	Weeding robot 2 [48]	4WD	Paddy field	Capacitive and azimuth sensors	–	–
	Weeding robot 3 [49]	Boat	Paddy field	GPS and IMU	–	–
Chemical weeding	AgriRobot [50]	4WD	Grape	RGB camera and LiDAR	–	FDA and GDA
	SAVSAR [50]	4WD	Grape	RGB camera and LiDAR	–	FDA and GDA
	Robotic sprayer [51]	4WD	Grape	RGB camera and laser	–	FDA and GDA
	RIPPA [52]	4WS	Lettuce, cauliflower and broccoli	RTK/GPS/INS and LiDAR	Hyperspectral and thermal cameras	ExG-ExR
	LadyBird [53]	4WS	Lettuce, cauliflower and broccoli	RTK/GPS/INS and LiDAR	Hyperspectral and thermal cameras	ExG-ExR
	BoniRob [54]	4WS	Sugar beet	–	RGB, NIR cameras and ultrasonic sensor	CNN
	Aerial robot [56]	UAV (Octocopter)	Grape	GPS and IMU	Multispectral camera	NDVI
	Bly-c-agri [60]	UAV (Hexacopter)	Grape	GNSS	–	–
Pollination	Pollinator robot [62]	4WD	Kiwi	Odometry	RGB camera	CNN
Pruning	Pruning robot 1 [65]	Mobile plataform	Grape	–	RGB camera	SVM
	Pruning robot 2 [66]	Mobile plataform	Grape	–	RGB-D camera	Faster R-CNN
General purpose	Swagbot [55]	4WS	General farms	GPS and LiDAR	RGB-D, IR and hiperspectral cameras	NDVI
	Thorvald II [67]	Many forms	General farms	Depends on the application	Depends on the application	Depends on the application
	Clearpath robots [68]	Many forms	General farms	Depends on the application	Depends on the application	Depends on the application
	AgroBot [69]	4WD	General farms	–	–	–

summarizes the characteristics of the various applications of robots in plant treatment.

According to the data in Table 3, the following aspects can be discussed:

• Disease identification: Using conventional RGB cameras, researchers Schor et al. [36] and Pilli et al. [37] were able to detect diseases in plants, with hit rates between 64–96%;
• Mechanical weeding: Several works, both in the research phase and commercially available, use mechanical tools to remove weeds, eliminating the application of herbicidal products of high financial value and allowing the cultivation of organic products. As previously described, with the use of a low-cost robot, researchers Sori et al. [48] report the various benefits generated by the mechanical removal of weeds;
• Chemical weeding: Most robots that perform this task have a specific computer vision technique/algorithm to reduce expenses with over spraying. Vegetation indices such as ExG-ExR and NDVI were used to extract the crop characteristics and perform its subsequent classification. After its proper classification, the specific spray system applies the herbicide to the weed. Therefore, the same spraying system for herbicidal products can be used for the precise application of fertilizers on plants classified as having a low health value [52,53];
• General tasks: As seen in the column “Locomotion Systems” in Table 3, the robots used have terrestrial (4WD, 4WS, track), aerial (hexacopter, octocopter) and marine (boat) systems for locomotion. Not only to avoid the re-creation of existing systems but also to speed up the transition process from research to the commercial stage, Swagbot platforms, Thorvald II, Clearpath and AgroBot were developed for use in carrying out different tasks and in different agricultural environments.

3.4. Robotic Applications in Agriculture for Harvesting

In addition to being a repetitive task and requiring agile execution, the harvesting activity requires a lot of effort from the harvester. According to Hayashi et al., harvesting operations represent around 25% of all hours of agricultural work in Japan [70]. Concerning financial costs (an important decision factor in farmers’ decision-making), in 2014, in Australia, labor rent represented between 20% and 30% of total cash costs [71] and in China, in 2019, it represented more than 75% of production costs and increasing annually [72]. This difference in values, between Australia and China, is due to the higher level of automation of the tasks of Australian crops. In this way, several works (see Figure 8) are being carried out proposing the use of robotic systems to perform agricultural harvesting activities. Several scientific articles address different image processing techniques in different types of cultures. In [73] during the design and implementation of an aided fruit-harvesting robot, Ceres et al. found that the main difficulties in the development of these types of systems were: driving the robot through the field from tree to tree and from tree row to tree row; detection and localization of fruits; and grasping and detaching of selected targets. Another relevant work carried out in the 1990s was the autonomous mobile robot AURORA for greenhouse operation, capable of autonomously navigating the greenhouse corridors using only the information provided by ultrasonic sensors [74]. The robot was built to be a multitasking platform, to perform tasks such as: harvesting, transportation of fruits and plan inspection. In this case, the project requirements were: navigation in unaltered greenhouses, low cost, flexibility, multi-functionality, supervizable autonomous operation and friendly user interface. In their work, Bac et al. analyzed about 50 applications of robots from different regions of the world in carrying out agricultural harvesting activities [18]. However, despite all the reviewed works proposing the use of robots to carry out harvesting activities, only the task of capturing fruits showed an improvement trend, when compared to the works done in the period between 1984 and 2014. In other activities, the evaluated robots have not yet performed well enough to replace a human harvester [18].

The Agrobot E-Series robot is made entirely of stainless steel and military-grade aluminum to withstand the diverse climatic conditions of the field. It has 24 independent Cartesian robotic arms that move around the robot’s structure to pick up strawberries. With a total of three wheels (the electric motor is located in the central part), the robot allows the mechanical structure adjustment to suit the crop dimensions. As it is a large robot (as shown in Figure 8a) it uses data from the LiDAR sensor to avoid collisions with any workers in the farm [75].

With the equivalent yield of 25 to 30 human harvesters the Berry 5 robot has a picking speed of 8 s per fruit, moving through strawberries beds at a speed of 1.6 km/h, harvesting up to eight acres of strawberries a day. Created by the American company Harvest CROO Robotics, this automatic harvester is making strides towards commercialization [76]. Like the Agrobot E-Series, the various mechanisms of the Berry 5 robot are protected by patents, making its scientific analysis difficult.

In 2017, researchers Leu et al. developed a green asparagus harvesting robot (named GARotics) [77]. Asparagus must be harvested when they reach a height of between 15.24 and 20.32 cm, otherwise, they will not be accepted by the market. This type of harvest is difficult to be automated because asparagus tends to group and the tenderness of the stalk makes it susceptible to breaking. In this case, two robotic arms with customized grippers were developed to catch the asparagus without damaging it. The robotic arms have a single pneumatic cylinder that transfers linear motion in a resulting circular motion. To identify asparagus ready to be harvested in real-time, the robot has an RGB-D camera that provides the planting data for its vision module, consisting of the following tasks: point cloud generator, camera calibration (via Template Point Cloud (TPC) and Model Point Cloud (MPC)) and online asparagus tracking (RANSAC and Euclidean clustering methods). As a result of the work, the German robot achieved an average locomotion speed of 0.2 m/s and a harvest cycle of 2 s per robotic arm, resulting in 90% of successful harvests [77].

A robotic lettuce harvesting system, Vegebot (depicted in Figure 8b), was developed by English researchers in 2018 [78]. As a very fragile vegetable, the challenges of this task are the correct vegetable identification and removing the lettuce without damage. Using two RGB cameras located above and at 45° from the vegetation, the head of the lettuce was identified based on the Region-based Convolutional Neural Network (R-CNN). After locating the lettuce, Vegebot extracts it using a 6-DoF robotic arm and a gripper system with closed-loop monitoring of the applied force. The system obtained a success rate of 91% in locating lettuces and 82% accuracy in the correct classification of vegetables [78].

In 2019, Ge et al. developed an algorithm to locate and collect strawberries using a robot (developed by Noronn AS) equipped with an RGB-D camera, seen in Figure 8c. The detection of strawberries was done using R-CNN and the collision-free path-planning algorithm was based on 2D images and the 3D point cloud. After several tests in real environments, 74.1% of the identified ripe strawberries were successfully harvested with an $F_1$ score (average harmonic precision) of 0.9 [79].

Sepúlveda et al. addressed robotic aubergine harvesting using dual-arm manipulation [80]. Through two 6-DoF robotic arms, as shown in Figure 8d, underactuated grippers with a set of three flexible fingers (off-the-shelf) and two cameras, the robotic system was used to analyze the benefits generated by the collaborative action of the manipulators. Just as a human picker often uses one of his hands to clear the path to reach the fruit and harvest it with his other hand, the robotic system was used to identify possible aubergines partially blocked by leaves, to lift the leaves so that the camera captures the image of the aubergine in its entirety and performs its collection. An algorithm based on the SVM classifier was used to classify the image among four classes: aubergines, leaves, branches and background. The proposed occlusion algorithm is based on the difference between the distances between the centroids of the aubergines and the leaves identified in the image, creating a vector pointing to the direction in which the leaf must be lifted to unblock the aubergine. After several experimental tests, the success rate of the harvesting robot was 95% for two isolated parts and 80% for an occluded part, both using two arms. The system has improved the total processing time (image processing, inverse kinematic and action) of fruit from 42.90 s with one arm to 26.54 s with two arms. The main challenges encountered by the authors were changes in lighting conditions, directly interfering with the system’s performance.

Using the modularization concept described in Section 3.3, the strawberry picker robot seen in Figure 8e uses the Thorvalds II robots’ locomotion system, from the Saga Robotics group. To improve its harvest time, it features a 3-DoF Cartesian-type dual-arm system. With simpler inverse kinematics to be calculated, the two Cartesian robotic arms were used not only to optimize harvesting efficiency but also to avoid the collision. In this sense, Xiong et al. developed a novel active obstacle-separation path-planning strategy for cluster picking [81]. The occlusion of fruits (by leaves, branches or other unripe fruits) is a recurring problem in several applications of harvesting robots. Thus, using an RGB-D camera and an algorithm based on Hue Saturation Value (HSV) color-thresholding, the picker robot identifies strawberries and their obstacles, calculates the trajectory necessary to reach the target and when it approaches an obstruction it clears the path by pushing obstacles to the side. The HSV color-thresholding technique allows less sensitivity to changes in ambient lighting. If the robotic arm does not pick the fruit on the first attempt, the system makes a new attempt. On the first attempt, the strawberry picker robot achieved a success rate of 97.1% for isolated strawberries and without neighbors and 5% for strawberries completely surrounded by unripe strawberries; on the second attempt, the robot achieved a success rate of 100% and 20% for the same cases as previously described. With the dual arm system approach, the robot reduces the fruit handling time from 6.1 s (with only one robotic arm) to 4.6 s (with both robotic arms) [81].

Visual perception is increasingly being improved. As an example of the current advances in this task, Kang et al. developed a prototype of an apple harvesting robot equipped with a 6-DoF robotic arm and a soft-finger based gripper (not to damage the surface of the apples) together with an RGB-D camera (as shown in Figure 8f) to perform the following tasks: vision perception, motion planning, fruit verification and fruit detachment [82]. The fruit recognition was done through the use of the Dasnet deep convolution neural network and the fruit pose computation through 3D Sphere Hough Transform (3D-SHT). The ambient light strongly interfered in the distance measurements captured by the RGB-D camera and, to solve this problem, the authors applied the distance-based denoising method on points processing before pose computation. Thus, all fruits that do not have a sufficient number of points or have severely in-balance length on the X, Y, Z axis are removed from the list of discovered fruits. The point cloud obtained by the RGB-D camera was used to model the environment where the apples are located, in which case the authors used an octrees-based representation of occupied space in working environments. The system obtained an $F_1$ score for accuracy of fruit detection of 0.871 and as improvements, the authors suggest the recognition of ripe and damaged fruits [82].

To perform real-time visual localization of the picking points for a ridge-planting strawberry harvesting robot, Yu et al. propose the Rotate-YOLO (R-YOLO) method, a variation of the original YOLO deep learning algorithm [83]. The bounding box is rotated by an angle $\alpha$, to follow the strawberry’s orientation, allowing greater precision in the location of the pick point. Specially customized to operate on a strawberry ridge-planting, the robot (seen in Figure 8g) has fiber sensors on its end-effector to speed up control, avoiding real-time distance measurement [83]. Using the R-YOLO recognition method proposed by the researchers, the robot achieved a strawberry detection accuracy rate of 94.43% with a speed of 0.056 s for 640 × 480 images captured using a conventional RGB camera.

As an example of harvesting robots that also use artificial intelligence algorithms, but are mounted on a mobile platform and are used to harvest sweet pepper in protected cropping environments, one can mention the Harvey platform and SWEEPER, shown in Figure 8h,i respectively. The Harvey platform opted for the use of Deep Convolutional Neural Network (DCNN), while the SWEEPER robot used the techniques of deep learning, shape and color-based detection algorithm and Hough Transform (HT) [84,85,86]. Both have 6-DoF robotic arms, but they have different capture methods and cutting systems for detachment, as Harvey performs the suction of sweet pepper employing a vacuum pump, while SWEEPER holds the sweet pepper through flexible fingers. As a result, the SWEEPER robot took an average of 4.3 s to detect sweet pepper and 14.5 s for detachment, while the Harvey platform took about 3.7 s and 2.2 s to perform the same activities.

Coconuts harvesting activities in developing countries are commonly done without any safety equipment. In addition to causing serious injuries, falling a coconut tree can be fatal. In this sense, the Indian researchers Megalingam et al. developed Amaran, an unmanned robotic coconut tree climber and harvester [87]. The Amaran robot climbs the coconut trees through a light mechanical structure composed of eight mechanum wheels, four located at the top and four at the bottom. Through a certain activation sequence, Amaran can move up, down, left or right. To detach the coconuts, Amaran has a 4-DoF robotic arm and a cutting tool as an end-effector, both customized to be light and not compromise the robot’s movement along the coconut tree, as shown in Figure 8j. Without any type of computer vision algorithm, the robot’s RGB camera serves to assist the human operator, located in a safe region of the ground, performing the remote control and monitoring of the robot. Taking advantage of the IoT concept, the Amaran robot can be controlled through an application for smartphones, via the Bluetooth communication protocol. After several tests, Amaran proved to be able to successfully climb trees up to 15.2 m in height, with slopes of 30° and with diameters ranging from 0.66 m to 0.92 m [87]. Although the total harvest time (preparation and harvest time) of the Amaran (21.9 min) is longer than that of a professional climber (11.8 min), the robot can climb as many coconut trees as necessary without exposing the human operator to harmful work to his health or even eventual fatal accidents.

Table 4. Comparison between the analyzed robotic applications for harvesting.

Robot	Robotic Arm	Final Application	Location Sensors	Sensors Used to Perform the Task	Computer Vision Algorithm	Success Rate (Cycle Time)
Agrobot E-Series [75]	24 Cartesians arms	Strawberry	LiDAR	RGB camera, ultrasonic and inductive sensors	–	–
Berry 5 [76]	Multiple robotic components	Strawberry	GPS and LiDAR	RGB camera	–	–
GARotics [77]	Pneumatic cylinder with two blades	Green asparagus	–	RGB-D camera	RANSAC and euclidean clustering	90% (2 s)
Vegebot [78]	6-DoF and a custom end effector	Lettuce	–	RGB camera	R-CNN	88.2% (31.7 s)
Noronn AS [79]	5-DoF	Strawberry	–	RGB-D camera	R-CNN	74.1%
Harvester robot 1 [80]	6-DoF dual-arm	Aubergines	–	RGB-D and ToF cameras	SVM	91.67% (26 s)
Harvester robot 2 [81]	3-DoF cartesian dual-arm	Strawberry	LiDAR and encoder	RGB-D camera	HSV color-thresholding	50–97.1% (4.6 s)
Harvester robot 3 [82]	6-DoF soft-finger based gripper	Apple	–	RGB-D camera	Dasnet, 3D-SHT and Octree	$F_1$: 0.81 (7 s)
Harvester robot 4 [83]	6-DoF	Strawberry	–	RGB and laser sensors	R-YOLO	84.35%
Harvey plataform [84]	6-DoF	Sweet pepper	–	RGB-D camera, pressure and separation sensors	DCNN	76.5% (36.9 s)
SWEEPER [85]	6-DoF with custom designed end effector	Sweet pepper	–	RGB-D camera	Deep learning, shape, color-based detection and HT	61% (24 s)
Amaran [87]	4-DoF	Coconut	–	RGB camera	–	80–100% (21.9 min)

summarizes the main characteristics of the robotic applications for harvesting.

When analyzing the data in Table 4, it is observed:

• Challenges: Despite the constant technological advances, the fruit occlusions and the changes in ambient lighting are still challenges that merit further scientific studies and work to enable the use of robots in agricultural environments;
• Simplicity and efficiency: In addition to the challenges of occlusion and changes in ambient lighting, the simplicity of construction and efficiency of the robotic system are two factors that allow to streamline the commercialization process. The system efficiency is directly related to the computer vision algorithms used and, in this sense, the improvement of such algorithms will increase the efficiency of the robotic system as a whole. To the authors knowledge, only the Agrobot E-Series and Berry 5 robots in Table 4 are in the commercialization phase;
• Evolution between 2014–2021: As previously described, Bac et al. [18] carried out a detailed study of 30 years of evolution (1984–2014) on harvesting robots. Thus, are compared the values (average; minimum–maximum) of his work with the analyzes of the present work (which vary from 2013–2021). Bac et al. reached the following values: harvest success rate (66%; 40–86%) and cycle time (33 s; 1–227 s) and this work found the following values: harvest success rate (81.17%; 50–100%) and cycle time (18.88 s; 2–36.9 s)—the cycle time of the Amaran robot was disregarded, as it is dependent on the skills of the operator. Thus, in general aspects, there is a 22.98% increase in the average harvest success rate and a 42.78% reduction in the average cycle time value, indicating an evolution in the performance of the harvesting robots.

3.5. Robotic Applications in Agriculture for Yield Estimation and Phenotyping

Through accurate data on the quantity and quality of the fruits growth, provided by more sophisticated tools, farmers can manage their crops more efficiently. Yield estimation is nothing more than monitoring the entire crop and estimate the produced fruit. On the other hand, several variables such as climate change and soil quality can interfere with the development of plants. Thus, by identifying the phenotype of the plants, it is possible to link to their respective genotype, making it possible to identify the proper growth conditions. It is noted, however, that for a robot to estimate phenotyping and/or yield it must have not only reliable sensory data, but also efficient computer vision algorithms. In 1998 and 2001 there were already researchers proposing the use of sensors and machine vision algorithms to detect crop rows and gather field information [88,89]. In this case, a camera and a RTK/GPS device were inserted in a tractor to be able to create spatial maps relating the crop height and width. Through ANN the robotic system obtained a hit rate of 84%, indicating that a machine vision system could be used as a crop prediction sensor. Thus, this subsection will address several robots developed over the years to perform the tasks of yield estimation and phenotyping, giving an emphasis on the ones developed in recent years.

The Shrimp robotic system shown in Figure 9a, equipped with six RGB cameras, was used to estimate the yield of apple orchards under natural lighting conditions. To locate each sampled image, the Shrimp platform relies on the integration of an Inertial Navigation System (INS) with a GPS. The image processing is based on Multiscale Multilayer Perceptron (MLP) and CNN and the detection of the apples was performed through Watershed (WS) segmentation and Circular Hough Transform (CHT). The Shrimp platform obtained an apple detection of 82.5%, $F_1$ of 0.791 and a coefficient of determination $r^2$ of 0.826, using CNN and WS [90].

With a highly competitive market, the yield of vines can vary from region to region, due to climatic conditions, variety, soil and the techniques specific to each agricultural producer. In this sense, monitoring the grapes throughout the crop allows the quality quantification of the harvested grapes. Two projects funded by the European Union’s Seventh Framework Program have been developed in this area, VINBOT and VineRobot, shown in Figure 9b,c respectively. VINBOT uses CNN to detect grapes, then computes the area of grape occupation in the images and estimates their respective weight in kilograms [91]. VineRobot monitors parameters such as grape yield, vegetative growth, vineyard water status and grape composition using the following techniques: Chlorophyll-based fluorescence, RGB machine vision and thermography [92].

Brazil is one of the largest food exporters in the world. For this reason, the Brazilian Agricultural Research Corporation (EMBRAPA) financed the development of the AgriBOT agricultural robot (seen in Figure 9d) to develop a modular robotic platform for data acquisition and yield estimation in orange and sugar cane crops. In 2011, the authors Abrahão et al. compared the navigation efficiency of algorithms D* and Focused D* to be used by AgriBOT with 4WS. The Focused D* algorithm proved to be more efficient than D* in environments in which maps were incomplete or inaccurate [93]. In 2016, Lulio and Lugli et al. implemented a J Segmentation (JSEG) algorithm, statistical Artificial Neural Networks (ANN) image segmentation techniques and sensory fusion in the AgriBOT robot, based on the extraction of objects from real natural scenes, identifying items such as fruits, grasses, stems, branches and leaves [94,95].

The 4WD Agrob V14 robot was developed to monitor the vineyards of the Douro region in Portugal, with steep slope terrain. The robot has RGB cameras, infrared (IR) sensors, LiDAR and encoders, as seen in Figure 9e and was designed to work autonomously even in cases of unavailability of the GNSS signal [96]. The high content of stones in the soil interferes with the odometer and IMU data. To solve these problems, Santos et al. [97] integrated the Simultaneous Localization and Mapping (SLAM) techniques with the data generated by Radio Frequency IDentification tags (RFID) located at the beginning and the end of each line of the vineyard. The authors conclude that Agrob V14 can overcome ditches, rocks and slopes of up to 30% inclination [97].

The robot Agrob V16 (shown in Figure 9f), developed for yield estimation and pruning tasks, approached another way to improve the robot’s location and positioning, based on a wireless sensor network. Integrating the concept of IoT and SLAM, Agrob V16 (which has Clearpath’s locomotion system mentioned in Section 3.3) reads the Received Signal Strength Indication (RSSI) signals generated by a Bluetooth Low Energy (BLE) transmission module and, according to the received signal strength, it estimates the position of the signal source. The fusion of encoder data with distance signals based on RSSI was done by an Extended Kalman Filter (EKF). After the application of the EKF filter, the use of RSSI allowed a 25% reduction in the standard deviation of the robot’s trajectory [98].

A biologically-inspired hexapod robot was developed to monitor the healthy growth of agricultural fields, such as agronomic information on soil nutrients. However, according to Iida et al., the use of high-precision navigation devices (RTK/GNSS) is not viable, due to the hexapod’s size and power supply limits [99]. Just as the locomotion system was biologically inspired by insects, the researchers presented a new way of guiding the robot without using RTK/GNSS, through the insect’s sense of smell. Through the use of an anemoscope and $C{O}_2$ gas sensors, the hexapod travels autonomously throughout the crop, following the sources of $C{O}_2$ and the wind direction. The hexapod robot in Figure 9g used the tripod gait, varying its speed according to the update control time and/or adjusting the length of steps. After experimental results, the authors suggest that the hexapod can be guided autonomously by air currents and simultaneously monitor the concentration of $C{O}_2$ gas discharged from crops and soil [99].

Like the hexapod robot, the TerraSentia robot also has small dimensions, monitors the crops and does not have an RTK/GNSS system. In this case, the TerraSentia robot depicted in Figure 9h was used to move between corn and sorghum crops, which have high height vegetation, having only one LiDAR to perform light detection under the canopy and ranging-based autonomous navigation [100]. TerraSentia moves around using a LiDAR-based navigation algorithm. After reading the LiDAR input data, TerraSentia performs a data filtering process, excluding outlier points and estimates its trajectory using a set of heuristics and least squares. Several tests were carried out and the robot covered more than 6 km of straight rows autonomously.

In [101] researchers have developed a canopy density estimation at four separate locations in South Australia using AgScan3D, a mobile vehicle-mounted 3D spinning LiDAR system, to generate a globally registered ray cloud. The AgScan3D is embedded in the rear of a Kubota farm vehicle and consists of a 3D spinning LiDAR, 3DM-Gx3 IMU and a GPS unit. The AgScan3D system applies a Continuous-Time SLAM algorithm into a globally registered 3D ray cloud, a series of extraction and segmentation algorithms for the ground and the rows of vines and makes use of the variable resolution method to perform the canopy density estimation. Through experimental tests, a total traversal of 160 km and approximately 93,000 vines were scanned and obtained repeatability with a root mean square error of 3.8% for the vehicle traveling at an average speed of 5 to 6 km/h.

The plant phenotyping can be extracted in various ways, such as, for example, by monitoring plant height, weight, biomass, shape, color, volume, light absorption and temperature [102]. Two robotic platforms were used together to extract the phenotypic characteristics of corn plants: Vinobot and Vinoculer, as shown in Figure 9i,j, respectively. Vinoculer is a fixed platform that constantly monitors height and extracts 3D data from the crop (reconstructed using Visual Structure From Motion—VisualSFM—depicted in Figure 10) and Vinobot extracts the individual phenotypic characteristics of each plant, moving around the entire crop. In this way, it is possible to correlate the data collected from Vinoculer (general data) with that from Vinobot (individual data). In addition to monitoring plant height, these robots calculate the Leaf Area Index (LAI) and measure light intensity and air temperature, both using multiple sensors. Filtering the point cloud based on the vegetation height allows reducing the interference of unwanted vegetation, such as weeds. Thus, the collection of such data from the farm greatly speeds up the phenotyping process [103].

The architecture consists of two robotic platforms: an autonomous ground vehicle (Vinobot) and a mobile observation tower (Vinoculer). The ground vehicle collects data from individual plants, while the observation tower oversees an entire field, identifying specific plants for further inspection by the Vinobot. The advantage of this architecture is threefold: first, it allows the system to inspect large areas of a field at any time, during the day and night, while identifying specific regions affected by biotic and/or abiotic stresses; second, it provides high-throughput plant phenotyping in the field by either comprehensive or selective acquisition of accurate and detailed data from groups or individual plants; and third, it eliminates the need for expensive and cumbersome aerial vehicles or similarly expensive and confined field platforms.

A customized helicopter (Pheno-Copter, depicted in Figure 9k) was used to estimate variations in a land cover of sorghum (early season), the temperature of cover in sugar cane (mid-season) and three-dimensional measures of crop lodging in wheat (late season), using concepts from RANSAC and Digital Elevation Models (DEM), showing the ability to meet different levels of needs and image coverage [104].

Designed for scouting and phenotyping applications, the Ara robot from ecoRoboticx (depicted in Figure 9l) can make corrections in RTK/GPS via GSM/3G communication; however, it does not provide any solution in algorithms and sensors, being designed to be integrated with several sensors of different manufacturers. Weighing around 130 kg, the robot can be controlled by a smartphone via WiFi or 3G/4G communication [105].

Table 5. Comparison between the reviewed robotic applications for yield estimation and phenotyping.

Task	Robot	Final Application	Location Sensors	Sensors Used to Perform the Task	Computer Vision Algorithm
Yield Estimation	Shrimp [90]	Apple	–	RGB camera	MLP and CNN
	VINBOT [91]	Grape	RTK, DGPS and LiDAR	RGB and NIR cameras	NDVI
	VineRobot [92]	Grape	–	FA-Sense LEAF, FA-Sense ANTH, ultrasonic and RGB camera	Chlorophyll-based fluorescence and RGB machine vision
	AgriBOT [93]	Orange and sugar cane	GPS/INS and LiDAR	RGB camera	–
	Agrob V14 [96]	Grape	LiDAR	RGB camera	SVM
	Agrob V16 [98]	Grape	RTK/GPS/INS and LiDAR	Stereo, RGB-D and RGB cameras	hLBP and SVM
	Hexapod [99]	General farms	–	$C{O}_2$ gas module, anemoscope and infrared distance sensor	–
	Kubota farm vehicle [101]	Grape	GPS and IMU	LiDAR	Continuous-Time SLAM
Phenotyping	TerraSentia [100]	Corn	RTK/GPS and LiDAR	RGB camera	LiDAR-based navigation
	Vinobot [102]	Corn	DGPS and LiDAR	Stereo camera and environmental sensors	VisualSFM
	Vinoculer [103]	Corn	–	Stereo RGB and IR cameras and air temperature sensors	VisualSFM
	Pheno-Copter [104]	Sorghum, sugarcane and wheat	–	RGB and thermal cameras and LiDAR	RANSAC and DEM
	Ara ecoRobotix [105]	General farms	RTK/GPS and compass	RGB camera	–

shows a summary of the aforementioned works.

The following observations can be made after analyzing the data in Table 5:

• Sensors: The micro observation of the biological phenomena of each plant, whether for yield estimation or phenotyping, requires specific and highly reliable sensors, from fluorescence level detection sensors, multispectral, Near-Infrared (NIR), IR, environmental to RGB cameras;
• SLAM: Whether due to physical dimensions, power supply requirements or vegetation height, the unavailability of systems based on GNSS devices induces the improvement of SLAM techniques. Thus, to improve navigation in these conditions, robots use both natural characteristics (such as the generation of trajectories based on the average distance between rows and the direction of the airflow) and artificial ones (such as the use of RFID tags and wireless sensors). Several SLAM algorithms and path planning techniques for agricultural and forestry robots are described in detail in [106,107,108];
• Artificial intelligence: Based on the specific characteristics of each crop, vegetation indices (such as NDVI and Chlorophyll-based fluorescence) and artificial intelligence algorithms (such as MLP, CNN and SVM) can be used. Therefore, one must seek to establish a balance between the computational complexity level of the proposed approach with the expected efficiency/result.

4. Discussion

After finding and discussing the main existing robotic systems presently available or in the research phase, several data were collected to be analyzed. Therefore, this section will discuss the research trends, common difficulties, the indices that hinder commercial development, as well as which countries are investing in the research of these types of solutions and, finally, what are the desirable requirements for robotic agricultural systems.

4.1. Agricultural Robots

When evaluating the various characteristics commonly found in all agricultural applications discussed in this article, it was possible to create the charts presented in Figure 11.

The data in Figure 11 reveal that, in general aspects, most are robotic systems applications in agricultural environments are directed towards the development of 4WD robots, without a robotic arm, used for the removal of weeds and use RGB cameras, as depicted in Figure 11a–d, respectively. The research shows that although 32.23% of the works use RGB cameras, most of them do not have/do not report the use of computer vision algorithms, as depicted in Figure 11e, where the specific functions are, for instance, Otsu method, HT and CV. Of all 62 projects analyzed, 80.65% are in the research stage (show in Figure 11f). Most of the work analyzed was done by Australian researchers/companies (see Figure 11g) and, according to the FAO [109], between 1997, 2007 and 2017, although Australia does not yet dominate the world market for agricultural production, it has been growing every 10 years.

Another interesting fact concerns the amount of work developed by researchers or companies by continent, depicted in Figure 11h. It is significant to note that no work was carried out by researchers/companies from the African continent, a continent characterized by having the highest rates of poverty, hunger and lack of qualified labor in the world.

4.2. Unsolved Issues

To improve current agricultural robotic systems, several proposals are presented to assist future researchers. According to Figure 11a, most agricultural robots are 4WD, however, the agricultural environment is classified as semi-structured and, in this case, 4WD robots are strongly affected by soil characteristics, as mentioned earlier in [97]. As reported in [23], there is still a very large trade-off between the quality and cost of cameras used in agriculture. Unlike the works carried out in the last century, there are currently several types of computer vision algorithms that can be implemented in high-performance embedded systems, but the choice of the most suitable algorithms for each type of situation still needs to be improved. As they are electronic systems that work in the same environment in which the robots are inserted, the IoT devices must be used in conjunction with the robots. Therefore, in general, it is possible to group the proposals in four areas: locomotion systems, sensors, computer vision algorithms and communication technologies.

4.2.1. Locomotion Systems

As noted in Figure 11, most locomotion systems for agricultural robots are 4WD. However, wheeled systems are strongly affected by the characteristics of local terrain, such as rocks and branches. Besides, the constant locomotion of these robots throughout the farm results in a high rate of soil compaction. Concerning UAV, its flight time improvements can increase its use in agricultural environments [57].

Another alternative for locomotion in unstructured environments is the use of legged robots, according to [15,18]. As advantages, these robots do not need constant contact with the ground to get around and adjust their posture according to the slope of the terrain, allowing them to navigate in rugged and difficult to access environments [110,111]. Figure 12 shows some commercially available off-the-shelf legged robots.

All robots shown in Figure 12 are quadrupeds and are already commercially available for sale. However, each one has its solutions for performing trajectory planning calculations, locomotion patterns, navigation systems and computer vision algorithms [112,113,114]. The common characteristics of these robots are: being relatively light, small, autonomous and have locomotion patterns that adapt to the environment. Despite allowing the reduction of damage to the ground, the robots feet in Figure 12 have a small contact area, creating great pressure on the foot placement region. In this sense, to prevent robots’ feet from penetrating soft soils and becoming trapped, it is necessary that legged robots have specifically designed foot-ground contact areas (based on the concepts of soil mechanics) to reduce pressure (increasing the foot-ground contact area) under the soil during locomotion. Thus, as they are widespread robotic platforms, their risks of rejection by the agricultural market are reduced.

4.2.2. Sensors

According to Figure 11d, the most used sensor in the reviewed works was the RGB camera. The RGB-D, thermal, hyperspectral and multispectral cameras, despite providing more information (depth, temperature and more spectral data), have a higher financial cost, hindering its practical implementation. The trade-off between financial cost and quality must be chosen according to the minimum system requirements to be developed, because in an agricultural environment the temperature, humidity and dust incidence can directly interfere with the proper functioning of the sensors. In this sense, the development of sensors with high Ingress Protection (IP) (IP65, IP66 or IP67), which operate with high temperature and humidity ranges and, mainly, with a low financial cost, may contribute to the construction of more robust agricultural robotic systems to variations in climatic conditions (sun and rain), extending its useful life.

4.2.3. Computer Vision Algorithms

Through the extraction of crop characteristics (such as ExG-ExR, NDVI, Chlorophyll-based fluorescence and RGB/thermal/hyperspectral/multispectral images), the use of artificial intelligence algorithms (such as MLP, CNN, R-CNN, R-YOLO and SVM) allows the identification of diseases, detection of weeds, selective application of herbicide/pesticide, location of fruits/vegetables/vegetables, classification of ripeness (ripe/unripe) and yield estimation. Again, the short-term (ambient lighting) and long-term (seasons) variations of the crop may interfere with the computer vision algorithm efficiency. Thus, it is proposed to improve and/or create new computer vision algorithms that adapt to the short and long-term changes of the crops and that are optimized to operate on devices with low processing power and cost.

4.2.4. IoT-Based Smart Agriculture

Just as the concept of smart cities is strongly linked to the use of IoT technologies [115,116], the concept of smart agriculture must also be. The use of various artificial intelligence techniques, such as CNN, demonstrated a high degree of adaptation to the rapid variation in natural lighting, changing seasons and crop growing. Thus, the various forms of integration of IoT devices used in agricultural activities [117,118], with the various types of artificial intelligence algorithms and with the numerous robotic systems reviewed and described in this article may contribute to better control, monitoring, preservation and standardization of processes, developing precise multi-purpose systems to solve short (harvest monitoring) and long-term (yield estimation) problems. In addition, the integration of IoT sensors with mobile robots has great potential for improving the concepts of parallelism and swarm of robots due to the possibility of exchanging Machine-to-Machine (M2M) information.

5. Conclusions

To propose new technical and scientific advances in the field of smart agriculture, it is first necessary to know the main existing works, exposing their advantages, limitations and common pitfalls in order to identify the real needs for improvements. After a systematic review of agricultural robotic systems applied in the execution of land preparation before planting, sowing, planting, plant treatment, harvesting, yield estimation and phenotyping, it was observed that 37% of robotic systems are 4WD, 64.52% don’t have a robotic arm, 22.06% are used in weeding tasks, 32.23% use RGB cameras, 35.48% not included/not reported computer vision algorithms, 80.65% are in the research stage, 16.67% are designed by Australian companies/researchers and 41.94% are developed by countries on the European continent. The main characteristics observed were the little use of the concept of off-the-shelf, parallelism, the swarm of robots, simple and efficient computer vision algorithms, in addition to multi-purpose platforms that adapt appropriately to the type of crop studied. To improve the current agricultural robotic systems, four main areas have been proposed for future research works: locomotion systems, sensors, computer vision algorithms and IoT-based smart agriculture. This paper covered about 62 agricultural robotic systems and, just as it was observed a 22.98% increase in the average harvest success rate and a 42.78% reduction in the average cycle of harvesting robots between 2014–2021 (described in Section 3.4), it is expected that, with the improvement of the previously mentioned areas, the development of agricultural robotic systems will continue to increase their efficiency and robustness. Therefore, it is believed that this work was able to show not only the notable advances in the field of mobile robotics but also to correlate the advantages of investing in technologies that act as tools for transforming nature.