Occupancy Grid and Topological Maps Extraction from Satellite Images for Path Planning in Agricultural Robots

Abstract

Robotics will significantly impact large sectors of the economy with relatively low productivity, such as Agri-Food production. Deploying agricultural robots on the farm is still a challenging task. When it comes to localising the robot, there is a need for a preliminary map, which is obtained from a first robot visit to the farm. Mapping is a semi-autonomous task that requires a human operator to drive the robot throughout the environment using a control pad. Visual and geometric features are used by Simultaneous Localisation and Mapping (SLAM) Algorithms to model and recognise places, and track the robot’s motion. In agricultural fields, this represents a time-consuming operation. This work proposes a novel solution—called AgRoBPP-bridge—to autonomously extract Occupancy Grid and Topological maps from satellites images. These preliminary maps are used by the robot in its first visit, reducing the need of human intervention and making the path planning algorithms more efficient. AgRoBPP-bridge consists of two stages: vineyards row detection and topological map extraction. For vineyards row detection, we explored two approaches, one that is based on conventional machine learning technique, by considering Support Vector Machine with Local Binary Pattern-based features, and another one found in deep learning techniques (ResNET and DenseNET). From the vineyards row detection, we extracted an occupation grid map and, by considering advanced image processing techniques and Voronoi diagrams concept, we obtained a topological map. Our results demonstrated an overall accuracy higher than 85% for detecting vineyards and free paths for robot navigation. The Support Vector Machine (SVM)-based approach demonstrated the best performance in terms of precision and computational resources consumption. AgRoBPP-bridge shows to be a relevant contribution to simplify the deployment of robots in agriculture.

1. Introduction

Agriculture is among the most critical sectors of the global economy. The sector has been adapted along years to fulfil the worlds population demand, which has doubled in the last 50 years [1]. The predictions point to a 60% increase of the world’s population until 2050. Furthermore, it will be expected to have more people living in urban areas [2]. Besides, a decrease in human resources for agricultural labour has been noticed in past years [3,4]. These statements indicate that the world’s agriculture productivity must increase sustainably and be less dependent on handcraft work with automatization and optimization of agricultural tasks. The strategic European research agenda for robotics [5] states that robots will improve agriculture efficiency. The literature presents some robotic solutions for precision agriculture. A robot equipped with a Light detection and ranging system (LIDAR) and vision sensors was proposed for monitoring orchards [6,7]. Mahmud et al. [8] presented a path planning approach for pesticide spraying in greenhouses, and Iqbal et al. [9] proposed a simulation of a robotic platform based on 2D LIDAR for navigation and phenotyping tasks, like measuring canopy height. Recently, a literature review under the subject of agricultural robotics concluded that robotic systems were most explored for harvesting and weeding. The study infers that optimization and further development of agricultural robots is vital [10]. However, the deployment of robots in agriculture is still a challenge.

To localize the robot and perform a path planning operation, usually, there is a need of a preliminary map of the field obtained from a previous visit of the robot at the farm through a Simultaneous Localisation and Mapping (SLAM) process. In extensive agricultural terrains, this would represent a time-consuming operation or impractical procedure.

In the context of vineyards, in particular steep slope vineyards (such as those placed in the Douro Demarcated Region (Portugal), UNESCO Heritage place), obtaining a preliminary map is critical. These scenarios present several challenges to autonomous robot navigation: Global Navigation Satellite Systems (GNSS) gets frequently blocked by the hills providing unstable positioning estimations, and the irregular sloppy terrain presents a challenge for path planning algorithms. To tackle some of these challenges, we proposed VineSlam [11] and Agricultural Robotics Path Planning framework (AgRobPP) [12]. An identified limitation in AgRobPP is its memory efficiency. The large dimensions of the vineyards would present a memory problem to the path planning algorithm, as large amounts of data would be required to construct a map.

To obtain a preliminary map and solve AgRobPP memory requirements, this work contribution proposes a novel solution called AgRobPP-bridge, with two stages: AgRob Vineyard Detector and AgRob Grid Map to Topologic.

The first stage performs vineyard rows detection from satellite images, which will provide a pre-map of the farm for the robot’s first visit, reducing the human operator need. This tool is based on a Support Vector Machine (SVM) classifier approach. AgRob Vineyard Detector also contains a tool to simplify the process of manual image annotation of crop rows in satellite images. Besides, an open-source tool based on deep learning techniques (ResNET and DenseNET), Semantic Segmentation Suite [13] was tested and bench-marked with our approach.

The second stage, AgRob Grid Map to Topologic, constructs a topological map of a vineyard. It takes the resulting grid map (or other) and extracts a topological map with image processing techniques and Voronoi diagrams. This tool also contains an A* search algorithm to navigate inside the topological map. The map is delimited into smaller zones considering this concept, which will allow path planning algorithms to be more efficient.

In this paper, Section 2 presents the related work of feature extraction from aerial images in agricultural scenarios and path planning approaches with topological maps. Section 3 shows the first stage of AgRobPP-bridge: AgRob Vineyard Detector. Section 4 describes the second stage of AgRobPP-bridge: AgRob Grid Map to Topologic. Section 5 reveals the results of the tool AgRobPP-bridge. The conclusions are described in Section 6.

2. Related Work

Robotic path planning is widely explored in literature [14,15,16], and the basic concept consists of the task of finding a collision-free path between two points. The majority of the approaches try to find the best possible path suitable for the required task. Path planning methods can be based on several concepts, such as potential field [17], sampling-based methods [18], cell decomposition [19], and nature-inspired algorithms, like the Genetic Algorithm [20]. Independently, path planning algorithms can be classified as off-line or on-line. The first category requires a previous full map of the environment with obstacles information, while, in the second category, it is possible to construct the map during the navigation [14].

The applications of path planning in agriculture are not so spread. A literature review under this topic was performed in a previous work [21], revealing the current research work on path planning for agriculture. According to this review, most of the approaches for path planning in agriculture consist of off-line path planners. Although there are some online options, it might be dangerous to start navigation in an agricultural environment without a previous map, which can still be completed along with robot navigation. Image analysis of high-resolution satellite images could simplify the mapping process and provide a prior map for path planning.

The detection of vegetation characteristics from analysis of aerial images is a general topic for diverse agricultural cultures. Images from Unmanned Aerial Vehicles (UAV) are predominant, but some approaches resort to satellite images. Mougel et al. [22] identifies patterns on a regular flat vineyard and on a peach groove with high-resolution satellite images. Similarly, Karakizi et al. [23] proposes a tool to extract vine canopy from very high-resolution satellite images. The Hough transform method technique is popular in the detection of patterns of points like lines or parametric curves. This method is widely used in the detection of crop lines as diverse plantations are sown in straight lines [24]. A weed mapping system in crops uses images from UAV [25]. In this work, the position of the weeds is provided in relation to the crop lines to improve the discrimination of weeds. This way, the authors have a precise method for detecting crop lines based on the Hough transform. The problem of detecting crop rows is also common in vineyards, having different studies with various approaches. Delenne et al. [26] delineates a vineyard from aerial images recurring to a row extraction tool. This tool starts by considering that all the rows are parallel, and fills the parcel with several orientated lines, eliminating false rows applying minimum local identification. An approach to detect vine block, rows and individual trees combines threshold and graph-based procedures from multispectral images [27]. Poblete et al. [28] detected vine rows in ultra-high-resolution images taken with an UAV. The authors benchmark different methods, such as k-means cluster, artificial neural network, random forest and spectral indices. Their conclusion indicated that all the methods had acceptable performances, except for the k-means. A skeletonization method with high-resolution UAV images was an approach chosen to simplify agricultural scenes, thus helping in the classification of different features, including like vine rows [29]. Comba et al. [30] presents an image processing algorithm to segment vine rows from UAV images. This work follows three different approaches: dynamic segmentation, Hough space clustering and total least squares. The authors claim to obtain an image that could be explored for robotic path planning. However, this is just applied to at regular vineyard with total straight line vegetation. To the best of our knowledge, the segmentation of vine-rows independently of their “configuration”, like steep slope vineyards, has not been addressed in the literature. Path planning operations in autonomous robotic navigation systems may be affected by the dimensions of agricultural fields. To store information about all of the surrounding environment (e.g., occupation grid map) requires a lot of computational memory. For example, a steep slope vineyard from a small producer has an area around 1 hectare, and big brand producers farms reach up to 70 hectares [31]. Dividing the space into smaller zones can help to solve this issue. Such a thing can be achieved with topological maps [32], which describe the world with vertices and edges instead of using a metric system like occupation grid maps from cell decomposition planners. There are various approaches for autonomous navigation and localization with topological maps [33,34]. Thrun et al. [35] extracts a topological map from a grid map using the Voronoi diagram, which consists of the division of the space by Voronoi segments. These segments represent all the points in the plane equidistant do the nearest sites. Graph partitioning techniques, like spectral clustering, are also referred to construct a topological map, starting from a method that subdivides the environment into a set of sub-maps [36]. Konolige et al. [37] proposes a navigation system with Dijkstra search algorithm using a hybrid map that contains a topological map and a grid map. The robot navigates locally with the grid map, and the global planning is performed in the topological map, generating a near-optimal path. In previous works, Santos et al. [38] resorted to the Voronoi diagram to create a topological map from 2D or 3D maps directed to indoor and structured environments, using a door detection method to finish the place delimitation. More recently, following a similar approach, this concept was adapted to a steep slope vineyard map [39]. However, the method is not fully adequate for these environments and needs further improvement. For example, the topological map in these previous works contains visible outliers, and the place delimitation is just present as a concept.

This work will extend the state of the art approaches to enable the extraction of grid-maps by considering aerial and satellite images (without a need to the robot visit the farm). Besides, it will extend the state of the art algorithms, to extract topological maps useful for improving path planning and localization performance in autonomous robotic systems.

3. Agrobpp-Bridge: Agrob Vineyard Detector

The segmentation task of vineyards in satellite images is divided into two stages: detection of a full vineyard crop in satellite images, and segmentation of paths and vine vegetation to construct a prior occupation grid map. The first stage was performed in a previous work [40] recurring to an SVM classifier. Now, in the second stage, we bench-marked two segmentation tools: “AgRob vineyard Detector”, our developed SVM tool, and “Semantic Segmentation Suite”, a state of the art framework with Tensorflow.

AgRob Vineyard Detector is the developed framework that contains an annotation tool to create image datasets and a segmentation tool. We considered a two classes classification problem: “Vineyard” and “Path” (not “Vineyard”).

3.1. Segmentation Tool

For the segmentation process, we use an SVM classifier that runs on Robot Operation System (ROS) (http://www.ros.org/.). The input of this tool is a region descriptor extracted from the image. Based on the training step, the SVM tool is able to classify the image pixels according to a class object. Figure 1 depicts the information stream of the classification process.

The region descriptor is based in Local Binary Pattern codes (LBP), a grey-level invariant texture primitive. Ojala et al. [41] presented the non-parametric LBP operator for textured image description. Originally, the LBP uses a grid of $3\times 3$ pixels for an arbitrary pixel over an input grey-level image. The LBP code is computed by comparing the grey-level value of the centre pixel and its neighbors within the respective grid. The pixels that are not covered by the grids are estimated by interpolation. Then, the LBP code is a binary number which results from a threshold stage concerning the centre pixel. The image texture is described with an LBP histogram (hLBP), built from all binary pattern of each image pixel as shown in Equation (1), where K is the maximal LBP pattern value. Based on hLBP, we considered the descriptor hLBP by color, as in Figure 2, which contains one LBP histogram per color, discretizing the color ranges into n colors in RGB (Red, Green and Blue) space. With this descriptor, each pixel is related to a color range, which will increment the histogram bin related to the LBP code extracted for that pixel. This descriptor will feed the SVM classifier. Here, the descriptor was modified to optimize the detection in vineyards. So, we concatenated the two descriptors, as shown in Figure 2, to describe the centre and its surroundings. The proposed descriptor considers histograms to describe patterns (LBP) and color. The vineyard rows and the path rows—where machinery/robots can move—have different patterns and colors, which can be easily captured by the presented descriptor. Theoretically, this should work for any permanent woody crops (e.g., Orchards or Olive Groves) because the cultures are disposed in rows (linear and/or contour lines), and the paths are aligned to these rows. However, to extend this work to other agricultural contexts, an extension to the dataset and SVM training may be required. For example, crops with paths with exposed soils (without vegetation), where the soil may have another color, would require this procedure.

$$\left\{\begin{array}{c}H(k)={\sum }_{m=1}^M{\sum }_{n=1}^{N}f(LB{P}_{P,R}(m,n),k), k\in [0,K]\hfill \\ f(x,y)=\left\{\begin{array}{cc}1,\hfill & x=y\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.\hfill \end{array}\right..$$

(1)

SVM is a traditional machine learning technique for classification problems. Despite being adequate for binary classification, some approaches decompose the problem into a series of two-class problems [42], allowing SVM to perform multi-class classification. The SVM concept implements the following idea: input vectors are non-linearly mapped to a high-dimension feature space, where a linear decision surface is constructed [43]. Considering a problem of separating the training data $(x_1,y_1),\dots (x_m,y_m)$ into two classes, where $x_i\in ℝ$ is a feature vector and $y_i\in \{-1,+1\}$ its class label. Assuming that a hyperplane can separate the two classes, $w·x+b=0$ in some space ℍ, the optimal hyperplane will maximize the margin. Change et al. [44] provides a deeper explanation about the SVM theory and its variant libSVM.

3.2. Annotation Tool

The annotation tool consists of a semi-automatic framework developed to ease the creation of training images datasets of vineyards in satellite images. A training dataset is composed of a set of images with a fixed resolution containing examples of images belonging to a specific class. In this case, there is a group of vineyard vegetation images and another group with vineyard paths images. The process of manually annotating these images is time-consuming and can lead to incorrect class annotations, which will decrease the accuracy of the segmentation process. So, as this process is based on the detection of vineyard lines, our annotation tool requires the user to manually draw a set of lines representing the vegetation and path lines of the vineyard, using any image editing tool (i.e., GIMP (https://www.gimp.org/) or Paint). The annotation tool will create a set of training images based in the line annotations with specified window size, as in Figure 3. This process can also be very time-consuming in the case of large irregular fields. Still, the method is always simpler than a complete manual annotation. In this case, with an annotation made entirely by hand, the user would have to select hundreds or thousands of images with pre-defined window size.

The selection of the window size is crucial and depends upon several factors, such as the image resolution and the distance between vine-trees. If this parameter is not correctly defined, an entirely new dataset can be created with our annotation tool in just a few seconds. If we considered a full manual annotation, the entire process would have to be restarted. Ideally, the size of this window should be enough to cover two vineyard lines. The distance between vineyard rows changes significantly between farms, and satellite images have very different resolutions. So, we have applied the fast Fourier transform (FFT) on the input images to obtain the distance between crop tree rows (in pixels units) and used this value to scale our descriptor window according to the image resolutions and distance between crop trees rows. An FFT is an algorithm that computes the discrete Fourier transform (DFT) of a sequence. Fourier analysis converts a signal from its original domain (in this work, pixels space) to a representation in the frequency domain. The higher frequency with high magnitude is correlated to the crop row spacing. So, to estimate the space between two consecutive vine lines, we calculate the Fast Fourier Transform (FFT) of various columns and rows of a grey-scale version of the image. As represented in Figure 4, eight FFT from four different columns and rows are calculated.

To estimate the desired width, the steps below are executed several times, which will provide different measurements, presenting to the user an average value and the value obtained at each estimation.

• Choose a column and a row of the selected image zone and calculate their FFTs.
• Choose the FFT with maximum magnitude value at the maximum index as this will be closer to the heading of the image.
• Calculate the distance between two lines: $width$ $=\frac{FF{T}_{size}}{Inde{x}_{max}}$.

3.3. Segmentation Semantic Suite

The Segmentation Semantic Suite [13] is an open-source tool constructed to quickly implement, train and test semantic segmentation models in TensorFlow. Tensorflow is one of the most popular Deep Learning (DL)-oriented frameworks. It allows to create, train, and execute models that can be transferred to heterogeneous devices. With Tensorflow, Convolutional Neural Networks (CNNs) can be used to perform image classification, object detection, and semantic segmentation. This segmentation tool performs automatic data augmentation, a process to enlarge the training dataset by applying a series of random transformations to the original images, such as rotation and translation. It also includes various state-of-the-art models for feature extraction, such as MobileNetV2 [45] and ResNet50/101/152 [46], as well as several segmentation models, like Mobile UNet [47] and Fully Convolutional DenseNet [48]. For the case study of this article, we considered the frontend feature extractor ResNet101 and the segmentation model FC-DenseNet103.

4. AgRobPP-Bridge—AgRob Grid Map to Topologic

AgRob Grid Map to Topologic is a framework developed to deal with big dimensions maps in autonomous robot navigation. As mentioned in our previous work [12], path planning in terrains with large dimensions is complex in terms of memory. This approach automatically divides an occupation grid map into smaller zones and finds the different possible connections between those places. Then, this information is saved into a graph struct, which allows the usage of a search algorithm to find the best possible transaction between two zones. In the resulting graph struct, a vertex represents a delimited place of the map, and an edge represents the connection between to vertices containing information about the Euclidean distance, as in Figure 5.

A typical A* search algorithm, in which pseudo-code is represented in Algorithm 1, was the choice to perform the search between two nodes in the graph space. With this, the large map gets reduced to the strictly necessary zones to navigate between two different places. Considering this method, the amount of computational memory gets substantially reduced.

Algorithm 1 A* algorithm [39]

1: Add origin node to O (Open list) 2: Repeat 3: Choose nbest (best node) from O so that $f(nbest)\le f(n)\forall n\in$ O 4: Remove nbest from O and add it to C (Closed list) 5: if nbest = target node then end 6: For all $x\in Q(nbest)$ which are not in C do:   if $x\notin$ O then    Add node x to O   else if g(nbest) + c(nbest, x) < g(x) then    Change parent of node x to nbest 7: until O is empty

The topological map concept for steep slopes vineyards had been addressed in a previous work [39]. However, this method was more complex and presented some outliers. For example, there were unnecessary and repetitive connections between different nodes. The resemblance to AgRob Grid Map to Topologic is in the Voronoi diagram. Both methods start with the extraction of a Voronoi diagram, but here the places delimitation follow a more straightforward approach. As already mentioned, the Voronoi diagram consists of the division of the space by Voronoi segments and Voronoi vertices. These segments represent all the points in the plane equidistant to the nearest sites, and the Voronoi vertices are the points equidistant to three or more sites [38]. Our previous approach started the construction of the topological map by defining a circle in each Voronoi vertice, then filtering these circles to eliminate overlaps. Until this point, the process in both methods is similar. While the previous work used Voronoi segments to find the connection between circles and resorted to parametric equations for the space delimitation process, the current approach is more simplistic, efficient, and effective, as will be explained below. The result of this method will provide us with a map sub-divided into smaller places, with the possible connections between these zones saved in a data structure. Furthermore, an A* search algorithm is available to search for the best transition between places, which will be useful for future approaches with path planning algorithms.

For the step by step demonstration of this method, an occupation grid map of a simulated steep slope vineyard will be considered, as in Figure 6, where the white color represents a free cell and black color an occupied cell. This image is the result of a 2D projection of the simulated 3D model of a steep slope vineyard created with a modeling software in previous work [11].

4.1. Voronoi Diagram Extraction

The resulting vertices and segments of the Voronoi diagram are represented in Figure 7. Its construction originates from a beforehand distance map, which contains the Euclidean distance of every cell to the closest obstacle. The development of the algorithm was based on the work of Lau et al. [49].

4.2. Topological Map Construction

The visualisation of the topological map is composed of a set of interconnected circles. Each circle represents a certain zone of the map, that it’s connected to the nearest possible circles, according to the occupation grid map. To construct it, the algorithm associates two parameters to each one of the Voronoi vertices, to define a circle: the circle location $\overrightarrow{r_c}=(x_c,y_c)$ that is the same as the Voronoi Vertex, and the circle radius through the distance map, $r_c=ma{p}_{dist}(x_c,y_c)$. With this circle, the algorithm checks, in all of the remaining stored circles, the following condition: $r_c(i)+r_c>=\sqrt{{(x_c-x_c(i))}^2+{(y_c-y_c(i))}^2}$. If the condition is true, the circle with a smaller radius is erased. The result of this operation is illustrated in Figure 8.

The next step consists of finding the connections between the circles. For that operation, all the pixels of each circle will have associated a unique label. Then, all the pixels containing a label are expanded until a different label is found. This operation is similar to a recursive process of a morphological operation of erosion until there are no more pixels without a label associated. The result is visible in Figure 9.

With this image, the process of finding the connections between the circles is simple. It is just necessary to check the zones where the label changes. So, a topological map is constructed and represented in Figure 10.

4.3. Place Delimitation

In this stage, the algorithm takes advantage of the expansion performed before, as in Figure 9, to define delimited places on the map. Then, these places are approximated to the nearest possible rectangle, and this information is saved into the graph struct. At this stage, it is possible to use A* search algorithm to find the best transition sequence between two different nodes. The result of this operation is visible in Figure 11, where A* was used to find the connection between the nodes S80 and S92. The final result presents a map that only contains the strictly necessary zones for robotic navigation between those two nodes.

5. Results

This section presents the results of AgRobPP-bridge. The mentioned segmentation methods, AgRob Vineyard Detector and Segmentation Semantic suite are demonstrated in two different vineyards to extract an Occupancy Grid Map. AgRob Grid Map to Topological map extraction is demonstrated with one of the extracted grid maps. The satellite images of the vineyards are publicly available on Google Maps, and we resorted to a public tool [50] to obtain high-resolution images with the necessary dimensions to cover an entire vineyard. One of the vineyards, as in Figure 12A, is a flat vineyard located at “Quinta da Aveleda” (41.205074, −8.307765), with an area of approximately 5.2 hectares. The other image, Figure 12B, corresponds to a portion of a steep slope vineyard located at “Quinta do Seixo” (41.167574, −7.553293) with an approximated area of 2.3 hectares. Both images were acquired at the maximum possible resolution with 300 pixels per inch.

5.1. Agrob Vineyard Detector Results

Two different training datasets were created with the annotation tool mentioned in Section 3.2, visible in Figure 13. The dataset contains two classes: “Vineyard” which includes the vine-trees, and “Path” that represents everything else. Although the class “Not Vineyard” may not necessarily include a path for robotic navigation, the main goal is to identify the vineyards. As SVM is a tool more suitable for binary classification, we simplified the problem to work with just these two classes. For the vineyard of “Quinta da Aveleda”, we annotated a portion of the image with a window size of $70\times 70$ pixels. The annotation on the steep slope vineyard was performed with a window of $45\times 45$ pixels. The SVM tests were performed in Ubuntu 18.04.3 LTS under ROS Melodic framework, in a computer with Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz × 12, 16 GB of memory. The application runs in CPU without any parallelization.

The accuracy results of the training process are expressed in

Table 1. Confusion Matrix of the SVM training process.

	Classes	N° of Images	Train Images	Test Images	Confusion Matrix		Accuracy (%)
					Path	Vegetation
Aveleda	Path	537	457	80	72	8	93.4%
	Vegetation	684	582	102	4	98
Seixo	Path	607	516	91	81	10	84%
	Vegetation	523	445	78	17	61

with a confusion matrix. About 15% of the images in the dataset are used to test the training process. The confusion matrix table indicates, for example, that, in 102 images belonging to “vineyard”, 4 were wrongly classified as “path”. Ideally, the values outside of the main diagonal of a confusion matrix should be 0.

The images in Figure 14 represent the SVM segmentation result with the flat vineyard at “Quinta da Aveleda” and the corresponding Occupancy Grid Map. The result in the steep slope vineyard is shown in Figure 15. The result is present in the form of a color map, which is related to the probability of each pixel to belong to the class “Vineyard Path”, where blue represents the lowest probability and red the highest. The grid map is obtained through a threshold process in the color map image. Two ground truth images were created, similar to the images in Figure 16 to calculate the accuracy of the method. These images were compared pixel by pixel to the final result of the SVM tool in order to determine quality metrics.

Table 2. SVM Precision. TP—True Positive; FP—False Positive.

	Classes	TP	FP	Accuracy (%)	F1-Score (%)
Aveleda	Vineyard	4,670,106	2,075,617	88.5	66.0
	Path	32,389,277	2,734,308
Seixo	Vineyard	339,554	460,578	87.7	54.8
	Path	3,658,598	100,345

presents the Accuracy and the metric F1-score, common in binary classification problems. This table is similar to a confusion matrix (Table 1), but, instead of presetting the number of images correctly identified, it considers the number of pixels by comparing the image to a ground-truth image. However, such data is not available, so we annotated the images manually and consider them as ground-truth, as in Figure 16. For the accuracy, we consider all the pixels correctly identified out of all pixels in the image. F1-score combines “Precision” and “Recall” metrics with a harmonic average. “Recall” refers to the number of pixels correctly identified as Positive (Vineyard) out of the total True positives (“Vineyard” and false “Path”). The “Precision” is the number of items correctly identified as positive (“Vineyard”) out of all pixels identified as positives (“Vineyard” and false “Vineyard”) [51].

5.2. Segmentation Semantic Suite Results

To compare the results of this tool with the SVM classifier, we created two training datasets using similar information sources, that is, using the same area of satellite images. Here the annotation process was manual and time-consuming because it must be performed to every individual pixel. Each pixel gets associated with a particular color, which is related to a specific class. We considered three classes: Vineyard, Path, and Background. The last one represents everything that is outside of the first two classes. The annotation is illustrated in Figure 16.

Two training processes were performed, and the graphics correlating the average loss and average accuracy with the number of epochs are visible in Figure 17. The prediction results are revealed in Figure 18, and

Table 3. Semantic Segmentation Suite Precision. TP—True Positive; FP—False Positive.

	Classes	TP	FP	Accuracy (%)	F1-Score (%)
Aveleda	Vineyard	5,722,413	910,572	87.4	81.5
	Path	7,361,845	2,244,559
	Background	25,421,493	2,386,748
Seixo	Vineyard	311,261	216,800	73.3	64.3
	Path	1,136,978	718,661
	Background	2,005,063	323,685

presents the accuracy and F1-Score of the prediction similarly to Table 2. The classes Path and Background were considered to be one single class (“Not Vineyard”) to calculate the F1-score. This application requires a GPU, so we ran these tests remotely with Google Colab Platform (https://colab.research.google.com).

5.3. Agrob Grid Map to Topologic Results

The results for this tool were demonstrated along Section 4 considering a simulated map of a steep slope vineyard. The present section presents the results of this tool applied to the occupation grid map obtained in the segmentation of “Quinta da Aveleda” vineyard, as in Figure 15, as this is the most complete grid map with the biggest area. As the image dimensions of the maps are considerable, 6490 × 6787 pixels, it is only possible to highlight part of the result. Figure 19 shows the resulting topological represented by circles and their connections. The place delimitation operation is illustrated in Figure 20. Then, we present an example of a path search operation between two nodes using A* search algorithm in Figure 21.

5.4. Results Discussion

The presented results of satellite image segmentation and topological map extraction are satisfactory. It is possible to extract an occupation grid map from satellite images and create a topological map with the results of the segmentation. The developed SVM (AgRob Vineyard Detector) tool demonstrated a similar performance when compared to a Deep Learning alternative using Semantic Segmentation Suite. Each approach has different characteristics, and

Table 4. Benchmark between the two presented segmentation tools.

	AgRob Vineyard Detector (SVM)	Semantic Segmentation Suite
Training Time	Low	High
Testing Time	High	Low
Computational Resources	Medium	High
Precision	Medium-high	Medium-high
Annotation Process Complexity	Medium-low	High
Annotation Process Time	Medium	High

presents a small benchmark between these methods according to the experience of this work. The training time with the SVM tool takes less than one minute, while, with the deep learning tool, this time can reach several hours, even with the process parallelized in a GPU. Such a thing does not happen in AgRob Vineyard Detector, which is running sequentially in CPU without any parallelization. However, the testing time that takes some seconds with the Semantic Segmentation Suite took about two hours in some of our experiences with the SVM tool. Nevertheless, as already mentioned, this tool is not optimized to reduce the processing time. The annotation process with the Deep Learning approach may be the main drawback. As the tool requires a pixel to pixel annotation, the process was performed manually and took about three hours in each image to be completed. The same process in the SVM took less than one hour, with the help of the annotation tool described in Section 3.2. The precision in both cases is acceptable, even though we are missing a real ground-truth image to make a proper evaluation. The accuracy in the two methods is higher than 73% and reaches 89%, but the F1-Score drops significantly in the SVM tool. This is happening because the precision of the class vineyard is lower than the class “Path”, as visible in Table 2. Such phenomena may be occurring due to the use of a manually annotated image as ground-truth. 80% of the pixels in the ground-truth image marked as vineyard were correctly identified, but this tool identified more extensive lines, which causes the precision to decrease. So, the decrease in F1-score may be caused by human error during the annotation process. With this experience, it could be concluded that AgRob Vineyard Detector is simpler to use and that there are no substantial gains in using the alternative framework. However, such claim can not be fully accepted without a proper evaluation of the data using an accurate ground-truth image. The AgRob Grid Map to Topologic tool revealed good results even when tested in a map with considerable big dimensions. Without this, a path planning algorithm would have to deal with a vast map of 6490 × 6787 pixels to perform a simple operation of finding a path between two near sites. This operation could cause memory problems and affect the performance of the path planning algorithm. Using the topological tool, and considering the most simplistic approach, the path planning tool would work with a much smaller map of 427 × 551 pixels, which represents a reduction of 99.5% of the area.

6. Conclusions

The proposed work presented an approach to deal with big dimensions of agricultural terrain in robotic path planning. For such purpose, we proposed the AgRobPP-bridge, a method to extract an Occupation Grid Map from a satellite image and a tool to construct a topological map from a Grid Map. Based on an SVM classifier, AgRoB Vineyards Detector identifies Vineyards from satellite images and produces an Occupation Grid Map. This tool was bench-marked with an alternative open-source framework, Semantic Segmentation Suite, which is constructed to implement, train, and test segmentation models in TensorFlow. The experience indicated that AgRoB Vineyards Detector is simpler to use, requires less computational resources, and gives a similar accuracy when compared with Semantic Segmentation Suite. However, as there is not a real ground-truth image, it is not possible to assure a reliable precision metric. For this purpose, the results were compared with a manually annotated image, being that, the resolution is not sufficient to ensure correct annotations. To the construction of the topological map began with the extraction of a Voronoi diagram, ending with the map with delimited places saved in a graph structure, with a simple A* search algorithm to find the best transition between different places. The experiments showed promising results when dealing with significant large maps. The tool is capable to efficiently extract the topological map, delimit the areas according to the nodes of the topological map, and search for a transition path between two different nodes. As future work, we will test the segmentation tool with higher resolution images obtained from a drone or an aeroplane and construct a reliable ground-truth image using land sensors from a ground robot. The topological tool will be applied to a path planning framework in order to solve computation memory problems when dealing with big maps.