Coverage Path Planning with Semantic Segmentation for UAV in PV Plants

Abstract

Solar energy is one of the most strategic energy sources for the world’s economic development. This has caused the number of solar photovoltaic plants to increase around the world; consequently, they are installed in places where their access and manual inspection are arduous and risky tasks. Recently, the inspection of photovoltaic plants has been conducted with the use of unmanned aerial vehicles (UAV). Although the inspection with UAVs can be completed with a drone operator, where the UAV flight path is purely manual or utilizes a previously generated flight path through a ground control station (GCS). However, the path generated in the GCS has many restrictions that the operator must supply. Due to these restrictions, we present a novel way to develop a flight path automatically with coverage path planning (CPP) methods. Using a DL server to segment the region of interest (RoI) within each of the predefined PV plant images, three CPP methods were also considered and their performances were assessed with metrics. The UAV energy consumption performance in each of the CPP methods was assessed using two different UAVs and standard metrics. Six experiments were performed by varying the CPP width, and the consumption metrics were recorded in each experiment. According to the results, the most effective and efficient methods are the exact cellular decomposition boustrophedon and grid-based wavefront coverage, depending on the CPP width and the area of the PV plant. Finally, a relationship was established between the size of the photovoltaic plant area and the best UAV to perform the inspection with the appropriate CPP width. This could be an important result for low-cost inspection with UAVs, without high-resolution cameras on the UAV board, and in small plants.

1. Introduction

According to REN21, over the past two years, global photovoltaic (PV) plants capacities and annual additions have grown and expanded rapidly. For instance, 621 GW were installed in the year 2019 and 760 GW in the year 2020 [1], despite the reduction in electricity consumption and shifted daily demand patterns due to the COVID-19 pandemic [2]. Additionally, it has become one of the most profitable options and is an energy resource that has recently decreased in cost. As a result, solar electricity generation has grown in residential, commercial, and utility-scale projects [3]. The future of PV generation will focus on optimizing hybrid systems [4] and improving the performance of each element of the system, as well as reducing their cost due to large-scale production [5]. Furthermore, the sector trend has been asking for low prices, and the competitive market has encouraged investment in solar PV technologies across the entire value chain, particularly in solar cells and modules, to improve efficiencies and reduce the levelized cost of energy (LCOE) [1]. As a result, PV power plants could grow almost sixfold over the next ten years, reaching a cumulative capacity of 2840 GW globally by 2030 and rising to 8519 GW by 2050, according to [6].

Thus, it is established that the number of PV plants and generated power have increased around the world. This implies specific technical challenges in their maintenance and operation (O and M) [7]. Some of these threats affect their production, which increases cost, and decreases profitability. The most common threats are the failures in the inverters and PV modules [8]. Through dirty equipment, the state of the environment, or manufacturing defects in the PV modules, PV plant energy generation can be curtailed by 31% in the worst cases [9,10,11].

One must consider that PV plants are commonly installed on roofs, rooftops, canopies, or facades for urban environments. Likewise, solar farms utilize rural environments, such as deserts, plains, and hills [7,12,13]. Depending on the location of the PV plant, the manual inspection tasks could be exhausting and take up to 8 h/MW, if the number of modules is considerable. The amount of inspection time increases for solar PV plants on rooftops or canopies, by virtue of aspects of their installation [14]. In addition, inspections to detect threats in the panels must be conducted by trained personnel. In some cases, problems occur in elevated installations, for which special training and certification is required. These jobs could put people and facilities at risk [15].

In recent decades, unmanned aerial vehicles (UAVs) have been increasingly used in inspection and patrol tasks [16,17,18]. UAV-based applications for PV plant inspections have many advantages in comparison with the manual inspection methods. The main advantages are flexibility, lower cost, larger area coverage, faster detection, higher precision, and the capacity to perform a superior and automatic inspections [11,17,18,19,20,21].

There are many approaches to performing an inspection with UAVs in PV modules. One of them used UAVs with a thermal imaging camera to take photos in the infrared spectrum to evaluate the UAVs parameters, such as height, speed, viewing angle, sun reflection, irradiance and temperature, all of which are necessary to perform defect inspection [22]. In a second approach, UAVs were implemented to inspect different solar PV plants, wherein analysis of the correlation between altitude and the pixel resolution was used to detect PV panel defects and features like shape, size, location, and color, among others, of a particular defect were also detected [7,23,24,25,26]. In a third approach, the authors proposed PV plant fault inspection with UAVs using image mosaics combined with orthophotography techniques to create a digital map; image mosaics were combined with multiple visible and/or infrared range images into a single mosaic image covering a large area [14,27,28]. Whereas the orthophotography technique is a vertical photograph that shows images of objects in true planimetric position [29]. Thus, these two techniques were integrated with previous works to achieve an advanced tool that allows monitoring and taking actions in the operation and maintenance (O and M) of PV plants [7]. In short, this tool has been used to detect defects and dust or dirt in PV modules [28]. Apart from this, some approaches have been developed for the detection of defects in PV modules using artificial vision techniques, machine learning, deep learning, and the integration therewith of the previous approaches [7,24,30,31,32,33]. Additionally, these approaches to planning the UAV’s flight path were configured from a ground control station (GCS) program. Irrespective of these approaches, an important aspect is that the UAV should automatically follow the path to cover important points in PV plants. Many research efforts have been made to calculate the paths of and solve the waypoint planning problem for UAV inspection of solar PV plant applications [34,35,36,37], but none of them propose the coverage path planning (CPP) as a method to complete this task.

The CPP, given a region of interest (RoI) in a 2D environment, consists of calculating the path that passes through each one of the points that make up the desired environment and must be found considering the limitations of movement [38,39]. CPP is classified as a classical NP-hard problem in the field of computational complexity. These problems were initially analyzed for indoor environments with mobile robots. However, with the development of GPS, CPPs began to be used for missions with UAVs. Due to the environment in which the task is performed and the obstacles present, precise localization in the environment is an arduous task, which makes the CPP a difficult problem [40]. Additionally, it is classified as a motion planning subtopic in robotics, and has two approaches, heuristic and complete. In heuristic approaches, the robots follow a set of rules defining their behavior, but do not present a guarantee for successful full coverage. These guarantee using the cellular decomposition of the area, which involves space discretization into cells to simplify the coverage in each sub-region, unlike complete methods, which cannot afford such processing. Another important issue mentioned by the authors of reference [39] is the flight time to fully cover the area, which can be reduced using multiple robots and by reducing the number of turning maneuvers. Finally, the available RoI information is important; several approaches accept previous knowledge of the robot’s respecting the search RoI (offline), while sensor-based approaches collect such information in real-time along the coverage (online) [41].

In the literature, CPP approaches are needed in several application areas, such as floor cleaning [42], agriculture [43,44], wildfire tracking [45], bush trimming [46], power line inspection [47], photogrammetry [39], visual inspection [48], and many more. Additionally, many surveys regarding CPP present several approaches and techniques for performing missions with, mostly, land vehicles [41,49,50]. The research interest in aerial robots (indoor and outdoor) has surely motivated the research of CPP [51]. This can be implemented in many UAVs platforms, such as fixed-wing, rotary-wing, and hybrid UAV (VTOL) [39]. Rotary-wing UAVs are inexpensive and have good maneuverability, and their small payload capacity limits the weight of on-board sensors and flight time. Hence, they are more suitable for CPP missions on a small scale. Additionally, the increasing usage of UAVs in applications with complicated missions has led to CPP methods being a very active research area for single and multiple UAVs, especially recently [39,52,53,54], As evidenced in a previous work [43], the classic taxonomy of coverage paths in UAVs are classified into no decomposition, exact cellular decomposition, and approximate cellular decomposition. The first performs the coverage with a single UAV, for which no decomposition technique is required, because the shape of the RoI has a non-complex geometry. The second divides the free space into simple, non-overlapping regions called cells. The union of all cells fills the free space completely. The cells in which there are no obstacles are easy to cover and can be covered by the robot with simple movements. The third is based on grids. They use a representation of the environment decomposed into a collection of uniformly squared cells [55], considering rectangular, concave, and convex polygons for RoIs. In addition to this, CPP performances were assessed with applied metrics according to [39]. The elemental approach most used to solve offline CPP problem sis the area decomposition into non-overlapping sub-regions [56], to determine the appropriate visiting sequence of each sub-region and to cover each decomposed region in a back-and-forth movement to secure a complete coverage path. As a result, the methods for obtaining complete coverage of an RoI are the exact and approximate cellular decomposition methods [57,58,59].

On the other hand, image processing helps to obtain a map of the robot or RoI. Robots, such as UAVs, need to know the RoI before commencing CPP [60], which represents where the PV plants are and can be determined in a process called boundary extraction [61,62]. Then, the deep learning (DL) image segmentation technique, also known as semantic segmentation [63], is achieved by applying deep convolutional neural networks (CNN), such as the U-Net network model [64,65] or the FCN model [66], which dramatically enhance the segmentation results. Once the segmentation is done and the mask is obtained or the RoI is identified, the GCS calculates the CPP that guides the UAV in the automatic plant inspection, during which it captures images of PV plants [67]. Most of the failures occur at the centimeter or millimeter level, and that poses a challenge for the inexpensive sensors available today [68]. Tests of coverage paths it can be conducted with drones in a real or simulated environment. The most viable option for this stage of the work is simulation, as verified by other research [69,70,71,72]. Simulation has been recognized as an important research tool; initially, simulation was an academic research tool, but with the advancement of computers, simulation has reached new levels. It is a remarkable tool that guarantees support in design, planning, analysis, and making decisions in different areas of research and development. Of course, robotics, as modern technology, has not been an exception [73,74].

This work is focused on implementing the best strategy of coverage path planning (CPP) over PV plants with UAVs using semantic segmentation in a deep learning server to obtain the RoI. The experimental results were obtained by simulating the CPP methods and using UAVs, such as the 3DR Iris and Typhoon aerial robots.

The key contributions of this work are as follows:

• This work proposes CPPs as a novel strategy for conducting an inspection flight over a PV plant with an UAV, since there are no previous reports of such work.
• This work used three CPP methods over three PV plants, which were modeled in a simulation environment to evaluate metrics and parameters. As a result, a relationship was found between the CPP width and energy consumption, and according to that, the best CPP method to implement.
• This work proposes a hybrid CPP method that uses image processing and a DL server to find the RoI quickly and accurately, becoming a semi-automatic process.
• A free simulation tool is provided, with an interface to simulate the inspection of PV plants with UAVs.

This paper is structured as follows. In Section 2, necessary definitions and the techniques used to obtain the results are described. In Section 3, the three CPP methods implemented are compared to show relationships among the CPP width, number of maneuvers, and energy consumption, with the aim of finding the best CPP method to implement. Finally, in Section 4, some conclusions are given.

2. Materials and Methods

In this work, three PV power plants were selected that met the image requirements of no light distortions, non-complex geometry, and grouped panels. These plants are in different parts of the world. The first PV plant has an area of $35,975$ square meters, located in Brazil (−22.119948323621525, −51.44666423682603), known as Usina Solar Unioeste 1 [75]. The second PV plant has an area of $25,056$ square meters, located in Iran (34.0504329771808, 49.796635265177294), known as Arak power plant [37]. The third plant is in the United States (38.55989816199527, −121.42374978032115); the plant has 1344 square meters of area, and it is part of a PV plant located on the roof of the California State University Sacramento Library (CSUSL) [76].

These plants were subjected to a series of processing stages, as shown in Figure 1. The RoIs were obtained from Google Maps satellite images with a predefined altitude, and limits, from which the image (input image) of the desired PV system was obtained as the first stage, shown in Figure 1. In the second stage, the image was entered into a DL server that was developed in this work, taking into account the previous work [65]. The DL server was launched with TensorFlow [77] and Flask [78]. This stage obtains a mask (image segmentation) of the PV plant, as shown in Figure 1. In the third stage it was necessary to apply a series of OpenCV functions (post-processing), as referenced in Algorithm A1 in the Appendix A, to adjust the mask (output mask) to the PV plant area, as seen in Figure 1. Then, the output mask or RoI was introduced in the GUI interface, where the CPP method was selected to internally execute. Later, the path GPS positions (CPP computed) were sent to the UAV through MAVlink commands [79,80]. The UAV executed these commands in the Gazebo platform (CPP simulation) and, at the same time, the simulation data was fed back the GUI interface. Simultaneously all GPS points reached by the UAV were drawn in the GCS platform (CPP). In this stage, the trajectory was validated. Each stage is described in greater detail in the following sections.

2.1. Deep Learning (DL) Server for Segmentation

A deep learning (DL) server for segmentation was necessary to extract the RoIs from the Google Maps images. Additionally, the server was used to achieve this task automatically with the process called semantic segmentation, wherein each pixel is labeled with the class of its enclosing object [65,66]. In previous work, a convolutional neural network was proposed, wherein a public database was used; this data was prepared, resized for training, and assessed with two network structures. The U-net network had the best performance, in terms of metrics, in the semantic segmentation task [65]. Then a DL server was employed to perform image segmentation and obtain the RoI [81].

2.2. Post Processing

In this step, a set of OpenCV functions were applied with Python 3.7. For example, in the first function, morphological operators like “Erode” and “Dilate” were applied to the images, then the “FindCountours” function was applied to help extract their contours. The contour can be defined as a curve that joins all the continuous points at the boundary of the PV installation. So, the “ContourArea” function was then used to find the area of the previous contour. Following this pattern, the area was compared with 400 others to filter the bigger area and eliminate the little areas belonging to false positives. Then the “ApproxPolyDP” function was used to approximate a shape of the contour to another shape with fewer vertices. Subsequently, the “DrawContour” function was used to draw the resulting contour [82,83]. Finally, the “Erode” morphological operators were used again, to expand the known area and compensate for the limitations of the mask with regard to the CPP method and some faulty occurrences caused by the false positives of the DL server. The pseudocode of the openCV functions used is shown in Algorithm A1 in the Appendix A.

2.3. 2D Coverage Path Planning Method in the GUI Interface

In previous studies of CPPs, there were many existing methods from which to select to solve the CPP problem. In this work, three methods based on CPP were selected, considering the following criteria: time of execution, ease of implementation, and more, which were used to cover the RoI. The methods were selected according to [38,39].

The boustrophedon exact cellular decomposition (BECD), which was proposed by [84], was the first selected. The CPP method is delineated in Figure 2.

The second was grid-based spanning tree coverage (GBSTC), which works with cellular decomposition, first proposed by [85], and depicted in Figure 3.

The third method selected for this project was grid-based wavefront coverage (GBWC), first proposed by [86]. The method is illustrated in Figure 4. Each of these CPP methods is explained in more detail in the following sections.

(a). The boustrophedon exact cellular decomposition (BECD) Method: This method takes the robot’s free space and obstacles and splits them into cells. These cells are covered by the robot using a back-and-forth pattern from the initial point to the final point, using maneuvers of 90 degrees to change direction from south to north or vice versa, as shown in Figure 2. This method improves the trapezoid decomposition technique, as it exploits the structure of the polygon to determine the start and end of an obstacle, and thus is able to divide the free space into a few cells that do not require a redundant step, and it permits the coverage of curved areas [84].

(b). Grid-Based Spanning Tree Coverage (GBSTC): This method is based on approximate cell decomposition and differs from the previous method in that the following postulates were considered. First, the method divides the space into grids of side L. Second, the robot only moves in perpendicular directions to the sides of the grid. Third, every grid is subdivided into four grids of side $L/2$. Finally, GBSTC discards space that is partly occupied by obstacles. Consequently, considering these previous postulates, the method consists of several stages: in the first stage, a graphic structure is defined, $S(N,E)$, where N is nodes, defined as the central point of each grid, and E is edges, defined as the line segments connecting the centers of adjoining grids, as shown in Figure 3a. In the second stage, the method builds a spanning tree for S, and employs this tree to plan a cover path as follows. Starting in grid I with a sub-grid of side $L/2$, the robot begins by travelling between adjoining sub-grids along a path that moves around the spanning tree at a constant distance and in a counterclockwise direction, finishing when the initial sub-grid, I, is found again, which means it is also in the final point, F [85]. An example of this method is illustrated in Figure 3b. The approximation depends on the side length, L, of the grid.

(c). grid-based wavefront coverage (GBWC): The first grid-based method proposed for CPP, GBWC is an offline method that uses a grid representation, in addition to applying a full CPP method. The method requires an initial grid, I, and a final grid, F. A distance transformation that propagates a wavefront from the final to the initial point is used to assign a specific number to each item on the grid. That is, the method first assigns a zero to the final item, and then a one to all its surrounding grids. Then all unmarked grids adjoining those marked one are numbered two. The process is incrementally repeated until the wavefront reaches the initial grid [86], as illustrated in Figure 4a.

Once the distance transformation is determined, a coverage path can be found by starting at the initial grid, I, and selecting the adjoining grid with the highest number that has not been explored. If two or more are unexplored, and adjoining grids share the same number, one of them is randomly selected, as shown in Figure 4b.

2.3.1. Metrics

The metrics evaluate the performance of the three CPP methods. Such assessment can be performed by considering five commonly used metrics to evaluate the effectiveness of the proposed CPP methods both theoretically and in simulation (dynamically) [7,41,43,62]. These five metrics are covered path length, flight time, energy consumption, redundancy of points traveled, and percentage of coverage of the total area. Each of these metrics is described below.

The $L_{ct}$ metric (covered path length) is the length of the entire path covered by the UAV from the initial to the final point. For a trajectory in a 2D plane composed of n points, assuming the initial point as $(x_1,f\left(x_1\right))$ and the end point as $(x_n,f\left(x_n\right))$, the $L_{ct}$ can be computed as shown in Equation (1):

$$L_{ct}=\sum _{i=1}^{n-1}\sqrt{{(x_i+1-x_i)}^2+{(f(x_1+1)-f\left(x_i\right))}^2}$$

(1)

where $(x_i,f\left(x_i\right))$, in which $i=1,2,\cdots ,n$, representing all the n points of the UAV flight path in 2D coordinates. More details can be found in [87].

The flight time metric of the PX4 SITL Gazebo model is the time required to travel the total flight path (takeoff, path travel, and landing) with a dynamic speed that considers the inertia and the variation in speed due to UAV turns angles. These data are collected through sensors in the Gazebo plugins [88].

The redundancy of points traveled, $R_{\%}$, corresponds to the number of points that are visited more than once from the total number of points that the path contains, Equation (2).

$$R_{\%}=\frac{P_{pc}}{P_{vmo}}$$

(2)

where $P_{pc}$, and $P_{vmo}$ correspond to the number of points the trajectory contains, and the number of points visited more than once.

The percentage of coverage of the total area, $C_{\%}$, measures the number of effectively covered points within the total number thereof by the points the area to be covered contains, given by Equation (3).

$$C_{\%}=\frac{P_v}{P_T}$$

(3)

where $P_v$, and $P_T$ correspond, respectively, to the total number of points visited and the total number of points in the area.

Moreover, the number of maneuvers metric, which is the number of turning maneuvers the UAV performs on a path, is often used as the main performance metric in coverage [89,90].

The energy consumption metric is computed from the voltage and current data from the power module of the PX4 SITL model [91]. Its value depends on parameters, such as the CPP width between lines and the speed and the height of the UAV at the time of implementing the CPP, which were configured in the interface and were simulated in Gazebo.

To validate the results of the methods described above, the BECD, GBSTC, and GBWC methods were implemented in two UAVs, simulated in Gazebo, and their performances were assessed by the metrics presented above [39]. The next section describes the results and compares the models in detail.

2.4. Simulation and Validation Platform

Based on [92], the Gazebo platform was selected to execute the simulation experiments, as it has extensive documentation on its webpage. In addition, it is the most mentioned and used simulation platform in previous work [73,74] that implemented path planning or CPP, used UAV sensors, or deployed several UAVs [92]. Gazebo also allows the modeling of commercial UAVs using the PX4 autopilot software [91], as shown in Figure 5. In addition, this figure shows one of the experiments conducted with the Typhoon UAV, flown over the CSUSL plant. The UAV sonar sensor, represented by blue lines, is also shown, as is the UAV camera, in the box at the upper right.

The integration of Gazebo [92], PX4 [91], Python, and the CPP methods was implemented using ROS (Robot Operating System) as middleware [93]. This tool allows communication among nodes. The nodes are processes, and each node has a task associated with it, such as sending a Mavlink command to control the UAV trajectory and permitting reading messages from the UAV to discern its status in flight, using a simulation mode referred to as software in the loop (SITL). This simulator provides the ability to run different vehicles, such as a plane, copter, or rover, without a need for any microcontrollers or hardware [94]. In addition, two PX4 autopilot rotary wing UAVs, the 3DR Iris and Typhoon UAV, were chosen because they have good maneuverability and are more suitable for small-scale CPP missions. Furthermore, these UAVs have been widely used in other research [67,70,95].

The GCS software (QGroundControl) was selected to validate the CPP calculated for each UAV and PV plant obtained, because it is the most compatible tool with the PX4 autopilot. It is also recommended on the PX4 webpage. On other hand, another compatible tool, namely a GUI, was designed using Qt to modify and vary the parameters and to convert the way points from RoI pixels to geo-referenced points [96].

3. Results and Discussion

The procedure described above—DL-server, post-processing, CPP, and metrics evaluation—are represented in Figure 1, was applied to three different PV plant images. The results obtained are described in the following sections.

3.1. Results with Deep Server, and OpenCV Functions

OpenCV functions and a DL server were combined to accurately extract the mask and to trade off the DL server results of the PV plant area. Then, the CPP methods used the mask as a region of interest (RoI).

The stages to obtain the results of the RoI in the three images are shown in Figure 6, Figure 7 and Figure 8. In the first stage, a high-resolution image from Google Maps, of predefined height and width, was obtained, and used as an input image in the DL server, depicted in Figure 6a. In the second stage, the output image of the DL server was the mask, shown in Figure 6b. In the third stage, the opening function was used in the mask, Figure 6c. In the fourth stage, the RoI was obtained using the draw contour method, as seen in Figure 6d. Finally, the RoI was blended with the input image with the aim of comparing the results in Figure 6e. The results were satisfactory and can be adapted, depending on the environment.

3.2. Results of the CPP Method

The CCP method results were obtained from six experiments that implemented the five metrics previously described. Each experiment was conducted by selecting a PV plant and a UAV, then choosing the CPP method and the width, speed, and height of the UAV over the flight path. All of this was done in the implemented GUI interface. The six tests were conducted by varying the CPP width parameter, while other parameters were not varied, and the same experimental conditions were maintained throughout.

For these experiments two UAVs were selected, the Typhoon [97] and 3DR Iris [98], as mentioned in Section 2.4. Three simulated PV plants were also chosen, as highlighted in Section 2. The metrics referenced in Section 2.3.1 were assessed for each test. Battery consumption was obtained from the GCS software with the SITL parameter activated. The rest of the data was collected from the Gazebo simulations. The experiments are explained in detail in the following sections, and some results are discussed.

3.2.1. The Three First Experiments with a Typhoon UAV

The three first CPP experiments were conducted with a Typhoon UAV, varying the CPP width between 0 and 20 m, depending on the PV plant being covered. For example, in the first and second PV plants, Unioeste 1, and Arak, respectively, the CPP widths were varied between 5 and 20 m. In the third CSUSL PV plant, the width was varied between 1 and 8 m, due to the method’s restrictions on running in small areas with a large width. Then, in each of these experiments, the resulting metrics were annotated in Tables, as shown in Appendix B.

The logged metrics were utilized to draw a clustered columns and lines chart for experiments 1, 2 and 3, performed with the Typhoon UAV, to highlight the most important information and to determine their correlations with each other. In these graphs the vertical axis was scaled logarithmically and the horizontal axis was labeled with the type of CPP method in use; in this way, it is possible to visualize the correlation of flight time, number of maneuvers, and path length covered with the energy consumption by each CPP method, as shown in Figure 9, Figure 10 and Figure 11. In addition, the relationship between the redundancy metric and the CPP width is identified, showing that this metric is higher in BECD, but does not affect energy consumption, which is the primary metric of interest.

The UAVs’ energy consumption, in conjunction with the CPP widths, were obtained from the energy consumption results logs, then a new

Table 1. Comparison of three CPP methods with regard to CPP width, for a simulated Typhoon UAV at three PV plants.

		Energy Consumption
PV Plant		Unioeste 1 (35,975 m${}^{\mathbf{2}}$)			Arak (25,056 m${}^{\mathbf{2}}$)			CSUSL (1344 m${}^{\mathbf{2}}$)
	CPP	BECD	GBSTC	GBWC	BECD	GBSTC	GBWC	BECD	GBSTC	GBWC
Width (m)
1		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	35%	47%	43%
2		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	20%	23%	24%
3		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	14%	15%	15%
4		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	11%	12%	12%
5		98%	100%	100%	80%	94%	94%	$N/A$	$N/A$	$N/A$
6		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	9%	8%	8%
8		74%	85%	88%	52%	62%	63%	7%	6%	5%
10		56%	62%	63%	$41.5$%	45%	43%	5%	$N/A$	5%
12		47%	54%	55%	36%	37%	35%	$N/A$	$N/A$	$N/A$
15		38%	43%	45%	27%	29%	26%	$N/A$	$N/A$	$N/A$
20		29%	32%	31%	21%	20%	19%	$N/A$	$N/A$	$N/A$

was composed of three large columns, corresponding to all the assessed PV plants, and each PV plant column is made up of three CPPs with their respective values of consumed energy. The rows of the table contain the CPP widths in increasing order from top to bottom, as shown in Table 1. Where it can be observed, some parameters, such as the energy consumption with regard to CPP width, were due to the size of the RoI. Additionally, it can be observed that the greater the CPP width, the lower the energy consumption, as seen in the columns of Table 1. For example, for the BECD CPP in Unioeste 1, with a width of 5 m, the percentage of energy consumed was 98%, while, for a width of 20, the energy consumed was 29%. It is also observed that the larger the PV plant, the UAV consumed all its energy in CPPs with narrow widths; on the other hand, if the plant is small, with the same width, the UAV does not have energy consumption problems. As can be seen in the row with a width of 8, where, for the Unioeste 1 and Arak PV plants, a lot of energy was consumed, between 88% and 52%, unlike the CSUSL plant, in which where the energy consumed was very little, between 7% and 5%. In the experiments for which energy consumption, with respect to CPP width, cannot be observed, ($N/A$) was used to annotate these results, which happened when the CPP width was very large with respect to the RoI, as this does not allow the generation of the route, or when the CPP width was very small with regard to the RoI causing a flight path in which the UAV consumes all its energy. In short, the UAV used can be undersized or oversized with respect to its intended PV plant.

The graphs in Figure 12 were constructed from Table 1 by quintic polynomial interpolations, which requires six data points to form a curve that passes through all given data points [99], where the abscissa axis is the CPP width and the ordinate axis is energy consumption. For the experiment conducted at the PV plant Unioeste 1, the graph in Figure 12a shows that the BECD method had the lowest energy consumption when the CPP width was in the range of 5 to 16 m, while GBSTC and GBWC had the lower energy consumption when the CPP width was in the range of 16 to 20 m. For the experiment tested at the Arak PV plant, the graph in Figure 12b shows that the BECD method had the lowest energy consumption when the CPP width was in the range of 5 to 10 m, while the GBSTC and GBWC had similar energy consumption when the CPP width was in the range of 10 to 15 m, and GBWC had the lowest energy consumption when the CPP width was in the range of 15 and 20 m. For tests conducted at the CSUSL PV plant, the graph in Figure 12c shows that the BECD method had the lowest energy consumption when the CPP width was in the range of 1 to $5.5$ m, while the method GBSTC had the lowest energy consumption when the CPP width was in the range of $5.5$ to 8 m. All plants were recreated in a simulation environment with their real dimensions.

3.2.2. The Last Three Experiments with 3DR Iris UAV

The last three CPP experiments were conducted with a 3DR Iris UAV, varying the CPP width between 1 and 20 m, depending on the PV plant covered. For example, in the first and second PV plants, Unioeste 1 and Arak, respectively, the CPP widths were varied between 8 and 20 m; in the third PV plant, CSUSL, the width was varied between 1 and 8 m, due to the restrictions of the method requiring it be run in small areas with a large width. Then, in each of these experiments, the resulting metrics were annotated in tables, as shown in Appendix C.

As in the previous chapter, the recorded metrics were used to create bar and line graphs for Experiments 4, 5 and 6, which are those conducted with the 3DR Iris UAV, to highlight the most important information and to determine correlations among the metrics. In these graphs, the vertical and horizontal axes were scaled logarithmically and labeled in the same manner as above, with the aim of visualizing the correlations between flight time, number of maneuvers and path length covered, with regard to energy consumption by each CPP method, as shown in Figure 13, Figure 14 and Figure 15 . Furthermore, the relationship between the redundancy metric and CPP width is identified, showing behavior similar to the previous UAV’s data.

The UAV energy consumption and CPP widths were obtained from the energy consumption results logs, then a new

Table 2. Comparison of three CPP methods with regard to CPP width, simulated in a 3DR Iris UAV on three PV plants.

		Energy Consumption
PV Plant		Unioeste 1 (35,975 m${}^{\mathbf{2}}$)			Arak (25,056 m${}^{\mathbf{2}}$)			CSUSL (1344 m${}^{\mathbf{2}}$)
	CPP	BECD	GBSTC	GBWC	BECD	GBSTC	GBWC	BECD	GBSTC	GBWC
Width (m)
1		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	52%	72%	72%
2		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	30%	35%	38%
3		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	21%	24%	22%
4		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	17%	17%	15%
5		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$
6		$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	12%	12%	11%
8		100%	100%	100%	88%	98%	98%	9%	9%	8%
10		96%	100%	100%	70%	76%	71%	7%	$N/A$	7%
12		80%	92%	95%	60%	63%	59%	$N/A$	$N/A$	$N/A$
15		64%	73%	78%	45%	48%	43%	$N/A$	$N/A$	$N/A$
17		60%	68%	67%	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$	$N/A$
20		48%	55%	54%	26%	26%	25%	$N/A$	$N/A$	$N/A$

was established, in which three large columns correspond to the PV plants, where each PV plant column contains three CPPs with their respective values of consumed energy, and the rows of the table contain the CPP widths in an increasing direction from top to bottom. Here it can be observed, as in the last table, that some parameters’ values, such as the energy consumption by CPP width, are due to the size of the RoI. It can also be observed that the greater the CPP width, the lower the energy consumption, as seen in the columns of Table 2. For example, using BECD at Unioeste 1 with a width of 10 m, the percentage of energy consumed was 100%, while for a width of 20 m, the energy consumed was 48%. It is also observed that, for larger PV plants, the UAV consumed all its energy in a CPP with a small width, whereas, for small plants of the same width, the UAV did not have energy waste problems. As can be seen in the row corresponding to a CPP width of 8 m, at the Unioeste 1 and Arak plants a lot of energy was consumed, between 100%, and 88%, unlike at the CSUSL plant, where the energy consumed was very little, between 9%, and 8%. Similarly, in the previous experiments, wherein the UAV’s energy was exhausted and the CPP width did not allow the generation of the route or when the CPP width was large with regard to the RoI, the results were scored with ($N/A$). To summarize, the UAV used is suitable for the last PV plant.

On the other hand, the 3DR Iris UAV, with its design, size, autonomy, and performance is suitable when the PV plant to be inspected is small, of an approximate size of 5000 square meters or less, such as the CSUSL PV plant, at 1344 square meters. Owing to such PV plant sizes, any coverage method used in this work can be used to cover an area with a CPP width of 1 meter, while minimizing the metrics of path length covered, flight time and maneuverability, to obtain lower energy consumption, as shown in the Table 2. In addition, this UAV did not perform well at large or medium-sized plants, since, to cover them using the CPP method, the width that must be provided is greater than 8 m; therefore, though the UAV’s inspection of the PV plant had no high-resolution cameras on board, they were needed; without them, the inspection cannot be guaranteed.

The graphs in Figure 16 were constructed from Table 2 using quintic polynomial interpolations, wherein the abscissa axis is CPP width and the ordinate axis is energy consumption.

Additionally, the graph in Figure 16a shows that the BECD method had the lowest energy consumption; when the CPP width spanned the entire test range, the GBSTC and GBWC methods showed a higher energy consumption at the Unioeste 1 PV plant. The graph in Figure 16b shows that the BECD method had the lowest energy consumption when the CPP width was in the range of 8 to 10 m, whereas BECD and GBWC had similar energy consumption, leaving the GBWC method with the lowest consumption, when the CPP width was in the range of 10 to 15 m. On the other hand, the GBWC method had the lowest energy consumption when the CPP width was in the range of 15 and 20 m; this test was conducted at the Arak PV plant. Finally, the graph in Figure 16c shows that the BECD method had the lowest energy consumption when the CPP width was in the range of 1 to 3 m, and the GBWC method had the lowest consumed energy when the CPP width was in the range of 3 to 8 m. The test was performed at CSUSL’s PV plant.

Summary Tables of the results of the metrics for each of the tests with the Typhoon UAV and 3DR Iris UAV are shown. All files, and logs for the experiments are available on GitHub at [100].

3.3. Discussion

The proposed strategy allows a semi-automatic and faster solution for achieving effective results when inspecting PV plants in geometrically simple areas, with certain limitations, although some results obtained in this work are theoretical results, such as from CPP width, for example; in practice, a camera of 14 megapixels is not adequate to inspect a PV plant with a CPP width of 20 m [37,101].

The results obtained in this work indicate that the most adequate method is BECD for a specific range of CPP widths, although it also shows adequate performance for various CPP widths in some RoIs. The GBWC showed a good performance when using a CPP width greater than 7 m in some RoIs, and all showed a relationship with the area to be covered, as shown in Table 1 and Table 2.

An analysis of the metrics used in this work, such as redundancy, $R_{\%}$, which does not represent a significant factor when comparing these three CPP methods, showed that, although the BECD was the method with the highest redundancy, it was also the method with the lowest consumption of energy. On the other hand, the other metrics, such as the $L_{ct}$ metric, flight time metric, and the number of maneuvers, are directly related to energy consumption, as can be seen in Appendix B and Appendix C. It also helps to conclude that the BECD method is more suitable for widths in a range between 0 and 7 m, due to the lower number of maneuvers in these ranges, as other authors have also mentioned [102,103]. The percentage of coverage of the total area, $C_{\%}$, always showed that coverage was total

The $L_{ct}$ metric, flight time and number of maneuvers were directly related to the consumption of energy for all the experiments performed, as seen in Figure 9, Figure 10 and Figure 11 and Figure 13, Figure 14 and Figure 15, and as confirmed by other authors [89,90], in addition to helping to reinforce that the most appropriate CPP for certain ranges is the BECD.

The BECD method is the best method among the three methods tested, in a specific width range, from 0 to 7 m, for all the RoIs tested, which means that for a 10 Megapixel camera with a horizontal field of view of 7 m, the CCP method’s good images, in terms of what to inspect of a PV plant can be obtained [23].

The implemented BECD method divided the RoI into small regions, and then, over these regions, it implemented the round-trip coverage pattern, called boustrophedon [38,39]. This pattern allows spending less energy consumption with small widths, due to its low number of maneuvers compared with the other methods, as shown in Appendix B and Appendix C.

On the other hand, the GBWC method allowed lower energy consumption, according to the results of the simulations, in widths greater than a certain value, but depending on RoI size; this method also allows an approximate coverage outside the RoI, a characteristic that is very important in this type of application, and that, perhaps, is not very attractive for terrestrial robotics, from which this type of method originates [86].

A RoI with many concave points is a great challenge for performance metrics, since they increase with the distance of travel, and also increase the number of maneuvers and therefore increase energy consumption, according to these results in the following works [67,103], a more detailed analysis on these characteristics was made.

In future works, the methods (BECD, GBWC) could be implemented in UAVs with characteristics similar to those used in this work; these characteristics are shown in

Table A1. Yuneec Typhoon UAV technical specification.

Typhoon UAV
Dimensions	520 × 457 × 310 mm (20.5 × 18 × 12.2 inches)
Weight	1980 g (69.8 ounces)
Battery	5400 mAh 4S/ 14.8 V (79.9 Wh)
Camera	12.4 megapixels, 14mm/F2.8
Flight Time	up to 25 min
Flight Speed	20o m/s
Payload Capacity	10.400 g
Motor	AC YUNH520120

and

Table A5. 3D Robotics UAV 3DR Iris technical specification.

3D Robotics 3DR Iris
Dimensions	10 cm in height, 55 cm motor-to-motor
Weight	1282 g
Battery	5100 mAh 3S
Camera	N/A
Flight Time	15–20 mins
Flight Speed	11 m/s
Payload Capacity	400 g
Motor	AC 2830, 950 kV

. Additionally, with these UAVs, an inspection could be conducted in at least one PV plant. Where one can think about the implementation of an expert system that selects the coverage path between the two types of methods (BECD, GBWC), according to the CPP width size, required for a camera with a certain resolution, and focal length.

Finally, the results obtained with the BECD, and GBWC methods differ from the results obtained by other authors [34,35,36,37]. Who did not implement CPPs to solve these types of problems; they used other types of solutions that have restrictions when inspecting PV plants with UAVs. On the contrary, this work considers the CPPs, and obtained interesting results for future real implementation. The proposed CPP will increase the possibility of using inexpensive UAV systems for the inspection of PV systems on roofs of houses, and commercial buildings, and also, of the use of CPP with small widths to complete inspections at centimeter scales of the panel where the flaws can be better seen.

4. Conclusions

In this work, a method for implementing CPP in UAV for PV plant inspection was presented. The method consisted of a series of steps, one of these was the deployment of a DL-based U-net model to establish a DL service system from which to extract the limits of PV plants by extracting the boundaries of PV plants from an image. To summarize, the method was accurate, and fast without depending on the image, with low request latency and response.

This experiment focused on three novel path planning methods in the PV inspection missions in order to find the best path for covering each of the three PV power plants with less energy consumption. A GUI interface was used to order the UAV’s maneuvers in the inspection of the simulated PV plants. The results of each CPP method in the simulation were compared. The best CPPs was the BECD, for a range of CPP widths of 0 to 7 m. These path planning algorithms can be performed by any multirotor UAV that receives Mavlink commands, can carry a camera sensor, and transmit real-time video to GCS.

Performance on the CPP tasks was measured using two different types of flying robots, a Typhoon UAV and a 3DR Iris UAV. It was demonstrated that the Typhoon UAV (or one with similar characteristics) is better suited for large or medium-sized PV plants (such as those in deserts, plains, and hills); instead, the drone-like 3DR Iris UAV is more suitable for small-sized PV generation (such as on roofs and rooftops, canopies, and facades). The proposed strategy allows such comparisons to be made and enables the selection of the most suitable UAV for each type of installation.

The values obtained for the metrics collected from each of the tests show a correlation between covered path length, flight time, number of maneuvers as regards energy consumption, and ensuring that the CPPs implemented in UAVs to inspect photovoltaic plants could be similar when implemented in real plants. The results also help to predict the energy consumption of a given UAV when performing a plant inspection.