5.1. Evaluation Framework
The scope of this paper is limited to detecting corrosion in indoor steel pipe networks using a system of multi-MAVs. We evaluated the performance of our system using a rigorous empirical methodology based on simulation experiments. We employed the Crazyswarm package [
48], which is a comprehensive quadrotor autonomous research testbed that integrates hardware and software components. It enables seamless transition from simulation to real-world deployment [
49]. We implemented all algorithms in Python 3.9 and ran all simulations on MAC studio M1 Ultra with 128 GB RAM. Our system adopted Crazyflie as the UAV model and Crazyswarm as the simulation platform; therefore, we selected an appropriate sensor for the mission and UAV and tuned the parameters to match realistic conditions. Following previous literature [
50], we used Crazyflie and an ultrasonic sensor (LV-MaxSonar-EZ2), which can detect objects within a range of 0–254 inches (6.75 m).
Figure 8 illustrates the Crazyflie UAV, and
Figure 9 depicts the ultrasonic sensor. We also considered the specifications of the Crazyflie when estimating the energy consumption of the UAVs [
51,
52].
Table 2 summarizes the values of the parameters related to UAV energy.
To evaluate the inspection mission, a template of the fire sprinkler system RCP using the Edrawmax tool was used as an input map to mimic a realistic scenario as much as possible [
53]. The input map is shown in
Figure 10 and is handled as an occupancy matrix after preprocessing (as per
Figure 11) it to form a 500 × 500 grid with cell size 0.5 m × 0.5 m.
The network locations were used as inspection points, and the defect locations were randomized within the inspection points to ensure representative samples in all the experiments. Furthermore, the defect locations were generated randomly using multivariate normal distributions to simulate hotspots within the specified radius where defects clustered together and were more likely to be found.
The controlled parameters in the simulation were as follows:
The number of UAVs varied between 2, 4, 6, 8, 10, 12, 14, and 16.
The number of defects (including hotspots and defect concentration within a single hotspot). The values of the different severity levels of the defects are shown in
Table 3.
To evaluate the performance of our proposed approach, we adapted the performance metrics from recent studies [
54], such as total travel distance (cost/fitness value), maximum tour length, running time, mean detection time, and average consumed energy. We developed 120 scenarios by varying the number of heuristics (5), the number of UAVs (8), and the severity levels of defects (3). Each scenario was run 30 times to reduce the variability in the performance measures [
54].
We used four well-established benchmark algorithms: ACO, PSO, OTA, and the random algorithm. The ACO algorithm was based on Dorigo’s original paper [
55] and followed the parameters listed in
Table 4. The termination conditions for ACO were either no improvement in 30 iterations or reaching 1000 iterations [
54].
For the second benchmark, we implemented a discrete version of PSO following the approach in [
56] that represents the solutions (paths) as vectors. The cost is the total distance of the path of all UAVs (the vectors). We computed this value using a distance matrix saved for all locations. The number of particles is equal to the number of UAVs, according to literature studies [
57]. However, in each particle, the local solution is composed of sub-vectors, each corresponding to one UAV. The sub-vectors do not have conflicts between them, and each network location is visited once in one and only one sub-vector. This ensures that if a location has a defect, it will be detected by only one UAV. We used the same suggested values for the algorithm parameters as in [
56]. Furthermore, we followed the same termination conditions as the ACO. The PSO algorithm parameters are shown in
Table 5.
The third algorithm implements an OTA strategy [
58], instructing an agent to search for its nearest unexplored cell in the zone. The final algorithm implements a random choosing strategy [
35], instructing an agent to search for its nearest unexplored cell in the zone. The OTA and random algorithms were utilized as baseline performance metrics.
Table 6 shows the particular parameters of the booby system. The value of the threshold X of the primary UAV algorithm was chosen empirically. Furthermore, we used k-means as the clustering algorithm. We found the optimal number of the cluster after plotting all possible k values against their internal evaluation indicators and choosing the best one [
59].
5.2. Results and Discussion
We conducted each experiment at least 30 times and plotted the results using a linear scale for the number of UAVs on the x-axis and a logarithmic base-2 scale for the performance metrics on the y-axis. The performance metrics included mean detection time, total traveled distance, maximum tour length, running time, and average energy used. We calculated these metrics for all benchmarks except for PSO and mean detection time. PSO does not support time inference because it defines the UAV paths as vectors during the initialization phase. Therefore, we cannot determine when a defect was found. All UAVs had a uniform battery energy level of δm and consumed energy at a constant rate for all movements.
5.2.1. Mean Detection Time
The mean detection time of a defect is an important metric to evaluate the performance of different algorithms for UAV-based inspection. It measures how long it takes for any UAV to detect a defect from the start of the simulation. We computed this value for each defect in each scenario and averaged it over multiple runs of each experiment.
Figure 12 shows the results for our booby algorithm and three other benchmarks. The booby algorithm consistently outperformed the other algorithms in all settings, reducing the mean detection time by at least 13% compared to the random algorithm, which had the shortest running time among the benchmarks. This indicates that our approach can efficiently detect defects regardless of their number and distribution complexity.
According to a thorough analysis, the booby algorithm outperforms other algorithms for several reasons. First, it clusters the map into a well-chosen number of zones based on the locations of defects. Second, it assigns UAVs to the zones in a way that considers their size and priority. Specifically, it allocates more UAVs to larger zones sooner than smaller zones, as larger zones may have more defects. Third, it inspects the zones efficiently and adapts its behavior when a defect is found. For instance, the secondary UAV prioritizes joining a zone that has more defects detected by another UAV. These reasons enable the booby system to discover defects faster than other algorithms. As the number of UAVs increased, the booby, OTA, and random algorithms demonstrated a decrease in mean detection time. In contrast, the ACO performance is much worse than random and OTA. The ACO’s mean detection time increases with the number of UAVs due to its solution construction method. Each ant updates its tour with an amount inversely proportional to the distance of the tour. As a result, more ants follow shorter paths with more pheromones and neglect remote areas, which delays most defect detections. Furthermore, the ACO algorithm iterates through many paths before finding the solution, which negatively affects the mean detection time.
5.2.2. Total Traveled Distance (Cost)
Figure 13 shows that the booby algorithm produced tours with lower costs than the benchmarks, especially when more than two UAVs were involved. The random algorithm performed poorly as expected due to its naive nature. PSO also had low performance in this problem because it requires continuous solution values for its velocity update, but TSP is a discrete problem [
60,
61,
62,
63]. Clerc’s study in 2004 was one of the first to propose discrete PSO and it is still widely cited in the recent literature [
64]. However, Clerc himself reported that discrete PSO was not as efficient as other algorithms [
65]. Similarly, recent studies have indicated that PSO is more suitable for continuous optimization problems [
63].
The metric cost analysis revealed that the booby algorithm had similar performance to OTA and ACO when U = 2. In these cases, there were fewer UAVs than clusters. This resulted in a limited number of primary UAVs initially and increased travel costs for the primary UAV later as it had to visit many available zones. Likewise, each zone was assigned a certain number of UAVs. This affected the total cost of each UAV because it had to cover most of the zone by itself.
5.2.3. Running Time
We measured the running time of each algorithm for every scenario by recording the time when the execution started and ended.
Figure 14 shows the results, which indicate that booby outperformed all benchmarks in terms of speed. It solved all scenario instances faster than the other algorithms, with an average speedup of at least three times over random in simple settings (
Figure 14a), three times over random in average settings (
Figure 14b), and 1.2 times over random in advanced settings (
Figure 14c). Moreover, booby’s running time did not increase exponentially as the number of UAVs increased, unlike PSO and ACO. This can be attributed to its efficient use of data structures such as k-d tree and dictionaries, as well as its ability to find a “good” solution within one iteration, unlike metaheuristic algorithms that require multiple iterations to converge.
5.2.4. Maximum Tour Length
The maximum tour length is an important metric for evaluating the performance of UAV inspection algorithms. It measures the longest distance any UAV travels after completing its assigned mission. To compare our approach with OTA and ACO, we calculated the average maximum tour length for all scenarios based on each algorithm’s results.
Figure 15 shows that our approach consistently achieved the lowest average maximum tour length while maintaining a low running time and minimizing the total traveled distance. To further illustrate this advantage, we also compared the maximum tour length of each scenario for our approach and OTA, which had a tradeoff between this metric and the running time.
Table 7 displays the percentage difference between these two algorithms, as well as their time ratio. The data indicates that our approach not only reduced the maximum tour length by more than 10% in most scenarios (highlighted in the table), but also enhanced the runtime in all cases. The main reason for this improvement is that our approach assigns distinct roles to UAVs and uses efficient data structures to select candidate zones based on proximity and availability. Finally, our approach requests a temporary UAV to assist in inspecting a zone only if a certain percentage of the zone remains uninspected after a certain amount of time has elapsed. This way, our approach ensures that all zones are inspected thoroughly and quickly by utilizing UAVs optimally.
5.2.5. UAV’s Average Consumed Energy
Initially, the energy levels of all UAVs are identical. As each UAV navigates to a new position, it consumes some energy for this movement. The average energy consumption is the ratio of the total energy consumption to the number of UAVs at the end of the execution.
Figure 16 shows how the average energy levels vary with different numbers of UAVs. There is a negative correlation between these two variables: more UAVs mean less average energy consumption. This trend is evident in the PSO and random results, but not in other benchmarks (OTA and ACO). A possible explanation is that adding more UAVs in PSO and random algorithms reduces some long tours taken by individual UAVs, which require more energy. However, such a reduction is not observed in other algorithms because they already favor shorter tours over longer ones regardless of the number of UAVs. This result confirms that PSO and random algorithms have lower average maximum tour lengths when more UAVs are added. Moreover, our approach has comparable results to OTA and ACO when there are few UAVs (U = 2, 4, and 6). However, our approach is more scalable because it minimizes the average energy consumption as more UAVs are added.