1. Introduction
Multi-UAV cooperative trajectory planning (MUCTP) is the planning of multiple safe, reasonable, and collision-free flyable trajectories based on the spatial location of UAVs, mission points, and threat obstacles in a complex planning environment [
1,
2]. Single UAV trajectory planning hardly accomplishes complex military missions due to its limited maneuverability and payload. MUCTP has the advantage of being more flexible and more efficient in performing missions compared with single UAVs; therefore, it gradually replaces single UAVs in performing complex missions. However, MUCTP also increases the difficulty of the combat system; for example, in the calculation process, the complex mountain type and radar detection area generate substantial no-fly data, and the combat system hardly parses these data in a short time; in the cooperative planning process, the heterogeneous UAVs’ own constraints and mutual constraints between multiple UAVs may lead to conflicting execution relationships between each UAV and mission point, eventually leading to mission failure [
3,
4].
In recent years, to improve the planning efficiency of MUCTP, researchers have proposed many algorithms, which have shown good performance in solving MUCTP problems, and these algorithms are mainly divided into three categories:
- (1)
Cooperative trajectory planning based on intelligent optimization algorithms
Intelligent optimization algorithms are a probability-based random search evolutionary algorithm, which guides the searching direction by constructing heuristic information, such as MUCTP fitness function [
5,
6]. In [
7], the author proposed MUCTP based on mission requirement constraints and studied a new dynamic particle swarm algorithm and comprehensive learning particle swarm algorithm, which improve the execution speed of MUCTP. Yi et al. [
8] proposed MUCTP based on the improved PF-RRT* algorithm, which uses a dichotomous method to create a new parent node at a node near an obstacle instead of updating the parent node in an existing random tree node, thereby greatly reducing the cost of each UAV trajectory.
- (2)
Cooperative trajectory planning based on reinforcement learning
Reinforcement learning (RL) is one of the paradigms and methodologies of machine learning, where each UAV obtains observation information from the current environment and takes specific behavior or strategies according to this information. At the same time, the learning system evaluates the quality of the behavior, and if the system evaluates the behavior correctly, it conducts positive rewards and strengthens the behavior training to maximize the benefits [
9]. Nie et al. [
10] adopted the Q-based learning algorithm to solve the shortest trajectory of each UAV and to calculate the cooperative range and obtained the trajectory group of each UAV by adjusting the selection strategy of each UAV. Liu et al. [
11] investigated a closed-loop model using the UAV six-degree-of-freedom nonlinear model to predict flight trajectory and adopted a trajectory mapping network based on deep learning to improve the planning calculation speed and prediction accuracy.
- (3)
Cooperative trajectory planning based on spline interpolation algorithm
The spline interpolation algorithm draws an approximately smooth curve according to key trajectory points discretely in 3D space. However, this method cannot be used alone when solving MUCTP and must be combined with other algorithms to determine the key trajectory points in advance [
12]. Qu et al. [
13] proposed UAV trajectory planning based on the RL gray wolf optimization algorithm, which determined the key trajectory points by improving the gray wolf algorithm, and then used the B-spline curve to reverse the control points to ensure that the fitted trajectory passed through all key trajectory points. Zong et al. [
14] proposed a trajectory planning framework in a 3D dynamic environment, designed a UAV trajectory predictor using the least squares method, and then used a segmented Bezier curve to represent the generated trajectory.
The differential evolution algorithm (DEA) is a population-based evolutionary algorithm that was first proposed by Storn et al. in 1995. This algorithm has stable operation, fast convergence speed, and good parallelism, and it performs well in solving global optimization problems with continuous variables [
15,
16,
17]. However, MUCTP still suffers from many difficulties, which are listed as follows:
- (1)
Compared with the two-dimensional plane, the three-dimensional space contains more information within higher complexity and experiences more difficulty in seeking key trajectory points. Although the DEA can search in 3D complex spaces, the complexity of space structures greatly reduces search efficiency. Therefore, how to simplify the space effectively according to the actual mission and environmental information is one of the challenges in optimizing the trajectory.
- (2)
The set of key trajectory points is the population individual of the DEA; the key trajectory points can not only display the optimization direction of the algorithm but also display the search efficiency of the algorithm. MUCTP requires quickly finding a better set of key trajectory points in a short time, which means that the diversity and convergence of population individuals must achieve a good balance. Therefore, how to solve the better key trajectory points quickly and accurately is another difficulty in measuring the performance of the algorithm.
The abovementioned difficulties must be addressed. In this work, we propose a cooperative multi-UAV trajectory planning based on a feasible domain space adaptive differential evolutionary algorithm (FDS-ADEA) using a centralized computational approach with fixed-wing UAV as the research object. The main contributions of this study are listed as follows:
- (1)
In terms of assigning grid point information: The current space segmentation method mostly adopts the 3D grid method, and each grid point has trajectory point information; however, no effective distinguishing information exists between grid points. Therefore, when constructing the grid point form, in addition to taking the UAV sequence, mission point sequence, and obstacle information as the main features, the corresponding trajectory cost information and coordination information are also added to avoid the blindness of the evolutionary direction in the process of algorithm search.
- (2)
In terms of constructing space structure: In consideration of the substantial 3D space information, some researchers find key trajectory points through numerous circular iterations, which seriously reduces the search efficiency. Through substantial experimental verification when the location of each UAV and mission point is determined, the planned trajectory often only appears in a limited area on both sides of the vertical section. In accordance with this idea, a 3D feasible domain space, which only calculates the key trajectory point evaluation indexes in the feasible domain as the key trajectory point set, is constructed in this work.
- (3)
In terms of constructing the adaptive algorithm: The diversity and convergence of key trajectory points mainly depends on the mutation strategy of the DEA, and determining the relevant mutation strategy through repeated testing is time consuming. Therefore, an adaptive rule based on fitness values is first constructed, and each individual can choose their own mutation strategies according to the adaptive rule in this work. Second, a reasonable coding correction method is used to delete or supplement eligible individuals. This method breaks away from the limitations and constraints of balancing exploratory and exploitative, increases the feasible domain coverage of the search space, and improves the efficiency of searching key trajectory points.
The rest of this paper is organized as follows: related work about simplifying 3D space structural models is presented in
Section 2;
Section 3 provides the 3D environment model, multi-UAV cooperative constraints, and objective function;
Section 4 describes the proposed algorithm in detail; and in
Section 5, the performance of FDS-ADEA is compared with other algorithms. Finally, a summary is presented in
Section 6.
4. Design of FDS-ADEA Algorithm
At present, research on MUCTP simulation space is mostly on 2D simulation space or 3D simulation space and order UAV trajectory planning. The reasons are as follows: (1) The amount of 3D spatial information is large, and key trajectory points are difficult to solve. Although key trajectory points can be found through numerous loop iterations, this method seriously reduces the search efficiency [
23,
24]. (2) Many algorithms adopt fixed parameter combinations and single-preference mutation strategies, which cannot satisfy the various needs in the dynamic optimization process of MUCTP.
Therefore, this section proposes a multi-UAV trajectory planning based on FDS-ADEA, which constructs a reasonable UAV feasible domain space through the location of UAV, mission points, radar scanning area, no-fly zones, and mountain terrain in space. Then, the adaptive DEA is used to construct the mutation strategy archive and the coding correction method to obtain the key trajectory point set. Finally, the flight trajectory is fitted by the B-spline function. The algorithm framework of FDS-ADEA is shown in Algorithm 2:
Algorithm 2: Framework of the FDS-ADEA |
Input: obj (The start points of the UAVs: ; The target points of the UAVs: ; The population size: ; The generations: ); The random sequence population: ; Auxiliary trajectory points: Auxi; Key trajectory points: ; Mutation rate: ; Crossover rate: . |
Output: Set of key trajectory points:; Fitted trajectory: . |
1 Construct threat barriers according to Equations (8)–(10). Choose a simulated . Determine the coordinates of each UAV and mission point in 3D space. |
2 The feasible domain space of each UAV is established to obtain the location of key trajectories; |
V = DirectionAngle (obj) |
3 The arccosine function is used to define the angle between the key trajectory points of each candidate; |
Angle = TargetArccosine (obj, Auxi, ); |
4 In accordance with Equation (11) and Equations (12)–(21), the objective function of MUCTP is established; |
fitness = DEFitness (obj); |
5 /* The FDS-ADEA algorithm is used to obtain the key flight path points */ |
Solu = FDS-ADEAoperate (obj, Mx); |
for gen = 1: obj.Gen |
Vs = MutationOperator (Mx); /* Mutation operator */ |
Cs = CrossOperator (Vs); /* Crossover operator */ |
Ft = DEFitness (Cs); /* Selection operator */ |
End |
Mt = Fitnessbest; /*Obtain the best key flight path points */ |
6 B-spline was used to fit the flight path; |
= SplineFittingFunction (obj,Mt); |
7 Obtain the parameters of each trajectory. |
4.1. Construct Feasible Domain Space for UAVs
Figure 4 is a schematic of setting the feasible region between the starting point and the mission point, where the black circle is the position of the UAV
, the black triangle is the location of the mission point
, and the cuboid represents the feasible domain space for generating key trajectory points. When the execution relationship between UAV
U and mission point
T is determined, the vertical section between the two is established and expanded into a feasible domain space. Then, the feasible domain space produces key trajectory points
.
Figure 5 shows the spatial structure of the feasible domain. The dark blue five-pointed star is the key trajectory point
, the gray dotted line is the execution relationship between the UAV and the mission point, and the hemispherical sphere is the radar scanning area.
is the feasible domain space of the UAV, and each key point is generated in
. At the same time, in accordance with the current maneuvering performance of fixed wing UAVs,
angles must be satisfied between key trajectory points. This angle range effectively prevents excessive bending of the trajectory point connection. The specific algorithm is shown in Algorithm 3.
Algorithm 3: Set the feasible flight area of the UAVs |
Input: (UAV), (Mission point), Radar (Radar position), (Number of key trajectories), , , (Range of UAV feasible domains), Pop (Population size); |
Output: Key flight trajectory points that satisfy the constraints; |
1 Construct auxiliary flight path points; |
(xf, yf, zf) = (obj.(1) + 10, obj.(1) + 5, obj.(1) + 5); |
2 Set feasible domain space parameters , , ; |
3 while (i < obj.Pop) |
4 while (i < ) |
5 comb = 1; |
6 while comb |
7 C = [Start(i, 1) + rand(1)*, Start(i, 2) + rand(1)*, Start(i, 3) + rand(1)*]; |
8 if C (1,3) > 50; /* Generate the flight assist plane */ |
9 C (1,3) = 50; |
10 end if |
11 Radar = (xi, yi, zi, ri); /*Extract the threat obstacle parameters */ |
12 Disence = sqrt(sum (bsxfun (@minus), Radar,C). ^2, 2)); /* Calculate the distance between flight path points and obstacles */ |
13 comb = any (Disence < Radius); |
14 end while |
15 The arc cosine formula is used to solve the position of the next flight path point; |
16 if () /* Determine flight path point generation Angle */ |
17 Pop(i + 1,:) = C; /* is used to judge whether the next point generated satisfies the angle */ |
18 i = i + 1; |
19 end if |
20 end while |
21 end while |
22 end |
4.2. Adaptive DEA
4.2.1. Classical Mutation Strategy
Mutation strategy is one of the key factors affecting the performance of the DEA; an excellent mutation strategy must have good exploratory and exploitation capabilities at the same time and must also maintain a good balance between the two. Currently, the mutation strategies applied to MUCTP are relatively homogeneous, for example, mutation strategies that facilitate solving more key trajectories in the search space: DE/rand/1 and mutation strategies conducive to finding the optimal key trajectory points quickly: DE/best/1. The schematic of the two mutation strategies are shown in
Figure 6a,b, respectively.
In
Figure 6a,
is the difference vector between individual
and
,
is the vector after scaling factor scaling, and
is the mutation vector obtained by summing the scaling vector with the individual
. The mutation vector
can be obtained in the same manner. As shown in
Figure 6a, the mutation strategy can effectively expand the exploratory nature and obtain more solutions that satisfy the conditions in the search space.
In
Figure 6b,
represents the experimental individual obtained by the exploitation mutation strategy; given that individuals with better fitness values are selected during each mutation process, the algorithm is prone to fall into local optimum, and the diversity of individual populations is missing.
4.2.2. Adaptive DEA Based on Fitness Values
In recent decades, researchers have designed various adaptive parameter combinations and improved mutation strategies, which show different preferences and good characteristics. However, similar to many practical applications, MUCTP is a “black box” problem. In contrast to its internal structure and characteristics, whether each UAV can effectively complete the corresponding mission must be considered more. Therefore, an adaptive DEA based on fitness values is proposed in this section. A schematic of the algorithm is shown in
Figure 7.
In ADEA: First, the entire population is randomly divided into multiple groups with the same number of individuals, and the fitness values of each parent population are calculated and preliminarily classified. Second, the archive of mutation strategies is constructed to store three mutation strategies with different characteristics: (1) mutation strategy I: “DE/rand/1” is beneficial to improve individual exploration, (2) mutation strategy II: “DE/target to best/2” is conducive to balancing the exploratory and exploitation of the population, and (3) mutation strategy III: “DE/best/1” is conducive to enhancing individual exploitation.
Table 1 shows the parameters of
F and
CR for each mutation strategy. The better offspring population is composed of better adapted parent individuals, and its main advantage is to guide the population to search for the optimal region quickly. While the primary mission of the inferior population is to maintain diversity, the better or moderate individuals can be regarded as their parent population individuals; therefore, inferior individuals have access to a great deal of exploitation and exploratory information. The adaptive mutation strategy I, II, and III selection methods are shown in Algorithm 4.
Algorithm 4: Adaptive mutation strategy selection |
Input: (Number of individuals); Mutation strategies I, II, III; |
Output: A population individual conforming to a fitness value; |
1 Determine the population size and randomly divide the population individually; |
2 The fitness value of each parent is calculated, and the individual is divided; |
3 for i = 1: |
4 if is the optimal fitness value, it is the better individual. |
5 Mutation strategy III can be selected to produce offspring ; |
6 else if is the middle fitness value, it is the moderate individual; |
7 Mutation strategy III or II can be chosen to produce offspring ; |
8 else if is the poor fitness value, it is the inferior individual; |
9 Mutation strategies III, II, or I can be selected to produce offspring ; |
10 end if |
11 end for |
12 end |
The key trajectory points of MUCTP candidates are all integer discrete sequences. If the sequence values are rounded to the mutation result, then the mutated solution obtains invalid values. If the invalid value continues to participate in the evolutionary calculation, the algorithm stalls. Therefore, invalid discrete values must be corrected, and the coding correction method can map the invalid sequence directly to obtain a new discrete sequence. For alternative key trajectory points to satisfy planning requirements, the coding correction method must satisfy the following conditions:
Rule 1: Mapping uniqueness: During the correction process, the mapped discrete sequences must be unique. As shown in
Figure 8, the invalid sequence individuals are 0, 9, 5. When differential individual 1 is selected as the gene mapping of a newly added individual, the position of the sequence in which the original differential individual 1 is located must be deleted, and the other differential values are mapped in turn. This method guarantees that the mapping is unique.
Rule 2: Mapping validity: The mapped invalid discrete sequences no longer participate in the later variation strategy calculation. Although the mapped sequence is overwritten in Rule 1, the variation strategy information is still stored in the individual. Therefore, the invalid mapping matrix must be removed. At the same time, the mapped individual sequence participates in the later mutation strategy calculation, and this method can improve the performability of the algorithm.
4.3. Cubic Spline Curves
The FDS-ADEA algorithm can quickly solve trajectories that satisfy the cooperating conditions, but these trajectories consist of a series of key trajectory point segments, which are not suitable as actual trajectories for UAVs to fly. Therefore, the currently obtained trajectory connection must be smoothed. Yakimenko, O [
25] proposed a direct method for rapid prototyping of near-optimal vehicle trajectories. Kaminer, I et al. [
26] proposed a cooperative control algorithm for small UAVs, whose main features include trajectory generation for multiple UAVs, which takes into account their aerodynamic characteristics and ensures deconfliction.
In this work, the three B-splines are used to smooth the trajectory, and all the points in the trajectory are used as the control points of the B-spline curve. By iteratively curve fitting in 3D space, smooth flyable trajectories can be obtained directly.
When
n = 3, the basis function of the B-spline can be expressed as:
The cubic B-spline matrix is as follows:
From the matrix Equation (26), the discrete starting point, auxiliary trajectory point, key trajectory points and mission points of each UAV can be used as the control points of the spline curve to approximate the smooth trajectories of the fitted UAVs.
4.4. Flowchart Based on FDS-ADEA Algorithm
Figure 9 shows the flowchart based on FDS-ADEA algorithm, including the construction of the UAV’s own constraints and co-constraints, the construction of the MUCTP objective function, and the principle of the adaptive differential evolution algorithm.
6. Conclusions
In this work, an adaptive DEA based on feasible domain space is proposed to solve the difficulties of MUCTP. In terms of spatial structure, a three-dimensional feasible domain is constructed, and the influence of feasible domain parameters on key trajectory points is analyzed in consideration of reducing the complexity search spaces. In terms of node information, environmental information, cooperative information, and mission information are added, and the recognition degree of individuals in three-dimensional spaces is improved to avoid the blindness of algorithm search. In terms of algorithms, the adaptive method and coding correction method are established to share information, such as fitness values, generate key trajectory points, and improve the algorithm search efficiency, which aims to balance individual exploration and exploitation. Finally, a cubic B-spline curve is combined to smooth the flight path to ensure the flying ability of UAVs. Experimental results show that the algorithm has faster convergence speed, stronger coordination ability, and more reasonable trajectory group when dealing with multi-aircraft cooperative trajectory planning.
Future research will focus on considering the cooperative multi-UAV trajectory in dynamic environments. Although the algorithm in this study can effectively deal with the problem of MUCTP in a static environment, in real combat scenarios, the real-time changing environment is often more complex and closer to reality. If the method of trajectory planning in static environment is still followed, the global trajectory will mistakenly guide UAVs into the radar scanning area or fail to reach the specified position, resulting in mission failure. Therefore, MUCTP in dynamic environments is more important than MUCTP in static environments.