An Evolutionary Game-Theoretic Approach to Unmanned Aerial Vehicle Network Target Assignment in Three-Dimensional Scenarios

Abstract

Target assignment has been a hot topic of research in the academic and industrial communities for swarms of multiple unmanned aerial vehicle (multi-UAVs). Traditional methods mainly focus on cooperative target assignment in planes, and they ignore three-dimensional scenarios for the multi-UAV network target assignment problem. This paper proposes a method for target assignment in three-dimensional scenarios based on evolutionary game theory to achieve cooperative targeting for multi-UAVs, significantly improving operational efficiency and achieving maximum utility. Firstly, we construct an evolutionary game model including game participants, a tactical strategy space, a payoff matrix, and a strategy selection probability space. Then, a multi-level information fusion algorithm is designed to evaluate the overall attack effectiveness of multi-UAVs against multiple targets. The replicator equation is leveraged to obtain the evolutionarily stable strategy (ESS) and dynamically update the optimal strategy. Finally, a typical scenario analysis and an effectiveness experiment are carried out on the RflySim platform to analyze the calculation process and verify the effectiveness of the proposed method. The results show that the proposed method can effectively provide a target assignment solution for multi-UAVs.

1. Introduction

With the innovation and rapid development of technology, unmanned aerial vehicles (UAVs) are widely used in search and rescue, road patrol, target surveillance, and other scenarios by virtue of their advantages of low cost, high reliability, and high scalability. Due to the complexity and rapid expansion of the UAV operating environment, it has become a hot field of scientific research to rationally assign the best target to the most suitable UAV at the least cost by virtue of autonomous perception and collaborative decision-making, and it is of great significance to effectively monitor and track multiple moving air targets in complex environments as well as gaming [1,2]. However, with the increase in the number of UAVs and the urgency of real-time surveillance, low-damage and high-efficiency many-to-many target assignment has become a crucial challenge [3,4,5].

In recent years, researchers have proposed various methods to address the issue of target assignment, including mathematical programming methods [6,7,8], heuristic methods [9,10], and optimization algorithms [11]. Orhan et al. proposed a dynamic target allocation framework based on artificial neural networks that optimizes the target assignment plan by simultaneously considering multiple objectives and constraints [12]. Zhu et al. proposed an intelligent decision-making framework based on reinforcement learning for multi-target assignment problems, which can effectively learn, adapt to changing environments, and optimize the overall system performance through decision-making [13]. Zou et al. derived a decision tree-based target assignment algorithm that considers various factors such as distance, angle, and speed to effectively allocate targets to multiple space vehicles and make accurate allocation decisions [14]. Zhen et al. derived a cooperative target allocation algorithm based on an improved Contract Network Protocol (ICNP) for heterogeneous UAV swarms. It considers UAV capabilities, target characteristics, and communication constraints to make real-time and optimal allocation decisions [15]. Shalumov et al. introduced a dynamic programming-based approach for weapon–target allocation in multi-agent target-missile-defender engagement that considers missile flight time, defender capability, and target priority to make real-time and optimal allocation decisions, which was shown to be effective through experimental results [16]. Duan et al. introduced a target allocation algorithm inspired by wolf behaviors for unmanned aerial systems (UASs), which considers target characteristics, UAS capabilities, and communication constraints. They can make real-time and coordinated allocation decisions [17]. Yeduri et al. proposed a novel method of energy and throughput management in a delay-constrained small-world unmanned aerial vehicle (UAV)–IoT network [18]. Bose et al. propose a novel method that computes the optimum height at which UAVs should hover, resulting in a maximum coverage radius with a sufficiently small outage probability [19]. Xia et al. proposed a multi-UAV cooperative target tracking system and designed a deep reinforcement learning (DRL)-based algorithm for intelligent flight action decisions of UAVs to track a moving air target using past and current position information [20]. Zhou et al. proposed a UAV swarm-based cooperative tracking architecture to systematically improve the UAV tracking performance [21]. Despite recent progress, existing air combat target assignment methods still have some limitations, such as over-reliance on perfect information, lack of adaptability to dynamic environments, and single objective optimization.

Traditional optimization methods have limitations in producing optimal target assignment plans. In existing studies, when multi-UAV networks target a single target, they are often treated as independent and unrelated individuals, and the target’s survival probability is multiplied. However, obtaining accurate probability values in actual situations is often significantly challenging. The adversary process between both sides is essentially a game, and the other side’s behavior can significantly affect the effectiveness of the multi-UAV network target assignment plan. Therefore, it is necessary to estimate the target assignment plan that the other side may adopt and comprehensively judge its impact to optimize and obtain the optimal target assignment plan for our side.

Evolutionary game theory, an outgrowth of traditional game theory, incorporates principles of biological evolution [22]. It combines the economic concept of “equilibrium” with the concept of “adaptation” to depict how groups adapt to their environment through learning, imitation, and trial-and-error under conditions of incomplete rationality, asymmetric information, and biases in expectations and environment [23]. Furthermore, evolutionary game theory prioritizes cooperation over destructive competition, making it a means of resource assignment that enables the maximization of overall effectiveness. This process ultimately leads to an evolutionarily stable state, providing a particle framework for studying interactions among multiple intelligent agents.

Numerous scholars have utilized evolutionary game theory to study cooperation problems, yielding significant results that have been widely applied to practical issues [24,25,26]. Given that the concept of stable cooperation and target assignment in a multi-UAV system, under limited information and resources, aligns with the principles of evolutionary game theory, a comprehensive analysis of the stable long-term trends of both parties engaged in the game becomes possible by leveraging these principles. Both sides in the target assignment problem face a choice between cooperation and competition. They must consider the balance between their own interests and the overall benefits. This decision-making process can be analyzed using the concept of evolutionary games, including the choice of game strategies, the dynamics of evolution, and the final equilibrium state. Evolutionary game theory can explore the effects of different decision-making strategies and evolutionary dynamics on the advantages of both sides, which can help to deeply understand the strategic decision-making and competitive mechanisms in multi-UAV network target assignments and provide effective solutions to the problem of optimal target assignment. The proposed method establishes a multi-UAV network target assignment evolutionary game model, with both sides’ UAVs in this model regarded as game participants and their respective target assignment plans serving as the game strategies. The payoff function of the game is constructed based on the total attack effectiveness and ineffectiveness achieved by both sides in the target assignment plan, with the confidence level used to measure the attack effectiveness of each UAV against its target [27]. Additionally, a multi-level information fusion method, based on evidence theory, is used to calculate the total attack effectiveness of multi-UAVs against multiple targets [28]. Based on the multi-UAV network target assignment evolutionary game model, a replicator equation is constructed to determine the evolutionarily stable strategy (ESS) and obtain the optimal assignment plan. Finally, the effectiveness of the proposed method is verified through typical case studies and experiments.

The main contributions of this paper can be summarized as follows: (i) evolutionary game theory is utilized to solve the multi-UAV network target assignment problem; (ii) a multi-level information fusion method based on evidence theory is designed to calculate the total attack effectiveness of multi-UAVs against multiple targets; and (iii) an experiment in different scenarios is conducted to demonstrate the effectiveness of the proposed method.

2. Problem Formulation and Model Construction

2.1. Problem Formulation

The problem studied in this paper can be described as follows: Both our multi-UAV and target multi-UAV consist of multiple UAVs of the same type and each of our UAVs evaluates the expected benefits based on environmental information, its own capabilities, and the threat of targets. Using this information, each of our UAVs will be able to determine the optimal target assignment plan for the target multi-UAVs, thus gaining an advantage in the game.

The state of each UAV is represented as $X=(x,y,z,v,\theta ,\phi )$, where $(x,y,z)$ represents the UAV’s three-dimensional coordinates, $v$ represents the UAV’s speed, $\theta$ represents the UAV’s pitch angle, and $\phi$ represents the UAV’s heading angle. Let $R$ be the set of multi-UAVs and $B$ be the set of targets. The multi-UAV assignment plan is denoted by ${\omega }_R={\left(a_{i,j}\right)}_{i\in R,j\in B},a_{i,j}\in \left\{\mathrm{0,1}\right\}$, where $a_{i,j}=1$ represents that the UAV $i$ selects the target $j$. Similarly, the targets’ assignment plan is denoted by ${\omega }_B={\left(b_{j,i}\right)}_{j\in R,i\in B},b_{j,i}\in \left\{\mathrm{0,1}\right\}$, where $b_{j,i}=1$ represents that target $j$ selects the UAV $i$. The relative situation diagram between the UAV $i$ and target $j$ is shown in Figure 1, where the red UAV represents our UAV and the blue UAV represents the target UAV.

Assuming that each UAV flies toward its target based on the target assignment results, and if the target is within attacking range and the situation is in favor, the UAV will attack its target. Let the number of attacks by our side’s UAV be denoted by $w{p}_R$. Similarly, let the number of attacks by the target side’s UAV be denoted by $w{p}_B$. Figure 2 provides a comprehensive overview of the multi-UAV cooperative target assignment problem, illustrating the entire flow of the problem.

2.2. Model Construction

We can model the problem of multi-UAV network target assignment as an evolutionary game model, denoted by $G=\left\{N,S,P,I\right\}$ with the following specifications:

Let $N=\left\{R,B\right\}$ be the participating entities of the two sides of the game: $R$ represents our side’s multi-UAV and $B$ represents the target side’s.

Let $S=\left\{S_R,S_B\right\}$ be the strategy space in the evolutionary game model, where $S_R$ is the set of our side’s pure strategies, and $S_B$ is the set of pure strategies for the target side. Specifically, the $I$-th pure strategy $s_R^I$ in $S_R$ corresponds to the $I$-th target assignment plan ${\omega }_R^I$ for our side and the $J$-th pure strategy $s_B^J$ in $S_B$ corresponds to the $J$-th target assignment plan ${\omega }_B^J$ for the target side. Let $m$ denote the number of our side’s multi-UAVs and let $n$ denote the number of the target side’s multi-UAVs. Let ${\tau }_R=n^m$ denote the number of pure strategies for our side and let ${\tau }_B=m^n$ denote the number of pure strategies for the target side. Therefore, we have that $S_R=\left\{s_R^1,\cdots ,s_R^I,\cdots ,s_R^{{\tau }_B}\right\}$ and $S_B=\left\{s_B^1,\cdots ,s_B^J,\cdots ,s_B^{{\tau }_R}\right\}$.

Let $P=\left\{P_R,P_B\right\}$ be the payoff matrix in the evolutionary game model, where $P_R={\left\{u_R\left(s_R^I,s_B^J\right)\right\}}_{s_R^I\in S_R,s_B^J\in S_B}$ is the payoff matrix for our side and $P_B={\left\{u_B\left(s_R^I,s_B^J\right)\right\}}_{s_R^I\in S_R,s_B^J\in S_B}$ is the payoff matrix for the target side. Specifically, $u_R\left(s_R^I,s_B^J\right)$ is the payoff value obtained by our side when we choose the pure strategy $s_R^I$ and the target side chooses the pure strategy $s_B^J$. Similarly, $u_B\left(s_R^I,s_B^J\right)$ is the payoff value obtained by the target side when they choose the pure strategy $s_R^I$ and our side chooses the pure strategy $s_B^J$.

Let $I=\left\{I_R,I_B\right\}$ be the probability space for both sides to choose their respective strategies in the evolutionary game model. Let $I_R={\left\{p\left(s_R^I\right)\right\}}_{s_R^I\in S_R}$ denote the probability space for our side, and let $I_B={\left\{q\left(s_B^J\right)\right\}}_{s_B^J\in S_B}$ denote the probability space for the target side. Specifically, $p\left(s_R^I\right)$ denotes the probability of our side choosing a pure strategy $s_R^I$, and $p\left(s_B^J\right)$ denotes the probability of the target side choosing a pure strategy $s_B^J$.

3. Construction of the Payoff Matrix

3.1. Establishment of a Situational Assessment Function

Situational assessment is an important basis for UAVs to determine their target assignment. This evaluation is primarily determined by factors such as the perspective of both sides, distance, approach speed, and relative altitude. For example, the UAV has various advantage functions, which are as follows:

(I)

Advantage function of perspective.

The advantage function of perspective is primarily related to the attack angle of the UAV. During the process of searching, tracking, and attacking targets, it is necessary to control the target within a certain angle range for the UAV to meet the requirements of detection accuracy and precision. The expression for the advantage function of perspective is given by:

$$S_{\alpha }=\frac{q_B-q_R}{2\pi }$$

(1)

where $S_{\alpha }$ represents the advantage of the perspective of the UAV against the target, $q_B$ denotes the angle between the velocity vector of the target and the tracking line of the target, and $q_R$ denotes the angle between the velocity vector of the UAV and the tracking line of the UAV, as shown in Figure 3, where the red UAV represents our UAV and the blue UAV represents the target.

(II)

Advantage function of approach speed.

The advantage function of approach speed is primarily related to the relative approach speed of the UAV with respect to the target, as shown in Figure 4. It is compressed using the arctangent function and increased by 0.5 to maintain its positive value. The expression for the advantage function of approach speed is given by:

$$S_{V_C}=0.5-\frac{1}{\pi }\mathit{arctan}\left(\frac{V_C}{100}\right)$$

(2)

where $S_{V_C}$ represents the advantage of the approach speed of the UAV against the target, and $V_c$ denotes the relative approach speed of the UAV with respect to the target, measured in units of meters per second (m/s).

(III)

Advantage function of altitude.

The advantage function of altitude is primarily related to the difference in altitude between the UAV and the target, as shown in Figure 5. It is controlled by a threshold altitude difference and a proportional coefficient that controls the rate of change of the advantage function when the UAV deviates from the optimal altitude value. The expression for the advantage function of altitude is given by:

$$S_h=e^{-{\left(\frac{h-h_x}{\sigma h_x}\right)}^2}$$

(3)

where $S_h$ represents the advantage of the altitude of the UAV against the target, $h$ denotes the difference in altitude between the UAV and the target, $h_x$ denotes the threshold altitude difference representing the optimal altitude difference for the UAV, and $\sigma$ denotes the proportional coefficient used to control the rate of change of the advantage function of altitude.

(IV)

Advantage function of attack.

The advantage function of attack comprises two parts: the advantage function of attack distance and angle, as shown in Figure 6 and Figure 7. The advantage function of attack distance represents the degree of advantage of the target side’s UAV being within the attack range of our side’s UAV and is given by:

$$S_r=e^{-{\left(\frac{r-R_0}{{\sigma }_r}\right)}^2}$$

(4)

where $S_r$ represents the advantage of the attack distance of our side’s UAV against the target side’s UAV, $r$ denotes the distance between UAVs on both sides, $R_{min}$ and $R_{max}$ denote the minimum and maximum attack distances, respectively, and ${\sigma }_r$ denotes the attack range, where ${\sigma }_r=\left(R_{max}+R_{min}\right)/2$. and $R_0=2\left(R_{max}-R_{min}\right)$. The value of ${\sigma }_r$ controls the rate of change of the attack advantage function when it deviates from the optimum. When the relative distance between UAVs on both sides is twice the maximum and minimum attack distances, the attack distance advantage function reaches its maximum value of 1, indicating that the target UAV will be within the attack range for a longer period.

The expression for the attack angle advantage function is given by:

$$S_c=e^{-{\left(\frac{\beta }{{\beta }_0}\right)}^2}$$

(5)

where $S_c$ represents the advantage of the attack angle of our side’s UAV against the target side’s UAV, $\beta$ denotes the angle between the tracking line of our side’s UAV’s tracking line and our side’s UAV’s velocity vector, and ${\beta }_0$ denotes the maximum off-axis angle. When $\beta =0$, the attack angle advantage function reaches its maximum value of 1, indicating that the velocity vector of our side’s UAV points toward the target side’s UAV. A large value of ${\beta }_0$ indicates better attack performance. When the velocity vector of our side’s UAV does not point toward the target side UAV, the rate of change of the advantage function of the attack angle decreases as the value of ${\beta }_0$ increases.

The total attack advantage function is related to both the attack angle condition and the attack distance condition. Therefore, it can be expressed as the product of these two advantage values, as shown in the following expression:

$$S_A=S_C\times S_r$$

(6)

(V)

Situation assessment function.

The situation assessment function is used to evaluate the effectiveness of UAV attacks. It is obtained by weighting the angle, closing speed, altitude, and attack advantage function value. Experts typically determine the weighting values based on the magnitude and importance of each advantage value, which may change according to the UAV’s flight performance and changes in the posture situation. The expression for the constructed situation assessment function is shown in Equation (7):

$$S=2\times S_{\alpha }\times S_{V_C}+S_h+2\times S_A$$

(7)

3.2. Multi-Level Information Fusion Method Based on Evidence Theory

A payoff matrix is constructed using a multi-level fusion method based on evidence theory, utilizing the situation assessment function developed in Section 3.1, as shown in Figure 8.

The effectiveness matrices for UAVs on both sides denoted as $T_R$ and $T_B$, respectively, are obtained by calculating the relative attack effectiveness of UAVs on both sides based on the situation assessment function. These matrices are defined in Equations (8) and (9):

$$T_R=\left[\begin{array}{ccc}{t}_{R11}& \cdots & t_{R1n}\\ \cdots & t_{Rij}& \cdots \\ t_{Rm1}& \cdots & t_{Rmn}\end{array}\right]$$

(8)

$$T_B=\left[\begin{array}{ccc}{t}_{B11}& \cdots & t_{B1m}\\ \cdots & t_{Bji}& \cdots \\ t_{Bn1}& \cdots & t_{Bnm}\end{array}\right]$$

(9)

Here, $t_{Rij}$ represents the attack effectiveness of the $i$-th UAV against the $j$-th target, and $t_{Bji}$ represents the attack effectiveness of the $j$-th target against the $i$-th UAV.

The first fusion level in the target assignment plan ${\omega }_R^I$ involves using evidence theory to fuse the attack effectiveness of UAV targeting the same target. Let $c_j$ denote the number of UAVs confronting target $j$, with ${\sum }_{j=1}^n{c}_j=m$, where $m$ denotes the total number of UAVs. In ${\omega }_R^I$, the joint attack effectiveness $E_{Rj}$ of $c_j$ UAVs attacking target $j$ is calculated by Equation (10):

$$E_{Rj}\left({\omega }_R^I\right)=t_{R{k}_{1}j}\oplus t_{R{k}_{2}j}\oplus \cdots \oplus t_{R{k_{c_j}}^j}$$

(10)

Here, $\oplus$ denotes the evidence theory fusion operator, and $k_1,k_2,\cdots ,k_{c_j}$ represent the UAVs in the assignment ${\omega }_R^I$ targeting the target $j$. $t_{R{k}_{1}j},t_{R{k}_{2}j},\cdots ,t_{R{k}_{c_j}j}$ represent the effectiveness of our side’s UAVs targeting $j$.

Using the theory of evidence, all the evidence $E_{Rj}\left({\omega }_R^I\right)$, where $j=\mathrm{1,2},\cdots ,n$, are fused at the second-level fusion, resulting in the overall attack effectiveness $E_R\left({\omega }_R^I\right)$ of $m$ UAVs against $n$ targets, as shown in Equation (11):

$$E_R\left({\omega }_R^I\right)=E_{R1}\left({\omega }_R^I\right)\oplus E_{R2}\left({\omega }_R^I\right)\oplus \cdots \oplus E_{Rn}\left({\omega }_R^I\right)$$

(11)

Similarly, we can use the above theory-based multi-level information fusion algorithm to calculate $E_B\left({\omega }_B^I\right)$.

To calculate the payoff value in the multi-UAV network target assignment evolutionary game model G, we first obtain the total effectiveness and ineffectiveness of both sides under the target assignment plans ${\omega }_R^I$ and ${\omega }_B^J$, respectively. Denote these values as $E_R\left({\omega }_R^I\right)$, $N_R\left({\omega }_R^I\right)$, $E_B\left({\omega }_B^J\right)$, and $N_B\left({\omega }_B^J\right)$. Using these values, we can calculate the payoff value for both the UAVs and targets under a given strategy combination $\left(s_R^I,s_B^J\right)$. The payoff value for the UAVs is the fusion of $E_R\left({\omega }_R^I\right)$ with $N_B\left({\omega }_B^J\right)$ divided by the fusion of $E_B\left({\omega }_B^J\right)$ with $N_R\left({\omega }_R^I\right)$, as shown in Equation (12). Similarly, the payoff value for targets is the fusion of $E_B\left({\omega }_B^J\right)$ with $N_R\left({\omega }_R^I\right)$ divided by the fusion of $E_R\left({\omega }_R^I\right)$ with $N_B\left({\omega }_B^J\right)$, as shown in Equation (13). Here, $N_B\left({\omega }_B^J\right)$ is 1 minus $E_B\left({\omega }_B^J\right)$, and $N_R\left({\omega }_R^I\right)$ is 1 minus $E_R\left({\omega }_R^I\right)$.

$$u_R\left(s_R^I,s_B^J\right)=\frac{E_R\left({\omega }_R^I\right)\oplus N_B\left({\omega }_B^J\right)}{E_B\left({\omega }_B^J\right)\oplus N_R\left({\omega }_R^I\right)}$$

(12)

$$u_B\left(s_R^I,s_B^J\right)=\frac{E_B\left({\omega }_B^J\right)\oplus N_R\left({\omega }_R^I\right)}{E_R\left({\omega }_R^I\right)\oplus N_B\left({\omega }_B^J\right)}$$

(13)

Iterating through all possible strategy combinations in the multi-UAV network target assignment evolutionary game model $G$, we can calculate the corresponding payoff values for UAVs on both sides. This allows us to generate the payoff matrix for UAVs on both sides, denoted as $P_R$ and $P_B$, respectively. The payoff matrix is a ${\tau }_R$ by ${\tau }_B$ matrix, where ${\tau }_R$ is the number of UAV strategies, and ${\tau }_B$ is the number of target strategies. The elements of these matrices are the calculated payoff values for each strategy combination, as shown in Equations (14) and (15), respectively.

$$P_R=\left[\begin{array}{ccccc}{u}_R\left(s_R^1,s_B^1\right)& \cdots & u_R\left(s_R^1,s_B^J\right)& \cdots & u_R\left(s_R^1,s_B^{{\tau }_B}\right)\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ u_R\left(s_R^I,s_B^1\right)& \cdots & u_R\left(s_R^I,s_B^J\right)& \cdots & u_R\left(s_R^I,s_B^{{\tau }_B}\right)\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ u_R\left(s_R^{{\tau }_R},s_B^1\right)& \cdots & u_R\left(s_R^{{\tau }_R},s_B^J\right)& \cdots & u_R\left(s_R^{{\tau }_R},s_B^{{\tau }_B}\right)\end{array}\right]$$

(14)

$$P_B=\left[\begin{array}{ccccc}{u}_B\left(s_R^1,s_B^1\right)& \cdots & u_B\left(s_R^1,s_B^J\right)& \cdots & u_B\left(s_R^1,s_B^{{\tau }_B}\right)\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ u_B\left(s_R^I,s_B^1\right)& \cdots & u_B\left(s_R^I,s_B^J\right)& \cdots & u_B\left(s_R^I,s_B^{{\tau }_B}\right)\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ u_B\left(s_R^{{\tau }_R},s_B^1\right)& \cdots & u_B\left(s_R^{{\tau }_R},s_B^J\right)& \cdots & u_B\left(s_R^{{\tau }_R},s_B^{{\tau }_B}\right)\end{array}\right]$$

(15)

4. Analysis of Evolutionary Game Model

Under bounded rationality, game players have limited initial knowledge, which can result in the adoption of suboptimal strategies. This leads to constant adjustment and improvement of the payoffs of both sides during the game. Due to the differences in interests, those with lower payoffs will improve their strategies by learning from those with higher payoffs, resulting in changes in the proportion of each strategy over time. The dynamic change of strategies can be defined as a function of time. The dynamic change rate can be represented by a replicator equation based on biological evolution ideas [29]. This equation describes the dynamic change process of strategies and analyzes the equilibrium state of the evolutionary game to solve the equilibrium solution of the multi-UAV cooperative target assignment evolutionary game model. The specific algorithm is described as follows:

(I)

Given the probability space $I_R$ and $I_B$ of the UAVs’ and targets’ choices of each strategy in the evolutionary game,

(II)

The payoffs and expected payoffs of different target assignment strategies for both sides can be calculated using the probability space $I_R$ and $I_B$, and the payoff matrix. Equations (16)–(19) show the formulas for calculating these:

$$U_{\underset{R}{\overset{I}{s}}}=\sum _{J=1}^{m}p(s_B^J)u_R(s_R^I,s_B^J)$$

(16)

$${\overline{U}}_R=\sum _{I=1}^{n}p(s_R^I)U_{s_R^I}$$

(17)

$$U_{s_B^J}=\sum _{I=1}^{m}q(s_R^I)u_B(s_R^I,s_B^J)$$

(18)

$${\overline{U}}_B=\sum _{I=1}^{n}q(s_B^J)U_{s_B^J}$$

(19)

(III)

The replicator equations for UAVs on both sides are established as follows:

$$R(p)=\frac{dp(t)}{dt}=p(s_R^I)(U_{s_R^I}-{\overline{U}}_R)$$

(20)

$$B(p)=\frac{dq(t)}{dt}=q(s_B^J)(U_{s_B^J}-{\overline{U}}_B)$$

(21)

(IV)

To solve for the evolutionary stable equilibrium, we can combine the replicator equations for UAVs on both sides obtained in the previous step (III) to construct the following system of replicator equations:

$$\left\{\begin{array}{c}R(p(s_B^1))=\frac{dp(t)}{dt}=p(s_R^1)(U_{s_R^1}-{\overline{U}}_R)=0\\ ⋮\\ R(p(s_B^m))=\frac{dp(t)}{dt}=p(s_R^m)(U_{s_R^m}-{\overline{U}}_R)=0\\ B(q(s_R^1))=\frac{dq(t)}{dt}=q(s_B^1)(U_{s_B^1}-{\overline{U}}_B)=0\\ ⋮\\ B(q(s_R^n))=\frac{dq(t)}{dt}=q(s_B^n)(U_{s_B^n}-{\overline{U}}_B)=0\end{array}\right.$$

(22)

Solving the above system of equations yields the evolutionary equilibrium strategy for the multi-UAV cooperative target assignment evolutionary game model, which is a strategy where the selection probabilities of each player’s strategies remain unchanged. However, some of the evolutionary equilibrium strategies may be unstable. This means that if both players deviate from this equilibrium state, the replicator equation will cause the evolutionary outcome to no longer converge to this strategy. Therefore, further analysis of the stability of the evolutionary equilibrium strategy is necessary to identify the stable strategies within the equilibrium and achieve optimal target assignment.

5. Simulation Results and Analysis

5.1. Introduction to the RflySim Simulation Platform

This study utilizes the RflySim platform, which is built in the Matlab/Simulink environment for flight simulation. The RflySim platform is a UAV flight control ecosystem released by the Reliable Flight Control Group at Beihang University. In reference [30], a comparison between the multi-wing flight entity experiment and the simulation experiment based on the RflySim platform validates the high accuracy level of the platform’s simulation. The platform has undergone quantitative analysis tests and comparison experiments with fault injection, and its platform’s credibility was found to be above 90% (with 60% being the lowest value in the accuracy confidence interval). This fully confirms the high fidelity and practicality of the platform. The platform’s core value is reflected in the software and hardware in the loop simulation, which includes the unique CopterSim, visual system plugins, and developed models. The software platform components are shown in Figure 9.

5.2. Analysis of the Typical Scenario

This section constructs a typical scenario of three UAVs and two targets, as shown in Figure 10, where the red UAV represents our UAVs and the blue UAV represents the targets. Each UAV had two attacks and the same probability of being damaged. The situational information of each UAV is provided in

Table 1. Situational information of UAVs and targets.

UAV	$\mathit{x}\left(\mathbf{m}\right)$	$\mathit{y}\left(\mathbf{m}\right)$	$\mathit{z}\left(\mathbf{m}\right)$	$\mathit{\theta }\left(\mathbf{r}\mathbf{a}\mathbf{d}\right)$	$\mathit{\phi }\left(\mathbf{r}\mathbf{a}\mathbf{d}\right)$	$\mathit{v}(\mathbf{m}/\mathbf{s})$
$R_1$	1121	3335	400	0	0.85	78
$R_2$	1538	2998	400	0	0.85	67
$R_3$	758	3043	400	0	0.85	84
$B_1$	152	503	420	0	−1.07	82
$B_2$	322	750	420	0	−1.07	70

.

In Table 1, the units of x,y,z are meters (m), the unit of v is meters per second (m/s), and the units of θ and φ are radians (rad). Both sides must strategize to maximize their advantage in the conflict. The target assignment problem is modeled as an evolutionary game model $G=\left\{N,S,P,I\right\}$.

The specific process of its construction is as follows:

(I)

$N=\left\{R,B\right\}$ represents the UAVs on both sides in the evolutionary game model.

(II)

$S=\left\{S_R,S_B\right\}$ represents the strategy space in the evolutionary game model, as shown specifically in

Table 2. Strategy space on both sides.

Our side	${\mathit{s}}_{\mathit{R}}^1$	${\mathit{s}}_{\mathit{R}}^2$	${\mathit{s}}_{\mathit{R}}^3$	${\mathit{s}}_{\mathit{R}}^4$	${\mathit{s}}_{\mathit{R}}^5$	${\mathit{s}}_{\mathit{R}}^6$	${\mathit{s}}_{\mathit{R}}^7$	${\mathit{s}}_{\mathit{R}}^8$
	$\left[\begin{array}{cc}1& 0\\ 1& 0\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right]$	$\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}1& 0\\ 0& 1\\ 0& 1\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 1& 0\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 1& 0\\ 0& 1\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 0& 1\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 0& 1\\ 0& 1\end{array}\right]$
Target side	${\mathit{s}}_{\mathit{B}}^1$	${\mathit{s}}_{\mathit{B}}^2$	${\mathit{s}}_{\mathit{B}}^3$	${\mathit{s}}_{\mathit{B}}^4$	${\mathit{s}}_{\mathit{B}}^5$	${\mathit{s}}_{\mathit{B}}^6$	${\mathit{s}}_{\mathit{B}}^7$	${\mathit{s}}_{\mathit{B}}^8$	${\mathit{s}}_{\mathit{B}}^9$
	$\left[\begin{array}{ccc}1& 0& 0\\ 1& 0& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 0& 0& 1\end{array}\right]$	$\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\end{array}\right]$	$\left[\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 1& 0\\ 0& 1& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 1& 0\\ 0& 0& 1\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 1& 0& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 0& 1& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 0& 0& 1\end{array}\right]$

.

(III)

$I=\left\{I_R,I_B\right\}$ represents the probability space in the evolutionary game model, with equal initial probabilities as shown in

Table 3. Initial probabilities for each strategy of both sides.

Our side	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^1\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^2\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^3\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^4\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^5\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^6\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^7\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{R}}^8\right)$
	0.125	0.125	0.125	0.125	0.125	0.125	0.125	0.125
Target side	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^1\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^2\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^3\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^4\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^5\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^6\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^7\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^8\right)$	$\mathit{p}\left({\mathit{s}}_{\mathit{B}}^9\right)$
	0.1111	0.1111	0.1111	0.1111	0.1111	0.1111	0.1111	0.1111	0.1111

.

(IV)

Construct the payoff matrices on both sides.

First, calculate the attack effectiveness of each UAV based on the situational advantage function, resulting in the effectiveness matrices $T_R$ and $T_B$, as shown in Equations (23) and (24):

$$T_R=\left[\begin{array}{cc}0.57& 0.26\\ 0.85& 0.60\\ 0.28& -0.30\end{array}\right]$$

(23)

$$T_B=\left[\begin{array}{ccc}1.39& 1.11& 0.99\\ 0.92& 0.59& 0.51\end{array}\right]$$

(24)

Then, calculate the total effectiveness for each target assignment plan, based on Equations (10) and (11), as shown in

Table 4. Total effectiveness for each target assignment plan.

Red Side	Target Assignment Plan	$\left[\begin{array}{cc}1& 0\\ 1& 0\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right]$	$\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}1& 0\\ 0& 1\\ 0& 1\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 1& 0\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 1& 0\\ 0& 1\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 0& 1\\ 1& 0\end{array}\right]$	$\left[\begin{array}{cc}0& 1\\ 0& 1\\ 0& 1\end{array}\right]$
	Total Attack Effectiveness	0.33	1.05	0.29	0.31	0.38	0.29	0.36	1.42
target side Side	Target Assignment Plan	$\left[\begin{array}{ccc}1& 0& 0\\ 1& 0& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 0& 0& 1\end{array}\right]$	$\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\end{array}\right]$	$\left[\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 1& 0\\ 0& 1& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 1& 0\\ 0& 0& 1\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 1& 0& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 0& 1& 0\end{array}\right]$	$\left[\begin{array}{ccc}0& 0& 1\\ 0& 0& 1\end{array}\right]$
	Total Attack Effectiveness	0.38	0.41	0.42	0.31	0.40	0.41	0.33	0.27	0.40

.

On this basis, calculate the payoffs for UAVs on both sides for each combination of strategies, based on Equations (12) and (13), resulting in the payoff matrices $P_R$ and $P_B$ for the UAVs on both sides, as shown in Equations (25) and (26).

$$P_R=\left[\begin{array}{ccccccccc}0.97& 0.96& 0.96& 1.02& 0.97& 0.96& 1.01& 1.07& 0.97\\ 0.72& 0.73& 0.73& 0.69& 0.72& 0.73& 0.70& 0.67& 0.73\\ 0.93& 0.92& 0.92& 0.97& 0.92& 0.92& 0.96& 1.02& 0.92\\ 0.96& 0.95& 0.94& 1.00& 0.95& 0.95& 1.00& 1.05& 0.95\\ 1.00& 0.99& 0.97& 1.05& 1.00& 1.00& 1.04& 1.10& 0.99\\ 0.94& 0.93& 0.93& 0.98& 0.93& 0.93& 0.97& 1.03& 0.93\\ 0.99& 0.98& 0.97& 1.03& 0.98& 0.98& 1.02& 1.08& 0.98\\ 0.40& 0.44& 0.45& 0.31& 0.42& 0.43& 0.33& 0.23& 0.43\end{array}\right]$$

(25)

$$P_B=\left[\begin{array}{cccccccc}1.03& 1.38& 1.07& 1.04& 1.00& 1.06& 1.01& 2.50\\ 1.04& 1.35& 1.09& 1.06& 1.01& 1.07& 1.03& 2.26\\ 1.04& 1.34& 1.09& 1.06& 1.01& 1.08& 1.03& 2.21\\ 0.98& 1.42& 1.03& 1.00& 0.96& 1.01& 0.97& 3.21\\ 1.04& 1.36& 1.08& 1.05& 1.00& 1.06& 1.02& 2.36\\ 1.04& 1.35& 1.09& 1.05& 1.01& 1.07& 1.02& 2.30\\ 0.99& 1.41& 1.04& 1.01& 0.97& 1.03& 0.98& 3.01\\ 0.94& 1.46& 0.98& 0.95& 0.91& 0.97& 0.92& 4.29\\ 1.04& 1.36& 1.08& 1.05& 1.01& 1.07& 1.02& 2.34\end{array}\right]$$

(26)

To obtain an evolutionary equilibrium solution for the multi-UAV network target assignment game, a replicator equation is constructed. This equation describes how the distribution of strategies evolves based on their relative payoffs. The resulting equilibrium solution represents a stable distribution of strategies that will persist over time. The evolutionary trend of strategies for UAVs on both sides can be visualized through Figure 11 and Figure 12, which show the proportion of different strategies over time.

It can be observed from Figure 11 and Figure 12 that to achieve higher payoffs, our side will eventually tend to select the 6-th strategy as the target assignment plan. In contrast, the target side will eventually tend to select the 2-nd strategy as the target assignment plan. The target assignment results of both sides are shown in Figure 13, where the red dots represent our UAVs, the blue dots represent the target UAVs, the red lines represents the strategy of our UAVs to select the target, and the blue lines represents the strategy of the target UAVs to select the target. This figure provides a visual representation of the final target assignments resulting from the use of the selected strategies by both sides. As can be seen in Figure 13, our $R2$ chooses target $B1$, and our $R1$ and $R3$ choose target $B2$; under this strategy, our side has the greatest benefit, choosing this strategy to eliminate $B2$ first, and $R2$ attacks $B1$ for cover, and targets $B1$ and $B2$ choose our $R3$. Under this strategy, the target side has the greatest benefit; choosing this strategy can eliminate $R3$ in a short period, reducing their own loss risk.

5.3. Results and Analysis of the Effectiveness Experiment

The advantage or disadvantage between both sides is largely dependent on the number of UAVs, their performance, and the relative situation between them. When the performance of the UAVs on both sides is equal, the advantage or disadvantage between the sides is primarily determined by the number of UAVs and the relative situation between them. To evaluate the effectiveness of the target assignment method for multi-UAVs based on the evolutionary game, we designed typical scenarios that include our advantageous scenario, our disadvantageous scenario, and an evenly-matched scenario by adjusting the number of UAVs on both sides. To compare the effectiveness of our proposed method, we selected the Min–Max strategy, PSO, GA and Hungarian Algorithm as baselines [31,32,33,34]. These methods represent commonly used approaches for multi-UAV network target assignment and provide a useful benchmark for evaluating the performance of our proposed method.

At the conclusion of the simulation experiment, three simulation results were defined from the perspective of our sides based on the remaining number of UAVs on both sides, as follows:

Our side victory: At the end of the simulation, our side has more remaining UAVs than the target side.

Our side defeat: At the end of the simulation, our side has fewer remaining UAVs than the target side.

Draw: At the end of the simulation, both sides have the same number of remaining UAVs.

The simulation results are analyzed through the winning rate, losing rate, and draw rate [35], for which the detailed descriptions and calculations are shown in

Table 5. Target assignment results on both sides.

Indicator Name	Indicator Description	Computational Formula
$r_w$	Probability of win	(Winning count/Total simulation count) × 100%
$r_d$	Probability of draw	(Draw count/Total simulation count) × 100%
$r_l$	Probability of loss	(Losing count/Total simulation count) × 100%

.

For each typical scenario, the target side used the proposed method outlined in this paper, while our side used the method proposed in this paper and the baselines to conduct 400 simulation experiments. Based on the simulation results, our side’s win, loss, and draw rates were calculated, as shown in

Table 6. Simulation experiment results.

Typical Scenario	Method	${\mathit{r}}_{\mathit{w}}$	${\mathit{r}}_{\mathit{d}}$	${\mathit{r}}_{\mathit{l}}$
Advantage (4v2)	Evolutionary Game	89.75%	2.75%	7.5%
	Min–Max	83%	6.75%	10.25%
	GA	85.25%	4.75%	10%
	PSO	87%	8.5%	4.5%
	Hungarian Algorithm	87.25	3.75%	9%
Evenly-matched (2v2)	Evolutionary Game	38.5%	27.25%	34.25%
	Min–Max	30.75%	24.5%	44.75%
	GA	31%	26.25%	42.75%
	PSO	37.5%	21.75%	40.75%
	Hungarian Algorithm	37.25%	27%	35.75%
Disadvantage (2v4)	Evolutionary Game	9.25%	4.25%	86.5%
	Min–Max	6.25%	2.25%	91.5%
	GA	5.5%	6.5%	88%
	PSO	7%	5.5%	87.5%
	Hungarian Algorithm	8%	3.25%	88.75
Averaged value of the three scenarios	Evolutionary Game	45.83%	11.42%	42.75%
	Min–Max	40%	11.17%	48.83%
	GA	40.58%	12.5%	46.92%
	PSO	43.83%	11.92%	44.25%
	Hungarian Algorithm	44.17%	11.33%	44.5%

.

The experimental results demonstrate that the proposed method outperforms the selected baselines to varying degrees in the three typical scenarios. In our advantageous scenario, the proposed method achieved a winning rate of 89.75%, a draw rate of 2.75%, and a loss rate of 7.5%, while the selected baselines achieved success rates ranging from 83% to 87.25%. In the evenly-matched scenario, the proposed method achieved a winning rate of 38.5%, a draw rate of 27.25%, and a loss rate of 34.25%, while the selected baselines achieved success rates ranging from 30.75% to 37.5%. In our disadvantageous scenario, the proposed method achieved a winning rate of 9.25%, a draw rate of 4.25%, and a loss rate of 86.5%, while the selected baselines achieved success rates ranging from 5.5% to 8%. Across our advantageous, evenly-matched, and disadvantageous scenarios, the proposed method achieved an average winning rate of 45.83%, an average draw rate of 11.42%, and an average loss rate of 42.75%, outperforming the average success rates of the selected baselines which ranged from 40% to 44.17%.

The results of our proposed method indicate its potential to both enhance advantages for our side in advantage scenarios, as well as mitigate disadvantages in unfavorable situations. By implementing evolutionary game theory, we are able to consider possible countermeasures that the target side may adopt in response to our actions. This allows our approach to outperform selected baseline methods by anticipating dynamic interactions between our system and the target. Evolutionary game theory takes into account the adversarial nature of real-world environments when assigning UAV targets. It models the evolution of combined strategies over time as both sides continuously adapt to each other’s decisions and equilibrium shifts. This makes evolutionary game theory a more robust and flexible framework compared to alternative approaches. Overall, evolutionary game theory enhances the resilience and long-term effectiveness of our target assignment scheme in practical situations involving strategic adversaries.

6. Conclusions

In this paper, the multi-UAV network target assignment problem is investigated in a 3D scenario to solve the assignment problem for moving air targets. The method uses evolutionary game theory to model the multi-UAV network target assignment problem, and constructs the payment matrices of both sides by establishing the situational assessment function and a multi-level information fusion algorithm. Then, the evolutionary equilibrium solution is solved by constructing replicator equations to find the optimal target assignment strategy for both sides, which confers the benefit of balancing each side’s interest and the overall benefit. Finally, the whole process of the proposed method is analyzed through the typical scenario and the effectiveness experiment is analyzed in comparison with other algorithms in three scenarios. The experimental results show that the win rate of the game under the proposed method is higher than that for the Min–Max strategy, PSO, GA, and Hungarian Algorithm, and the performance of the proposed method is the best.

In future work, we plan to expand our analysis to consider factors such as opponent occupancy uncertainty and resource adequacy and uncertainty to further study the problem of multi-UAV network target assignment. We aim to improve the applicability and robustness of our solution method. Additionally, we will incorporate a UAV maneuvering decision-making algorithm to maximize the exceptional efficiency of the UAV during the movement process. These future efforts will help to advance our understanding of multi-UAV application scenarios and provide valuable insights into the development of effective target assignment strategies.