Attention Mechanism and Neural Ordinary Differential Equations for the Incomplete Trajectory Information Prediction of Unmanned Aerial Vehicles Using Airborne Radar

Abstract

Due to the potential for airborne radar to capture incomplete observational information regarding unmanned aerial vehicle (UAV) trajectories, this study introduces a novel approach called Node-former, which integrates neural ordinary differential equations (NODEs) and the Informer framework. The proposed method exhibits high accuracy in trajectory prediction, even in scenarios with prolonged data interruptions. Initially, data outside the acceptable error range are discarded to mitigate the impact of interruptions on prediction accuracy. Subsequently, to address the irregular sampling caused by data elimination, NODEs are utilized to transform computational interpolation into an initial value problem (IPV), thus preserving informative features. Furthermore, this study enhances the Informer’s encoder through the utilization of time-series prior knowledge and introduces an ODE solver as the decoder to mitigate fluctuations in the original decoder’s output. This approach not only accelerates feature extraction for long sequence data, but also ensures smooth and robust output values. Experimental results demonstrate the superior performance of Node-former in trajectory prediction with interrupted data compared to traditional algorithms.

1. Introduction

Unmanned aerial vehicles (UAVs) pose a significant risk to the safety of airplanes [1,2,3] when they intrude into no-fly zones, as ground-based airport facilities struggle to detect such diminutive vehicles. In practical applications, airborne radar systems offer a means of detecting and distinguishing UAV formations from varied perspectives [4], making the prediction of UAV trajectories based on limited historical radar data a valuable pursuit [5].

Due to clutter and jamming signals, a radar’s output often contains discontinuities or errors. Swift and accurate prediction of target trajectories is paramount for preventing target loss [6]. Nevertheless, trajectory prediction based on airborne radar faces two major challenges [7,8].

Firstly, traditional filtering algorithms such as the Kalman filter (KF) and extended Kalman filter (EKF) can indeed predict interrupted data, but their application is limited in the case of prolonged interruptions. As the interruption time interval lengthens, the accuracy of the predicted target trajectory plummets. Additionally, to construct a target trajectory from effective data points, methods such as the nearest neighbor (NN) algorithm, probability data association (PDA) algorithm, and joint probability data association (JPDA) algorithm [9,10,11] are frequently employed. However, the performance of these methods heavily relies on the availability of valid data points. To ensure the validity of these points, a validation gate rule is typically applied, where only plots within certain thresholds for range, velocity, and angle are considered as candidates for trajectory formation after association with previous data points. If the validation gate is set too wide, jamming points may sneak in, resulting in significant errors in the predicted target trajectory. Conversely, a narrow validation gate may lead to target loss when the trajectory point fails to associate with new points for an extended period.

Due to the limited extrapolation capabilities of the aforementioned algorithms, their prediction accuracy suffers when historical data are inaccurate. To address this problem, machine learning algorithms are employed for accurate trajectory prediction. For instance, radar measurement data are used to dynamically fine-tune the modeled aircraft quality, enhancing the precision of aircraft trajectory forecasts. Alternatively, in-flight airspace environmental parameters and spatiotemporal attributes are incorporated into machine learning and hidden Markov models to predict trajectories under uncertain conditions [12]. Furthermore, the target trajectory points captured by airborne radar are analyzed as time-series data. Here, LSTM-based algorithms, which employ gate recurrent units (GRUs), demonstrate excellent performance in predicting target trajectories [13,14,15]. Notably, transformer-based algorithms, leveraging the self-attention mechanism, can discern input data points and assign weights based on their mutual relationships and impact on the output.

Apart from these methods, neural network-based approaches offer flexibility, requiring minimal manual parameter adjustment [16]. In comparison to LSTM and GRUs, neural networks excel in multi-step prediction tasks. One such example is the Informer algorithm, a modified transformer variant, tailored for time-series data processing [17]. This algorithm enhances self-attention with prob-sparse self-attention, significantly reducing time complexity from $O\left(L^2\right)$ to $O\left(LlogL\right)$ [17]. While LSTM, transformers, and their derivatives are proficient in predicting time-series data, maintaining prediction accuracy in the presence of prolonged data errors or interruptions remains a challenge [18].

Secondly, the interruption of data often results in irregular sampling, rendering traditional data prediction methods ineffective. For instance, recurrent neural network (RNN)-based or transformer-based approaches struggle to handle irregular time-series [18,19]. The conventional RNN approach involves dividing time into equal intervals and utilizing averages for input or aggregation, but this pre-processing can distort data information, particularly for real-time measurements that contain significant hidden details [19,20].

To tackle irregularly sampled data, neural ordinary differential equations (neural ODEs) offer exceptional capabilities. Neural ODEs are a family of continuous-time models that utilize deep neural networks to parameterize the derivatives of hidden states [19,20,21]. Through defining the hidden state as the solution to an ODE initial value problem, the hidden state can be computed at any point in time using an ODE solver. Additionally, the adjoint method is proposed to replace the backpropagation algorithm, as the gradient does not directly pass through the ODE solver during forward propagation. This significantly reduces model memory consumption, as only the hidden state of the last time step needs to be stored, eliminating the need to store intermediate values during forward computation [22,23,24].

In this paper, we introduce the Unite Neural Ordinary Differential and Informer (Node-former) method, which demonstrates robust trajectory prediction accuracy, even in the presence of prolonged and significant data perturbations. The methodology can be succinctly outlined in three stages, as shown in Figure 1. Initially, we selectively retain interference data within an acceptable error range, thereby mitigating the adverse effects of data perturbation. Secondly, following the elimination of disrupted data, the remaining effective data are characterized as irregularly sampled. To address this irregularity, we employ the neural ordinary differential equations (NODEs) approach, transforming the computational interpolation into an initial value problem (IPV). This approach effectively circumvents the loss of informative features that can occur with traditional interpolation methods. Finally, we enhance the Informer algorithm’s encoder with prior knowledge to facilitate the trajectory prediction process. To mitigate fluctuations in the original decoder’s output, we utilize an ODE solver as the decoder. Experimental results demonstrate that, compared to other interpolation and trajectory prediction algorithms, our proposed Node-former method significantly improves trajectory prediction performance under prolonged interrupted conditions.

In this paper, we present the following key contributions:

(1) Integrating Neural Ordinary Differential Equations (NODEs) with Attention Mechanism for UAV Trajectory Prediction: This innovative strategy enables us to gracefully manage irregularly sampled data, leveraging NODEs while also broadening our view to encompass global context through the self-attention mechanism. This synergy allows our model to delicately capture temporal fluctuations and long-distance correlations within UAV trajectory data;

(2) Presenting the Node-former Model for Enhancing Radar Interference Management: We propose the Node-former model, a thoughtful three-stage methodology uniquely designed to tackle radar interference in target trajectory data. By mitigating the disruptive effects of interference, our model strives to produce more refined trajectory forecasts. The Node-former model signifies a valuable advancement in bolstering the dependability and accuracy of trajectory prediction amid radar interference scenarios;

(3) Comprehensive Simulation Studies Illustrating Performance Enhancements: Through meticulous simulation experiments, we have demonstrated that our proposed method consistently outperforms conventional trajectory prediction methods across diverse experimental setups. This performance improvement underscores the effectiveness, resilience, and strength of our approach in the realm of UAV trajectory prediction. Our findings not only reinforce the theoretical underpinnings of our work but also underscore its practical relevance and potential for practical deployment.

This paper is organized as follows. In Section 2, we delve into the problem model and analysis for target trajectory prediction by an airborne radar. This section provides a comprehensive understanding of the challenges and complexities involved in accurate trajectory estimation. Section 3 outlines the workflow of the proposed Node-former method and the target trajectory prediction module. We detail the steps involved in the method, highlighting how it addresses the issues mentioned in Section 2. Section 4 presents the simulation results and experimental outcomes, demonstrating the effectiveness and robustness of our Node-former method for trajectory prediction, especially under the conditions of long-time interrupted data. Finally, Section 5 summarizes our work, highlighting the key contributions and implications of our findings. We discuss potential future directions and applications of the Node-former method.

2. Problem Model and Analysis

In this section, we introduce the problem of target trajectory prediction using an airborne radar, specifically focusing on the challenges posed by interrupted data points. As illustrated in Figure 2, when an airborne radar observes a UAV target, ideal echo data can be collected when the jammer on the UAV is powered off. However, when the jammer is activated, the received data become interrupted, leading to significant errors in tracking and trajectory prediction for the UAV target.

To address this issue, we propose a novel method called the Unite Neural Ordinary Differential and Informer (Node-former). This method aims to achieve high accuracy in trajectory prediction, even in the presence of prolonged and strong jamming interference. Leveraging the combined power of neural ordinary differential equations and the Informer architecture, our Node-former method is designed to effectively handle the challenges posed by interrupted data and ensure reliable performance under adverse conditions.

The key challenge in this research is predicting the target trajectory when the target becomes lost within historical track data. To achieve accurate trajectory prediction, it is crucial to first detect and identify any data that have been interfered with. To tackle this problem, we break it down into two distinct steps: handling jamming data and making predictions using historical data.

When describing the target points, the target trajectory without any interruptions can be mathematically expressed as follows:

$$\mathit{x}\left(t\right)=(x_0, x_1,\cdots , x_N)$$

(1)

where $\mathit{x}$$\left(t\right)$ is the function that represents the normal target trajectory data.

As depicted in Figure 2, when the jammer is active, the radar data being observed experience interruptions. In this study, these interruptions are primarily attributed to ambient noise combined with multipath effects. Ambient noise can be modeled using a Gaussian distribution, whereas the reflected waves resulting from multipath effects adhere to a Rayleigh distribution. To detect these interrupted data points, we use range gate pull-off (RPGO) and velocity gate pull-off (VPGO) techniques [25,26,27]. Due to the presence of interrupted data, the radar’s range and speed gates gradually exhibit significant observation errors. Consequently, the interrupted target trajectory can be mathematically expressed as

$$\mathit{x}\left(t\right)=(x_0, x_1,\cdots , x_N)+(j_0, j_1,\cdots , j_N)$$

(2)

where $\mathit{j}\left(t\right)$ represents the jamming trajectory data.

The primary reason for losing track of a target, apart from the misdirection of the transmitted beam caused by jamming that drags the radar gates out of alignment, is the target’s maneuvering, which can cause it to move outside the radar’s tracking range. Accurate prediction of a target’s position during a tracking loss hinges on estimating the position with the highest probability of being the target’s current location by analyzing and learning the target’s maneuvering behavior.

As demonstrated in Equation (2), when the target trajectory is interrupted, our approach can detect the effective, non-interrupted data. It is these effective data samples that we rely on for predicting the target trajectory. Following the detection of jamming data, the remaining effective data samples can be expressed as

$$\mathit{x}\left(t\right)=(x_0, x_1,\cdots , x_k, 0 ,\cdots ,0 , x_{k+L},\cdots , x_N)$$

(3)

where N is the effective data length and L is the interrupted length. Data jamming detection identifies invalid samples. Interrupted samples may miss targets; therefore, predicting their trajectories is crucial.

Problem Analysis

In this research, we initially eliminate from the jamming data the radar measurement traces that deviate from the motion pattern of the intended target, thus establishing a dedicated data-filtering module. When jamming persists for extended durations, it becomes inefficient to discard all jammed data; thus, it suffices to preserve as much data as possible while eliminating those with significant errors.

To execute this filtering function, we adopt a neural network architecture comprising multiple linear layers. The key advantage lies in transforming the target’s performance into a simplified classifier. Through training the neural network to predict whether the target falls within a predetermined error range, we eliminate the need for precise target measurements. However, the filtered data no longer adhere to a regular sampling pattern, necessitating a processing algorithm that is capable of handling irregularly sampled data. Therefore, we employ neural ODEs, which excel in handling such irregular sampling, for data processing.

The historical data encompass multifaceted features, including the target’s time, speed, and distance. The prediction task focuses on identifying the feature that significantly impacts the target’s location and assigning weight parameters accordingly. Inspired by human attention, which filters out unimportant information and focuses on priorities to grasp patterns and predict outcomes, we introduce an attention mechanism. This mechanism applies to the prediction process by observing and learning from historical data, allocating attention to specific features to capture movement patterns and predict the target’s trajectory.

This study introduces an enhanced transformer structure tailored specifically for the task of time-series prediction. While the traditional sequence-to-sequence approach can provide a one-time output, it suffers from a limitation known as the long short-term forecast (LSFT) problem. Specifically, the output values in this approach are generated independently, without considering the previously predicted values, resulting in significant data fluctuations.

To address this issue, we modify the original decoder component of the transformer to incorporate an ODE solver. This modification ensures that each output data value is correlated with the preceding values, resulting in a smoother predicted trajectory. Leveraging the ODE solver, we are able to capture the temporal dependencies within the time-series data and generate predictions that are both accurate and continuous.

3. Proposed Method

To address the challenge of predicting target trajectories in the presence of interference data, this section provides an in-depth examination of the structural framework of the proposed Node-former methodology. We delve into the rationale behind the design choices and elucidate the innovative aspects of each constituent component.

3.1. Algorithm Framework

The proposed algorithm encompasses two integral components. The first is the anti-jamming module, tasked primarily with identifying jamming source data, eliminating it following an error assessment, and subsequently calculating interpolation values to compensate for the eliminated data. The second component, the track prediction module, leverages historical information to generate trajectory forecasts. This historical information is compiled by concatenating radar measurement data with interpolated information processed with the anti-jamming module. Figure 3 visually depicts the algorithm’s structural framework. The subsequent sections elaborate on each algorithmic segment in detail.

3.2. Data Pre-Processing

The data pre-processing is conducted on the input radar’s original measurement data. The radar can identify whether the measurement information is jammed at this time by detecting the channel and then giving a jamming flag. Through the jamming flags, the normal radar measurement data can be separated from the jamming data. As shown in Figure 4, radar jamming data will be sent to the jamming data filter module for processing in the next step.

3.3. Jamming Data Filtering

In addressing the jamming data, this study employs a neural network with multiple linear layers to filter out points with significant errors, retaining measurements with minor errors for use as observations. An error threshold $\alpha$ is set for the selected column vectors, and observations within this threshold are deemed approximations of the radar’s internal anti-jamming strategy’s true values. For irregularly sampled jamming datasets, data exceeding the $\alpha$ threshold are set to null (NA). This is illustrated in Figure 5.

Here, the start and end positions of the jamming data segment are chosen from the radar’s normal measurement state. This approach ensures that, even if most or all jamming data values exceed the error range, the first and last values can serve as endpoints, facilitating the anti-jamming module’s operation and enhancing interpolation accuracy.

Notably, this methodology mitigates the issue of poor historical data quality stemming from prolonged data disturbances, as correcting each individual value in such cases can be challenging.

3.4. Anti-Jamming Module

After the jamming data filtering module, the resulting data exhibit irregular sampling. Given the challenges that the RNN and transformer architectures face with such data, this study proposes using neural ODEs to address this issue. Neural ODEs decode network hidden variables and model dynamics under time continuity, and interpolate missing data points.

This study primarily employs an anti-jamming structure based on neural ODEs (NODEs), which can be interpreted as ResNets with an infinite number of layers [28]. The hidden state $h\left(t\right)$ is formulated as an ODE initial value problem in a continuous time-series model, defined by ordinary differential equations, as shown in Equations (4) and (5) [19,21,22]:

$$h_{t+1}=h_t+f(h_t+{\theta }_t)$$

(4)

$${\displaystyle \frac{d}{dt}}h\left(t\right)=f\left(h\left(t\right), t, \theta \right)$$

(5)

where f is the function that represents the hidden state dynamics through the neural network parameter $\theta$.

ODE–RNN is a generalized application of RNN to neural differential equations. In Equation (6), the hidden states between observations are defined as

$$h_i^{\prime }=ODESolve\left(f_{\theta }, h_{i-1},\left(t_{i-1},t_i\right)\right)$$

(6)

RNN is used to update the hidden state of the intermediate process based on the following observations:

$$h_i=RNNCell\left(h_i^{\prime }, x_i\right)$$

(7)

The hidden state can be calculated at any expected moment using the ODE solver by iterating Equations (6) and (7), as shown in Equation (8):

$$h_0,\dots ,h_N=ODESolve\left(f_{\theta }, h_0,\left(t_0,\dots ,t_N\right)\right)$$

(8)

In this study, we enhance the encoder–decoder structure using ODE–RNN, which combines ODE–LSTM as the encoder and an ODE as the decoder. ODE–LSTM updates hidden states between observations, while the ODE handles interpolation between them. To approximate the posterior at $t_0$, we propagate the ODE–LSTM encoder backward in time from $t_N$ to $t_0$, yielding an estimate of the initial state’s posterior q. Solving the ODE, we can obtain the potential state at any point of interest, as illustrated in Figure 6.

According to the time-series model generated using the ODE definition, the initial hidden state determines the following for the entire track:

$$z_0\sim p\left(z_0\right)$$

(9)

$$z_0,\dots , z_N=ODESolve\left(f, z_0,\left(t_0,\dots ,t_N\right)\right)$$

(10)

$$x_i\sim p\left(x_i|z_i\right), i=0, 1,\dots N$$

(11)

where $z_0$ is represents the initial state randomly drawn from the probability distribution $p\left(z_0\right)$. $x_i$ represents the observation conditioned on the latent variable $z_i$.

The mean and standard deviation of the approximate posterior parameters q as a function of the final hidden state concerning the ODE–LSTM are given as follows:

$$q\left(z_0|{\left\{x_i,t_i\right\}}_{i=0}^N\right)=\mathcal{N}\left({\mu }_{h0},{\sigma }_{h0}\right)$$

(12)

$$\left[{\mu }_{z0},{\sigma }_{z0}\right]=g\left(ODE\_LST{M}_{\varnothing }\left({\left\{x_i,t_i\right\}}_{i=0}^N\right)\right)$$

(13)

where g is the neural network that transforms the mean and variance of the final hidden state in the encoder.

The encoder and decoder are trained by maximizing the evidence lower bound (ELBO) [21,28] using the following equation:

$$ELBO\left(\theta ,\varphi \right)=E_{z_0\sim q_{\varphi }\left(z_0|{\left\{x_i,t_i\right\}}_{i=0}^N\right)}\left[\log p_{\theta }\left(x_0,\dots ,x_N\right)\right]-\mathrm{KL}[q_{\varphi }\left(z_0|{\left\{x_i,t_i\right\}}_{i=0}^N\right)\left|\right|p\left(z_0\right)]$$

(14)

The ODE solver is a fifth-order solver with adaptive steps. It is found in the torchdiffeq Python package.

The radar measurement data of multiple dimensions are encoded as latent variable $z_0^{\prime }$ by ODE–RNN, and $z_0^{\prime }$ is entered into the feed-forward network g to obtain its ${\mu }_{z0}$ and ${\sigma }_{z0}$. The distribution of $z_0^{\prime }$ is obtained by $\mathrm{N}({\mu }_{z0},{\sigma }_{z0})$, and the latent variable $z_0$ is obtained next. $z_i$ is calculated by the solution $ODESolve(f,z_0,(t_0,\cdots ,t_N))$, and the predicted value is found by decoding $z_i$.

The anti-jamming module processes jamming data as shown by the flow in Figure 7.

3.5. Data Contact

The next step is to collocate the anti-jamming processed data in the previous section with the normal radar measurement data. According to the time stamp of each data point, the data corrected by the anti-jamming module are concatenated with the normal measurement data to form the historical data as the input of the prediction module.

3.6. Track Prediction Module

Track prediction is the most important part of this study. For time-series data prediction, this study combines the transformer algorithm architecture with the nodes algorithm connotation. Transformer architecture is more suitable for LSFT problems, and, here, an improved self-attention is applied to effectively reduce the computational complexity of the model [29,30].

3.6.1. Self-Attention

In the self-attention mechanism, the input signals are classified into a query (Q), key (K), and value (V). For a given query, the attention weight (a) corresponding to the $k_i$ in the self-attention mechanism is obtained according to the correlation $a(q,k_i)$ between Q and $k_i$, as follows [16,30,31]:

$$a(q,k_i)={\displaystyle \frac{q·k_i}{∥q∥\times ∥k_i∥}}$$

(15)

On the other hand, the self-attention formula uses the result in Equation (14) to operate on the pair with the corresponding V to obtain the attention output, as shown in Equation (16), which is collated to obtain Equation (15), with d being the dimensions of Q and K:

$$f\left(q,\left(k_1,v_1\right),\dots \left(k_n,v_n\right)\right)=\sum _{j=1}^{n}Softmax\left(a(q,k_i)\right)·v_i$$

(16)

$$A\left(Q,K,V\right)=Softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d}}}\right)V$$

(17)

Based on the formula derivation and experiments presented in the relevant literature [17], we can understand that a minority of query and key determines the result of Equation (17), which means that the output in the sequence has a high correlation with only a few inputs. Through randomly sampling the query-wise score from the attention map, a method is defined to select Q. The Q is sorted into active query and lazy query. The probability sparse is found by sampling K to obtain the K-sample, and then finding each $q_i$ in Q about the K-sample to find the $M(q_i,K)$, as shown in Equation (18) [17,32]:

$$M\left(q_i,K\right)=\ln \sum _{\mathrm{j}=1}^{{\mathrm{L}}_{\mathrm{k}}}exp\left({\displaystyle \frac{{\mathrm{q}}_{\mathrm{i}}{\mathrm{k}}_{\mathrm{j}}^{\mathrm{T}}}{\sqrt{\mathrm{d}}}}\right)-{\displaystyle \frac{1}{{\mathrm{L}}_{\mathrm{k}}}}\sum _{\mathrm{j}=1}^{\mathrm{n}}{\displaystyle \frac{{\mathrm{q}}_{\mathrm{i}}{\mathrm{k}}_{\mathrm{j}}^{\mathrm{T}}}{\sqrt{\mathrm{d}}}}$$

(18)

where $L_K$ represents the length of the sequence acquired this time. The first half of the formula represents the value of $q_i$ on all keys; the latter is the arithmetic mean. More than the arithmetic mean of $q_i$ is regarded as an active query, and the rest is regarded as a lazy query.

Setting upper and lower bounds on $M(q_i,K)$, Equation (18) can be approximated to derive Equation (19):

$$\overline{\mathrm{M}}\left(q_i,K\right)={\max }_{\mathrm{j}}\left\{{\displaystyle \frac{{\mathrm{q}}_{\mathrm{i}}{\mathrm{k}}_{\mathrm{j}}^{\mathrm{T}}}{\sqrt{\mathrm{d}}}}\right\}-{\displaystyle \frac{1}{{\mathrm{L}}_{\mathrm{k}}}}\sum _{\mathrm{j}=1}^{\mathrm{n}}{\displaystyle \frac{{\mathrm{q}}_{\mathrm{i}}{\mathrm{k}}_{\mathrm{j}}^{\mathrm{T}}}{\sqrt{\mathrm{d}}}}$$

(19)

where $\overline{\mathrm{M}}\left({\mathrm{q}}_{\mathrm{i}},\mathrm{K}\right)$ represents the query sparsity measurement, which can be used to find $\overline{Q}$.

For track prediction, because of the correlation between track points, more attention should be assigned to the track points that are closer to the moment of target loss. Therefore, to help improve the allocation of attention, this research introduces a time-series-based residual formula term for the attention scores. Based on the above derivation, a new ProSparse self-attention is acquired. Equation (20) [12,22,33] is as follows:

$$A\left(Q,K,V\right)=Softmax\left({\displaystyle \frac{\overline{Q}{K}^T}{\sqrt{d}}}+ln\left({\displaystyle \frac{1}{t_N-t_1}}\right)f\left(t_N\right)\right)V$$

(20)

where $\overline{Q}$ is composed of the top-u $q_i$ selected by Equation (18) based on the size of the calculation result and the other unselected $q_i$ (the unselected $q_i$ are directly given by the mean value) and f is the fitting function obtained from the time residuals. Thus, the space complexity $O\left(L^2\right)$ generated by the self-attention mechanism is reduced to $O(LlnL)$.

3.6.2. Knowledge Distillation

A knowledge distillation mechanism is introduced to reduce the computation time by effectively saving memory overhead to halve the sequence length. The layer j to j + 1 distillation operation is shown in Equation (21) [12]:

$${\mathrm{X}}_{\mathrm{j}+1}^{\mathrm{i}}=\mathrm{MaxPool}\left(\mathrm{ELU}\left(\mathrm{Conv}1\mathrm{d}\left({\left[{\mathrm{X}}_{\mathrm{j}}^{\mathrm{i}}\right]}_{\mathrm{AB}}\right)\right)\right)$$

(21)

where MaxPool represents the maximum pooling, ELU is the activation function, Conv1d represents the one-dimensional convolution operation, and ${\left[X_j^i\right]}_{AB}$ represents the ProSparse self-attention operation with multi-head attention.

The historical data in Figure 8 are encoded by the encoder and fed into the decoder composed of the ODE solver, which then outputs the predicted data after a fully connected layer, as shown in Figure 9.

The prediction module in this study improves the encoder by introducing a time function variable to adjust the attention weights to accelerate the convergence speed. Additionally, it uses the ODE solver as the decoder so that the output data values can make full use of the previous prediction values to ensure that the trajectory change curve is smooth.

3.7. Implementation Details of the Node-Former Algorithm

To facilitate the understanding of the implementation details of Node-former, we provide a pseudocode outlining the key steps involved in both the training and testing phases of the algorithm. The training pseudocode Algorithm 1 details the process of optimizing the model parameters to fit the training data, while the testing pseudocode Algorithm 2 outlines how the trained model is utilized to make predictions on loss data. These pseudocodes, presented below, offer a concise yet comprehensive view of the Node-former methodology.

Algorithm 1: The training pseudocode of Node-former
	Input: Jamming radar training dataset $\mathcal{D}$, jamming filtering network $\beta$, anti-jamming network $\theta$, track prediction network $\varphi$, update epochs for jamming data filtering phase $K_1$, update epochs for anti-jamming phase $K_2$, update epochs for track prediction phase $K_3$
1	// Jamming Data Filtering Training Phase
2	for $u pdate epoch k=1\dots K_1$ do
3			Sample a minibatch sample pairs X from $\mathcal{D}$
4			Divide the sample pairs X into two parts: $X_{Jam}$ and $X_{Norm}$, as described in Section 3.2
5			Train the filtering network $\beta$ using the divided parts $X_{Jam}$, as described in Section 3.3
6	end
7	// Anti-jamming Training Phase
8	for $u pdate epoch k=1\dots K_2$ do
9			Sample a minibatch sample pairs X from $\mathcal{D}$ and divide into two parts: $X_{Jam}$ and $X_{Norm}$
10			Apply the trained filtering network $\beta$ to $X_{Jam}$ and get the irregular sample pairs $X_{Jam}^{filter}$
11			Train the anti-jamming network $\theta$ using the irregular pairs $X_{Jam}^{filter}$, as described in Section 3.4
12	end
13	// Track Prediction Training Phase
14	for $u pdate epoch k=1\dots K_3$ do
15			Sample a minibatch sample pairs X from $\mathcal{D}$ and divide into two parts: $X_{Jam}$ and $X_{Norm}$
16			Apply the trained filtering network $\beta$ and the trained anti-jamming network $\theta$ to $X_{Jam}$
17			Output the correctd sample pairs $X_{Cor}$ and contact with normal data $X_{Norm}$ to get $X^{\prime }$
18			Train the track prediction network $\varphi$ using the new sample pairs $X^{\prime }$, as described in Section 3.6
19	end

Algorithm 2: The testing pseudocode of Node-former
	Input: Jamming radar testing dataset $\mathcal{D}$, jamming filtering network $\beta$, anti-jamming network $\theta$, track prediction network $\varphi$, filtering threshold $\alpha$
1	Sample a test sample $X_{test}$ from $\mathcal{D}$
2	Divide the sample X into two parts: $X_{Jam}$ and $X_{Norm}$
3	// Jamming Data Filtering Testing Phase
4	Apply the trained filtering network $\beta$ to the divided part $X_{Jam}$
5	Following the filtering threshold $\alpha$ to select $X_{Jam}$ and output the irregular data $X_{Jam}^{filter}$
6	// Anti-jamming Testing Phase
7	Apply the trained anti-jamming network $\theta$ to the irregular input $X_{Jam}^{filter}$
8	Output the corrected sample $X_{Cor}$ and contact with normal part $X_{Norm}$ to obtain final input $X^{\prime }$
9	// Track Prediction Testing Phase
10	Apply the trained track prediction network $\varphi$ to the final input $X^{\prime }$
11	Output the subsequent predicted part $X_{pred}$

4. Presentation of Results

This section describes in detail the design of the simulation experiment, the logic of the program operation, and the comparison of the experimental results. Here, the aircraft’s radar was used instead of the airport ground radar, and the jamming data were created by jamming the radar function through the jammer.

4.1. Simulation Design

Firstly, we tested the robustness of the algorithm in terms of its anti-jamming capabilities while tracking the target, as given equation by Equation (22) [34,35,36]:

$${\mathit{x}}_{t+dt}=\left(\begin{array}{ccccc}1& {\displaystyle \frac{sin({\omega }_t\Delta t)}{{\omega }_t}}& 0& {\displaystyle \frac{cos({\omega }_t\Delta t)-1}{{\omega }_t}}& 0\\ 0& cos({\omega }_t\Delta t)& 0& -sin({\omega }_t\Delta t)& 0\\ 0& {\displaystyle \frac{1-cos({\omega }_t\Delta t)}{{\omega }_t}}& 1& {\displaystyle \frac{sin({\omega }_t\Delta t)}{{\omega }_t}}& 0\\ 0& sin({\omega }_t\Delta t)& 0& cos({\omega }_t\Delta t)& 0\\ 0& 0& 0& 0& 1\end{array}\right){\mathit{x}}_t+{\mathit{u}}_t$$

(22)

where ${\mathit{x}}_t$ represents the vector that encompasses position, velocity, and rotation rate. ${\mathit{u}}_t$ represents the process noise.

The state vector ${\mathit{x}}_t\stackrel{\Delta }{=}[a_t,{\dot{a}}_t,b_t,{\dot{b}}_t,{\omega }_t]$ contains the position $(a_t,b_t)$, velocity $({\dot{a}}_t,{\dot{b}}_t)$, and rotation rate ${\omega }_t$ of the tracked target. The initial state of the tracked target was $x_0\stackrel{\Delta }{=}[-\mathrm{10,000}\mathrm{m},100\mathrm{m}/\mathrm{s},-\mathrm{50,000}\mathrm{m},-300\mathrm{m}/\mathrm{s},-0.0053\mathrm{rad}/\mathrm{s}]$. The measurement equation is given by Equation (23):

$${\mathit{y}}_{t+dt}=\left(\begin{array}{c}{r}_{t+dt}\\ {\theta }_{t+dt}\end{array}\right)=\left(\begin{array}{c}\sqrt{a_{t+dt}^2+b_{t+dt}^2}\\ \mathrm{atan}(b_{t+dt},a_{t+dt})\end{array}\right)+{\mathit{v}}_{t+dt}$$

(23)

${\mathit{y}}_t$ represents the vector that encompasses $r_{t+dt}$ and ${\theta }_{t+dt}$, where $r_{t+dt}$ and ${\theta }_{t+dt}$, respectively, represent the slant distance and azimuth angle of the target. ${\mathit{v}}_{dt}$ represents the measurement noise.

The state equation generates a trajectory of motion, and we conducted a comparative analysis of the Node-former algorithm and the Kalman filter algorithm by manipulating the proportion of outliers in the observed data within the measurement equation. This comparison aimed to quantitatively evaluate the performance differences between the two algorithms in handling data with uncertainties and noise.

Next, we subjected the algorithm to a more complex environment and chose a simulated air combat game. The data collected were generated by the DCS (Digital Combat Simulator) world, and the aircraft simulation model consisted of the six-degree-of-freedom motion model and the radar model. We modeled jamming data through using a jammer to interfere with the radar.

In the simulation program, the engagement area and preparation area of both sides were set, and the initial slant distance of both sides of the aircraft in each field was 20 km. The initial altitude was 500 m, and the aircraft was generated at any position in its respective preparation area. The red aircraft’s radar was used as an airport monitoring radar, and the blue aircraft was a drone that was about to break into the no-fly zone.

The red aircraft formation is tasked with driving away the blue aircraft formation. Figure 10 illustrates the approaching of both formations. Figure 11 showcases an operation where the red formation successfully drives away the blue formation. In Figure 11a, the red formation’s radar tracks the blue formation and launches missiles, prompting the blue formation to initiate evasive maneuvers. In Figure 11b, the blue formation activates ECMs (Electronic Countermeasures) to prevent being hit. In Figure 11c, one blue aircraft is shot down, while the other is successfully driven away from the no-fly zone in Figure 11d. Throughout this process, to maximize the number of target tracks, any blue aircraft entering the Field of View (FoV) of any red aircraft’s radar is detected. Figure 12 depicts a radar interface, where, upon encountering interference, the displayed distance may experience fluctuations, and the affected targets are highlighted in yellow as sources of interference.

The specific process shown in Figure 10, Figure 11 and Figure 12 shows the radar interface.

The simulation used the north–east–down (NED) coordinates. The data consisted of the target track measurement output from the radar and the real track of the target according to the NED calculated from the latitude, longitude, and altitude data recorded by the respective INS of the aircraft and target. For the target data $Rd{r}_{Rang{e}_i}$, measured by the radar in the current moment, i can be expressed as the equation

$$Rdr{m}_i=\left(Rd{r_{Range}}_i,Rd{r_{El}}_i,Rd{r_{Az}}_i,Rd{r_{Vclose}}_i,Rd{r_{Vr}}_i,Jam\_Flag\right)$$

(24)

where $Rd{r}_{Rang{e}_i}$ represents the current moment radar measurement slant distance; $Rd{r}_{A{z}_i}$ represents the current moment’s radar measurement of the target pitch angle under NED coordinates; $Rd{r}_{A{z}_i}$ represents the current moment’s radar measurement of the target azimuth angle under NED coordinates; $Rd{r}_{Vclos{e}_i}$ represents the current moment’s radar measurement close velocity along the Line of Sight (LOS); $Rd{r}_{Rang{e}_i}$ represents the current moment’s radar measurement of the target radial velocity along the LOS; and $Jam\_Flag$ represents the moment that the radar suffers jamming.

The navigation fusion information reported by the Ins can be expressed as $In{s}_i$:

$$In{s}_i=\left(My\_Rol{l}_i,My\_Pitc{h}_i,My\_Vel\_Lo{s}_i\right)$$

(25)

where $My\_Rol{l}_i$ represents the current moment’s roll angle of the measurement aircraft; $My\_Pitc{h}_i$ represents the current moment’s pitch angle of the measurement aircraft; and $My\_Vel\_Lo{s}_i$ represents the current moment’s measurement of the aircraft velocity along the LOS.

The target true location information can be expressed as $Tg{t}_i$:

$$Tg{t}_i=\left(Rang{e}_i,E{l}_i,A{z}_i\right)$$

(26)

where $Rang{e}_i$ represents the current moment’s true slant distance of both sides; $E{l}_i$ represents the current moment’s true elevation angle; and $A{z}_i$ represents the current moment’s true azimuth angle.

4.2. Mission Flow

In this study, the task was broken into two objectives to be accomplished. The algorithm flow is shown in Figure 13.

The first step was data input and pre-processing. The data in the radar were used as input for the pre-processing and were separated by judging the radar jamming flag.

The second step involved judging when the target state was steadily tracked by the radar. The jammed data were passed through the error judgment evaluation module, and the data output when the radar anti-jamming strategy was judged to be effective was selected, so that the data with smaller error values were sampled in the jamming data.

In the third step, the jamming data sampled with the judgment module were input into the module of error correction to form the corrected radar target data, which were stitched together with the normal, non-jammed radar measurement data to form the target data. Here, the judgment was made whether the target was lost at the current moment. If the target was still in the tracking state, the target data were sent directly to the user interface for display, and, if the target was lost, the next step was taken.

In the fourth step, the target data for some time before the target was lost were sent into the track prediction module to predict the target track for the next period, and the predicted target information was sent to the user interface for display.

4.3. Simulation Experiments

All the experiments were conducted in an environment configured as shown in

Table 1. Environment configuration.

Item	Parameter
CPU	Intel Xeon Gold 6248R @3 GHz
GPU	NVIDIA Quadro RTX 4000
RAM	512 GB

.

Initially, the experiment was conducted to generate a motion trajectory using Equation (22). Following this, the trajectory was observed using Equation (23), with a step size of 500. The acquired measurement data were then subjected to a thorough analysis using three distinct algorithms: the Node-former algorithm, the Kalman algorithm, and the Robust-Kalman algorithm. This multi-algorithm approach aimed at providing a comprehensive trajectory analysis. The experimental workflow, incorporating these three algorithms, is visually depicted in Figure 13.

In this experiment, we replaced the perturbed data with outliers. Figure 14 and Figure 15 demonstrates the effectiveness of the three algorithms under different ratios of outliers.

As can be seen from the trajectory and measurement plots in Figure 14a, the performance of the three algorithms was basically the same when there were no outliers and only noise.

As the ratio of outliers increased, as illustrated in Figure 15, we observed a gradual decline in the accuracy of the measured distance and angle. In this context, the Kalman filter algorithm faced challenges in efficiently recognizing and accommodating these outlier points, leading to less-than-optimal tracking performance. Similarly, while the Robust-Kalman algorithm demonstrated an improved ability to handle a higher number of outliers compared to the traditional Kalman algorithm, it too exhibited limitations as the outlier percentage escalated. Ultimately, when the outlier ratio reached 70%, the Robust-Kalman algorithm struggled to maintain effective functionality.

In comparison, our Node-former algorithm showed promising robustness, indicating that its anti-interference capabilities were functioning well. Although we recognize that further improvements are always possible, the Node-former algorithm consistently demonstrated a stronger capacity to process trajectory data amidst high outlier levels, compared to both the Kalman and Robust-Kalman algorithms. This finding underscores the potential of our algorithm in deal with complex and noisy datasets.

Subsequently, we implemented the Node-former algorithm in a more intricate environment and undertook a comparative analysis with several prevalent deep learning algorithms. Through the DCS game, 3000 data sets were generated. This study used 80% of the data as the training set and 20% as the test set.

The simulation set the time interval of each radar measurement target information report to 10 ms. The period of the target data reported by the radar was 0.1 s, the flight altitude was from 100 m to 1500 m, and the maximum speed was 140 km/h.

During training, the batch size was set to 50, the learning rate to $1\times {10}^{-4}$, the weight decay to $1\times {10}^{-5}$, the input size to 13, and the output size to 3, according to Equations (18)–(20), respectively, for the range, the azimuth angle, and the pitch angle of the target.

The hidden dim of the LSTM algorithm was set to 256, the number of layers was set to 2, and the LSTM dims were set to 256. The embedding dim of the transformer algorithm was set to 128, the number of multi-heads was set to 8, and the number of encoder layers was set to 12.

The screening threshold of the jamming data filter module in the Node-former algorithm was set to ${0.5}^{\circ }$, the embedding dim of the Node-former was set to 128, the number of multi-heads was set to 8, and the number of encoder layers was set to 12.

After training, the measurement data from the radar in the test set were fed into the algorithm to obtain the results. In this study, the anti-jamming and prediction capabilities of the network were evaluated separately. There were some charts of errors from the test set. Figure 16 and Figure 17 show the errors of the anti-jamming phase and the error curve of the prediction phase in some scenarios from the test set, respectively.

The results of the LSTM output show that, although LSTM could suppress jamming based on the experience obtained from training, the long-time large error data input still slowly affected the output results. The transformer was less sensitive to jamming data than the LSTM, but its output data values varied more per frame, which were less stable compared to the LSTM. The Node-former algorithm had the advantages of both: the accuracy of the output data was more insensitive to the jamming data and the error curve was smooth.

As can be seen from Figure 17, for long sequences of prediction, radar extrapolation could not accurately predict the location of the target. LSTM also began to gradually diverge with the increase in the number of time steps. Case results for the transformer algorithm were better compared to LSTM, but its output error fluctuations were still large. In engineering applications, data fluctuations can be detrimental to the operator. The prediction results of the Node-former algorithm had better convergence, while the output data curves were smooth and conformed to the target motion pattern.

The LSTM, transformer, and Node-former algorithms were tested separately. The experimental results can be found in

Table 2. RMSE of algorithms.

Model	Anti-Jamming			Track Prediction
	RMSE (Az)	RMSE (El)	RMSE (Range)	RMSE (Az)	RMSE (El)	RMSE (Range)
LSTM	0.188°	0.036°	$174.56$ m	0.361°	0.105°	$145.92$ m
Transformer	0.180°	0.038°	$236.46$ m	0.215°	0.070°	$218.39$ m
Node-former	0.055°	0.015°	$69.13$ m	0.140°	0.059°	$95.85$ m

.

Table 2 shows that LSTM was better at processing the target distance information and the transformer algorithm was better at processing the target angle information. The Node-former architecture was better at processing both the target range and the azimuth pitch angle. The Node-former range RMSE was better than that for LSTM, and the azimuth and pitch angle RMSEs were better than those for the transformer architecture, proving that the algorithm architecture has advantages in processing such data.

In summary, Node-former improved the azimuth RMSE to ${0.555}^{\circ }$, pitch RMSE to ${0.015}^{\circ }$, and distance RMSE to $69.13$ m in the anti-jamming phase. In the prediction phase, Node-former improved the azimuth RMSE to ${0.140}^{\circ }$, pitch RMSE to ${0.059}^{\circ }$, and distance RMSE to $95.85$ m.

Figure 18 shows the comparison of the track generated after inputting the Node-former prediction data into the simulation with the real track of the target. It can be seen from the figure that the track output by the algorithm structure matched the real track and can therefore satisfy the requirement for early removal of a target intending to enter a no-fly zone.

5. Conclusions

Predicting the trajectories of UAVs that may enter no-fly zones can be challenging. This study explored the problem of target track prediction in target loss scenarios when the radar is subjected to jamming. The main conclusions are as follows:

(1) Experiments have proven that Node-former exhibits stronger anti-jamming performance compared to conventional Kalman filter, Robust Kalman filter, LSTM, and Transformer algorithms, regardless of whether it is applied to 2D or 3D trajectory tracking. It employs a novel approach that involves interpolating irregularly sampled data after filtering out the tracking error predictions derived from jamming data through the Nodes algorithm;

(2) The RNN algorithm over-allocates weights to feature dimensions of a large order of magnitude based on the loss function, while the transformer algorithm is more balanced in assigning weights. The RNN output is smooth in the track information, while the transformer output drastically changes the track information. Node-former takes into account the advantages of both, as it inherits the advantages of the transformer algorithm in assigning weights. Additionally, to avoid the track points from jumping during track prediction, the decoder in Node-former uses an ODE solver to make the output data more stable and continuous;

(3) This article provides a three-stage methodology to solve the problem of target trajectory prediction based on interference data scenarios. It can replace each unit module in the form of components while maintaining the architecture in an unchanged manner. This provides room for improvement in adopting cutting-edge algorithms and is easy to implement in engineering applications.

We acknowledge that our proposed approach, while effective in addressing the challenges of anti-jamming and trajectory prediction for UAVs, does have certain limitations. One primary limitation lies in its relatively high computational complexity compared to traditional methods like Kalman filters, which can translate into increased resource consumption. To mitigate this issue and make our approach more efficient, we plan to explore lighter versions of our algorithm in future work. Specifically, we aim to incorporate sparse or frequency-based attention mechanisms, which have shown promise in optimizing performance while reducing computational demands. These mechanisms could contribute to a more resource-conscious implementation, enhancing the practicality of our method. Furthermore, we recognize that our method may encounter difficulties in generalizing to scenarios that it has not been explicitly trained on, indicating a limitation in its generalization ability. To improve this aspect, we plan to incorporate a wider variety of baseline algorithms into our future work. By comparing our approach against these diverse baselines, we can gain a more comprehensive understanding of its strengths and weaknesses and identify potential areas for improvement. Additionally, we intend to test our algorithm in more complex environments, exposing it to a broader range of conditions and challenges. This rigorous testing will help us evaluate the robustness of our method and identify strategies to further enhance its generalization capabilities. By incorporating these enhancements, we aim to make our approach more versatile and reliable, i.e., better suited for real-world applications.