STFTrack: Spatio-Temporal-Focused Siamese Network for Infrared UAV Tracking

Abstract

The rapid popularity of UAVs has encouraged the development of Anti-UAV technology. Infrared-detector-based visual tracking for UAVs provides an encouraging solution for Anti-UAVs. However, it still faces the problem of tracking instability caused by environmental thermal crossover and similar distractors. To address these issues, we propose a spatio-temporal-focused Siamese network for infrared UAV tracking, called STFTrack. This method employs a two-level target focusing strategy from global to local. First, a feature pyramid-based Siamese backbone is constructed to enhance the feature expression of infrared UAVs through cross-scale feature fusion. By combining template and motion features, we guide prior anchor boxes towards the suspicious region to enable adaptive search region selection, thus effectively suppressing background interference and generating high-quality candidates. Furthermore, we propose an instance-discriminative RCNN based on metric learning to focus on the target UAV among candidates. By measuring calculating the feature distance between the candidates and the template, it assists in discriminating the optimal target from the candidates, thus improving the discrimination of the proposed method to infrared UAV. Extensive experiments on the Anti-UAV dataset demonstrate that the proposed method achieves outstanding performance for infrared tracking, with 91.2% precision, 66.6% success rate, and 67.7% average overlap accuracy, and it exceeded the baseline algorithm by 2.3%, 2.7%, and 3.5%, respectively. The attribute-based evaluation demonstrates that the proposed method achieves robust tracking effects on challenging scenes such as fast motion, thermal crossover, and similar distractors. Evaluation on the LSOTB-TIR dataset shows that the proposed method reaches a precision of 77.2% and a success rate of 63.4%, outperforming other advanced trackers.

1. Introduction

With the rapid spread of UAV technology [1,2], the risk of UAV abuse has significantly increased, posing a considerable threat to public safety [3,4]. Effectively countering non-cooperative UAVs has become an urgent social security problem.

As a critical part of the Anti-UAV system, accurately tracking non-cooperative UAVs forms the basis for further countermeasures. In recent years, infrared-detector-based visual tracking has become an increasingly popular solution for Anti-UAV missions thanks to its numerous advantages, including its compact size, good visual quality, and round-the-clock operation. Thus, infrared UAV tracking has served as a hot topic in the Anti-UAV field [5].

Given the initialized bounding box of the UAV in the first frame, the purpose of infrared UAV tracking is to automatically locate the interested UAV in successive frames. Compared with natural object tracking, infrared UAV tracking poses three distinct challenges: (1) the small-scale and sparse texture features of infrared UAVs make them easily susceptible to environmental disturbances; (2) fast-moving and frequent out-of-view attributes of infrared UAVs often result in tracking failures; (3) environmental thermal crossover and distractors make it difficult to distinguish UAVs from other aerial objects in infrared images. Given these challenges, designing a robust and efficient tracking method for infrared UAVs remains a great challenge.

Recently, visual object tracking has undergone rapid development, especially for the Siamese-based tracker, which has achieved many remarkable outcomes. For instance, SiamFC [6] reformulated the visual tracking task as a template matching problem, providing a new paradigm for subsequent Siamese-based trackers. SiamRPN [7] introduced the RPN module into visual tracking, realizing accurate location and precise box regression. Since then, many improvements have been made regarding online updating [8,9], robust feature extraction [10,11,12], and similar distractor suppression [13,14,15]. Despite these developments, the aforementioned algorithms, known as short-term tracking methods, are based on a local smoothing assumption, which limit the search range to a fixed neighborhood around the target center in the previous frame. When the UAV suddenly moves or becomes out-of-view, as is common in infrared UAV tracking, this assumption will result in error accumulation and even tracking failures.

Long-term tracking methods exhibit global search capabilities to recapture the lost target, making them well-suited for infrared UAV tracking. Existing long-term tracking algorithms can be divided into two types: combined tracking methods and global tracking methods. The former, represented by TLD [16], SPLT [17], and LTMU [18], mainly consists of a short-term tracker and a global detector. In case the local tracker fails, the global detector is activated to relocate the lost target, achieving a local-to-global search pipeline. Some work has aimed to accomplish infrared UAV tracking by enhancing such algorithms. For instance, Zhao et al. [19] proposed a unified infrared UAV tracking framework, combining a local tracker, camera motion estimation module, bounding box refinement module, re-detection module and model updater. Despite performance gains, this algorithm entails multiple independent modules and a loose overall structure, hindering optimal parameter tuning. Moreover, these combined tracking methods struggle to accurately determine when to switch, given weak semantic features and environmental interference in complex infrared scenes. The latter, represented by GlobalTrack [20] and SiamRCNN [21], employs Siamese-based global search tracking, foregoing local search and tracking targets across the entire frame to avoid drift from unreliable switching. Owing to their compact structures and end-to-end optimization, global tracking methods are gaining popularity in infrared UAV tracking. Fang H. et al. [22] designed a YOLO-based global tracker called SiamYOLO, achieving efficient tracking for infrared UAVs. Chen J. et al. [23] incorporated a local tracking strategy based on spatio-temporal attention into the SiamRCNN and realized an adaptive switching between local tracking and global detection, leading to robust tracking for infrared UAVs. Shi X. et al. [24] developed a graph attention-based Siamese tracker for infrared UAV based on SiamRCNN, using a Lucas-Kanade optical flow to select either local tracking or global re-detection. To address the challenge of infrared UAV tracking, the above tracking methods have improved the global tracking method by adopting different strategies, enhancing the performance of infrared UAV tracking. However, these strategies are post-processing approaches for tracking results, without improving the feature expression of infrared UAV targets at the feature level. Therefore, there still exist issues of unstable tracking in complex environments with thermal crossover and similar distractors.

We contend that two critical factors contribute to the above phenomenon. One is the fixed search region. Neither local search nor global search is optimal for infrared UAV tracking. Local search facilitates search accuracy by focusing on a small region, but at the cost of reduced recall due to the limited search range. Global search is conducive to search recall by recapturing lost targets, but it also introduces more background interference, making infrared UAVs fall into the chaotic background, and leading to decreased accuracy. Consequently, it is favorable to incorporate the local constraints with a global framework for more robust tracking. The other is the weak discrimination of target features. The lack of color and texture of infrared UAVs makes it challenging to describe the difference between targets and distractors. Moreover, the expansion of a search region will lead to a large increase in negative samples. Excessive simple negative samples interfere with the discriminative ability of features, thus making it more difficult to distinguish the target from hard negative samples, i.e., similar distractors. Therefore, it is urgent to strengthen the discriminability of target features to accurately identify the target among confusing candidates.

Motivated by the above analysis, we propose a novel spatio-temporal-focused Siamese network for infrared UAV tracking called STFTrack. Different from the aforementioned methods, STFTrack realizes an adaptive target search region mechanism, effectively reducing the interference from background negative samples. Additionally, STFTrack employs a two-stage tracking framework, implementing a two-stage target-focusing mechanism by incorporating instance-level feature metric learning to enhance the discrimination of hard negative samples. Specifically, to achieve search region focusing, we propose a spatio-temporal information-guided region proposal network (STG-RPN). Under the guidance of template target features and motion cues, STG-RPN predicts a probability distribution of the target location over the entire image. This distribution further guides prior anchors to focus on candidate regions, enabling a soft adaptive search region selection mechanism. With this focused search region, interference from negative samples due to an expanded search region is alleviated, enhancing the discriminability of target features and generating high-quality candidate targets. To further focus on the infrared UAV among the candidate targets, we design an instance discrimination region-CNN (ID-RCNN). Using metric learning, the target characteristics are comprehensively described from the perspective of classification and measurement, which further expands the feature difference between the target and similar distractors. Experiments on the Anti-UAV dataset and LSOTB-TIR demonstrate that the proposed method can effectively handle interference caused by similar distractors and thermal crossover, thereby enabling the tracking of infrared UAVs in challenging scenarios.

In summary, this paper makes the following contributions:

(1)

We propose a spatio-temporal-focused Siamese network for infrared UAV tracking, which incorporates spatio-temporal constraints within a Siamese-based global search tracking framework, employing a two-level global-to-local focusing strategy.

(2)

To achieve search region focusing, a spatio-temporal information-guided region proposal network is proposed, which implements an adaptive region selection strategy to generate high-quality candidate targets.

(3)

To improve the accuracy of the target discrimination, we propose an instance discrimination region-CNN, which specifically focuses on the target among the candidate targets. Through metric learning, we further enhance the discrimination of the target features.

2. Proposed Method

The proposed method is a two-stage tracking framework, including the candidate target generation stage and the instance discrimination stage. The overall framework, as shown in Figure 1, consists of a Siamese feature extraction network, a STG-RPN, and an ID-RCNN. The Siamese feature extraction network is used to extract deep features from template images and search images. The STG-RPN generates candidate targets that are spatio-temporally similar to the template target. The ID-RCNN further identifies the UAV among the proposed candidates.

2.1. Deep Feature Extraction Based on the Siamese Network

Due to factors such as the small scale, lack of color, and inconspicuous texture of infrared UAVs, combining semantic and location characteristics within single-scale features is challenging. One commonly employed approach to address this issue is fusing low-level features and high-level features. For this, we construct a feature pyramid-based Siamese network, which performs a multi-scale feature fusion to obtain more robust deep features. The network comprises a template branch and a search branch, with shared weights in both branches. Given a pair of infrared images, i.e., the template image ${\mathit{I}}_Z$ and the search image ${\mathit{I}}_x$, a weight-shared backbone $\phi$ is first used to extract respective feature maps $\mathit{z}=\phi \left({\mathit{I}}_Z\right)$ and $\mathit{x}=\phi \left({\mathit{I}}_x\right)$. Then, we adopt the FPN [25], denoted as $\varphi$, to fuse the multi-level features from the backbone, obtaining multi-scale template image features $\mathit{Z}=\{{\mathit{Z}}_i\left|i=\right.1,2,3\}=\varphi \left(\mathit{z}\right)$ and search image features $\mathit{X}=$$\{{\mathit{X}}_i\left|i=\right.1,2,3\}=\varphi \left(\mathit{x}\right)$ from three levels (P3, P4, and P5). In addition, we extract multi-scale template target features ${\mathit{F}}_i$ from the template image features ${\mathit{Z}}_i$ using

$${\mathit{F}}_i=\psi \left({\mathit{Z}}_i,{\mathit{B}}_1\right),$$

(1)

where $\psi$ represents the ROIAlign operation and ${\mathit{B}}_1$ is the initial bounding box. By virtue of the weight-sharing feature, the output features of the two branches are semantically consistent, facilitating the identification of common objects in both images.

2.2. Candidate Target Generation Based on STG-RPN

During infrared UAV tracking, the target search region plays a critical role in the tracking performance. The traditional local search can be viewed as a hard motion-constrained selection strategy that conducts a search only in a limited area and ignores the remaining area information. While effective at mitigating background interference, it struggles to cope with the rapid movement and out-of-view attributes of UAVs. Conversely, the global search employs a uniform selection strategy that searches the entire frame without motion assumptions. While this strategy is effective in recovering lost UAVs, it suffers from more background interference, leading to tracking instability.

To address this issue, we introduce local constraints into the global search frame- work and propose a spatio-temporal information-guided RPN (STG-RPN). Under the guidance of spatio-temporal information, target search region focusing is achieved in a soft-selection way, which further alleviates the sample imbalance. The overview of STG-RPN is depicted in Figure 2, and mainly includes three phases: target location prediction, anchor shape prediction, and feature adaptation.

2.2.1. Target Location Prediction

Target tracking aims to search for the specific object that matches the appearance of the template target and its previous motion pattern. To achieve this, we use time–space cues to locate infrared UAV in three stages: spatial location prediction, temporal location prediction, and spatio-temporal fusion prediction.

From a spatial perspective, the target infrared UAV is often found in regions that share similarities with the template target. Thus, spatial location prediction aims to identify these similar template target locations in the search image and generate a location probability map. Given the template target features ${\mathit{F}}_t\in {ℝ}^{k\times k\times c}$ and search image features ${\mathit{F}}_s\in {ℝ}^{w\times h\times c}$, the spatial location probability map ${\mathit{F}}_{sl}\in {ℝ}^{w\times h\times 1}$ is formulated as

$${\mathit{F}}_{sl}=\sigma \left(f_c\left({\mathit{F}}_t\otimes {\mathit{F}}_s\right)\right),$$

(2)

where $\otimes$ denotes the deep-wise cross-correlation operator [10], which is used to encode the template target information into the search image features. $f_c$ indicates the channel aggregation function, which is defined as a 1 × 1 convolution that converts the channel number from c to 1. $\sigma$ represents the Sigmoid function that compresses the output range to $\left[0,1\right]$.

From a temporal perspective, the ideal motion of the infrared UAV may be a continuous process. However, due to the camera’s jitter and the shooting interval, the UAV’s motion state can be classified into two categories: smooth motion and sudden motion. In the case of smooth motion, the motion state before and after moments is highly correlated, allowing for the use of a motion model to predict the current position of the UAV based on its previous state. Conversely, if the UAV experiences sudden motion changes, the prediction becomes unreliable, and it is likely that the UAV may appear at any location in the search image.

In this study, the Kalman filter framework is adopted to estimate the motion state of the infrared UAV. We formulate the target motion state as a four-dimensional vector $\mathit{X}=\left[x_c,y_c,{\dot{x}}_c,{\dot{y}}_c\right]$, where $\left(x_c,y_c\right)$ represent the coordinates of the target center point and $\left({\dot{x}}_c,{\dot{y}}_c\right)$ denotes the speed of the center point. Thus, the target position in the next frame can be predicted as

$${\mathit{X}}_t=\mathit{A}{\mathit{X}}_{t-1}+\mathit{B}{\mathit{U}}_t,$$

(3)

where $\mathit{A}$ is the state transfer matrix corresponding to the target motion model, $\mathit{B}$ is the control parameter, and ${\mathit{U}}_t$ is the control amount at time t (zero in this study). Here, we approximate the inter-frame displacement of the UAV as a uniform motion model. Accordingly, the observational model is expressed as ${\mathit{Z}}_t=\mathit{H}{\mathit{X}}_t+{\mathit{v}}_t$, where $\mathit{H}$ is the observation matrix and ${\mathit{v}}_t$ is the observation noise with covariance $\mathit{R}$ (assumed to be Gaussian). The covariance of prediction error is ${\tilde{\mathit{P}}}_t=\mathit{F}{\mathit{P}}_{t-1}{\mathit{F}}^{\mathit{T}}+\mathit{Q}$, where ${\mathit{P}}_{t-1}$ is the Kalman estimation error covariance and $\mathit{Q}$ is the process noise covariance.

In order to determine the motion state of the UAV, a maneuver detection method based on pseudo-innovation is adopted. In the Kalman filtering framework, the innovation refers to the difference between the predicted observation and the true observation. In our case, the highest peak position of the spatial location map ${\mathit{F}}_{sl}$ is selected as the estimate of observation. Thus, the pseudo-innovation can be formulated as ${\mathit{n}}_t=\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{a}\mathrm{x}({\mathit{F}}_{sl})-{\tilde{\mathit{Z}}}_t$, where ${\tilde{\mathit{Z}}}_t$ denotes the predicted position. Based on ${\mathit{n}}_t$, we define a novel statistical variable, namely using normalized innovation squared (NIS), as the indicator for maneuver detection. NIS can be calculated as

$${\mathit{\epsilon }}_t={\mathit{n}}_t^{\mathrm{T}}{\mathit{S}}_t^{-1}{\mathit{n}}_t,$$

(4)

where ${\mathit{S}}_t$ indicates the pseudo-innovation covariance and ${\mathit{S}}_t=\mathit{H}{\tilde{\mathit{P}}}_t{\mathit{H}}^{\mathrm{T}}+\mathit{R}$. Since ${\mathit{\epsilon }}_t$ follows a chi-square distribution, we set the maneuver detection threshold as

$$P\left\{{\mathit{\epsilon }}_t\le {\mathit{\epsilon }}_{\mathrm{m}\mathrm{a}\mathrm{x}}\right\}=1-\eta ,$$

(5)

where $\eta$ denotes the confidence score, we set $\eta =0.05$. In our case, ${\mathsf{\epsilon }}_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the chi-square critical value corresponding to $\eta$; if ${\mathit{\epsilon }}_t>{\mathit{\epsilon }}_{\max }$, then the UAV is considered to have maneuvered.

When the target moves smoothly, we Gaussianize the position of the motion prediction ${\tilde{\mathit{Z}}}_t$ based on the prediction error covariance ${\tilde{\mathit{P}}}_t$ and obtain the temporal location prediction map ${\mathit{F}}_{tl}\in {ℝ}^{w\times h\times 1}$. However, when the UAV motion suddenly changes, we set ${\mathit{F}}_{tl}=\mathit{I}$. Therefore, the temporal location prediction map ${\mathit{F}}_{tl}=\left\{{\mathit{f}}_{tl}^i\left|i\in (w,h)\right.\right\}$ can be calculated as

$${\mathit{f}}_{tl}^i=\left\{\begin{array}{l}\mathrm{e}\mathrm{x}\mathrm{p}\left(-{‖{\mathit{m}}_i-{\tilde{\mathit{Z}}}_t‖}^2/\alpha p_t^{}\right),\mathrm{i}\mathrm{f}\mathrm{m}\mathrm{o}\mathrm{v}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{s}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}\hfill \\ 1,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\hfill \end{array}\right.,$$

(6)

where ${\mathit{m}}_i$ is the coordinate of the i-th point in ${\mathit{F}}_{tl}$, ${‖\cdot ‖}^2$ represents the Euclidean distance, $p_t$ is the variance of the position prediction error, $\alpha$ denotes the tradeoff hyperparameters, and $\alpha =5$ in our case. After obtaining the spatial and temporal location maps, we further utilize element-wise multiplication to fuse them. Thus, the target location map is calculated as

$${\mathit{F}}_l={\mathit{F}}_{sl}\odot {\mathit{F}}_{tl},$$

(7)

where $\odot$ denotes the Hadamard production.

Remark 1: 

When the infrared UAV moves smoothly, the regions far from the predicted position in the spatial location map are suppressed incrementally by the temporal location map, which helps to reduce interference from similar objects. While the UAV motion suddenly changes, the temporal location map transforms into a unit matrix. In this case, the target location probability map degenerates into a spatial location map, serving as a preventive measure against inaccurate motion predictions.

Subsequently, a probability threshold is applied to binarize the target location probability map. We select the high-confidence regions to establish anchor points, thus achieving the region focus at the anchor level. Figure 3 illustrates an example of the target location prediction process. We can find that a focused search region for setting anchor points is extracted under the guidance of spatio-temporal information.

2.2.2. Anchor Shape Prediction

Following the prediction of the anchor location, the optimal anchor shape is estimated to obtain the anchor box that is most similar to the ground truth. Given the encoded features ${\mathit{F}}_m\in {ℝ}^{w\times h\times c}$, we employ a nonlinear transformation to map ${\mathit{F}}_m$ into the shape prediction maps ${{\mathit{F}}^{\prime }}_{sh}\in {ℝ}^{w\times h\times 2}$, whose each channel represents the length and width of the anchor box. Here, we model the nonlinear transformation using a 1 × 1 convolution layer with two output channels and a Leak ReLU activation function.

To improve the stability of our prediction process, we adopt the deflation transformation, as described in [26], to convert the length and width of the anchor box into $dh$ and $dw$. The deflation transformation is formulated as

$$w=\tau s{e}^{dw},h=\tau s{e}^{dh},$$

(8)

where $s$ is the step size of the feature map and $\tau$ is the scale factor ($\tau =8$ in this study). Combining the anchor point and anchor shape, we can derive preset anchor boxes that are focused on the target.

2.2.3. Feature Adaptation

Infrared UAVs exhibit variable scales, affecting the shape prediction of anchor boxes at different locations. As a result, it leads to a feature mismatch problem, which is detrimental to tracking accuracy. To tackle this problem, we implement the feature adaptation module proposed in [26], which enhances the adaptability of features to different shapes of anchor boxes. First, we employ a 1 × 1 convolution layer to calculate the offset field according to the anchor shape. Then, a deformable convolution layer using this offset is applied to adjust the original features. The feature adaptation process is expressed as

$${\mathit{F}}_a^{}={\mathcal{N}}_T\left({\mathit{F}}_m^{},w,h\right)$$

(9)

where ${\mathit{F}}_a^{}$ is the adapted features, ${\mathit{F}}_m$ is the encoded feature, and $\left(w,h\right)$ denotes the anchor shape. ${\mathcal{N}}_T$ represents the feature adaptation module which is implemented by a 3 × 3 deformable convolution layer.

We use the adapted features to perform classification and regression on predefined anchor boxes, and to obtain high-quality candidate targets. The structure of the classification head and regression head is consistent with traditional RPN [27], except that we use masked convolution [28] instead of traditional convolution. During tracking, the masked convolution applies the anchor region as a filter, confining the feature computation within the region of interest while ignoring irrelevant ones. This leads to a significant increase in the computational efficiency compared to classical point-by-point calculation.

2.3. Target Discrimination Based on ID-RCNN

Extracting discriminative features from infrared images is a challenging task for neural networks due to the weakness of color and texture information. Therefore, it is difficult to differentiate infrared UAVs from other similar targets. In particular, when the infrared UAV is caught in background noise, the candidates generated by STG-RPN may incorporate many similar distractors. These interferers will seriously affect the determination of tracking results, thus causing tracking instability.

Based on the QG-RCNN [20], we design the ID-RCNN module to handle the above challenge. Our ID-RCNN incorporates deep metric learning (DML) to enhance the discriminability of target descriptions, allowing for better focus on the desired target within candidates. The structure of the ID-RCNN is illustrated in Figure 4, mainly consisting of two branches, i.e., the QG-RCNN and DML. The QG-RCNN branch is responsible for the classification and regression of candidate proposal, while the DML branch is leveraged to the determination further of confusable candidates.

2.3.1. QG-RCNN Branch

The QG-RCNN consists of two stages: feature modulation and the RCNN. Given the template target feature ${\mathit{F}}_t\in {ℝ}^{k\times k\times c}$ and the j-th candidate feature ${\mathit{F}}_p^j\in {ℝ}^{k\times k\times c}$, the feature modulation can be expressed as

$${\tilde{\mathit{F}}}_p^j=h_{out}\left(h_t\left({\mathit{F}}_t\right)\odot h_p\left({\mathit{F}}_p^j\right)\right),$$

(10)

where $\odot$ denotes the Hadamard production, $h_t$ and $h_p$ are the feature projection functions realized by a 3 × 3 convolution with one-pixel padding, and $h_{out}$ is the output projection function that retains the same size and channels as ${\mathit{F}}_p^j$. Following this, the classical RCNN is adopted to predict the classification score $S_p^j$ and regression parameters ${\mathit{B}}_p^j$ of each candidate.

2.3.2. DML Branch

Let $\mathit{X}\in {ℝ}^{k\times k\times c}$ be the source space features and $\hat{\mathit{X}}\in {ℝ}^{1\times 1\times n}$ be the embedding space features. The DML aims to find a feature mapping function $f_{emb}:{ℝ}^{1\times 1\times c}\to {ℝ}^{1\times 1\times n}$, which ensures that positive candidates are close to the template target and far from negative ones in the embedding space [29]. To facilitate the measurement between the template target and candidates, we first use global average pooling (GAP) to convert the source space feature into a vector. Then, $f_{emb}$ is applied to map this vector into the embedding feature $\hat{\mathit{X}}\in {ℝ}^{1\times 1\times n}$. We model the feature mapping function as

$$f_{map}\left(\mathit{X}\right)=f_2\left(f_a\left(f_1\left(\mathit{X}\right)\right)\right),$$

(11)

where $f_1$ and $f_2$ indicate full connection layers, and $f_a$ denotes the Leak ReLU function.

2.3.3. Target Discrimination

We apply $f_{emb}$ to convert the template target and candidates into a consistent embedding space, where a smaller feature distance indicates higher similarity. Accordingly, we define the embedding similarity between two targets as

$$D_p^j=\phi \left({\widehat{\mathit{F}}}_p^j,{\hat{\mathit{F}}}_t\right)=\exp \left(-{‖{\widehat{\mathit{F}}}_p^j-\hat{\mathit{F}}‖}_2^2\right),$$

(12)

where ${\hat{\mathit{F}}}_t$ and ${\widehat{\mathit{F}}}_p^j$ represent the embedding feature of the template target and the j-th candidate, respectively. Following the QG-RCNN and DML processing, the affinity between the template target and candidates can be comprehensively described in terms of both classification and measurement perspectives. We further integrate the similarity and the classification scores to generate the final discriminant scores, denoted by

$${\kappa }_p=\left\{D_p^j\times S_p^j\left|j=1,2,\dots ,N\right.\right\},$$

(13)

where $N$ indicates the total number of the candidates. We adopt the similar approach as GlobalTrack to determine the final tracking result. Specifically, we select the candidate with the highest discriminant score as the final target and employ regression parameters to generate a more precise bounding box.

2.4. Loss Function

In this study, we train the proposed model end-to-end using a multi-task loss function, including STG-RPN and ID-RCNN losses, denoted as

$${\mathcal{L}}_{total}={\mathcal{L}}_{STG-RPN}+{\mathcal{L}}_{ID-RCNN},$$

(14)

For the STG-RPN part, there are three learning objectives, i.e., spatial location prediction, anchor shape prediction, and RPN prediction for anchor boxes. The joint loss is calculated as

$${\mathcal{L}}_{STG-RPN}={\lambda }_1{\mathcal{L}}_{loc}+{\lambda }_2{\mathcal{L}}_{shape}+{\mathcal{L}}_{RPN}^{},$$

(15)

where ${\lambda }_1$ and ${\lambda }_2$ denote the weighting factors, ${\mathcal{L}}_{loc}$ represents the spatial location loss, ${\mathcal{L}}_{shape}$ indicates the anchor shape loss, and ${\mathcal{L}}_{RPN}^{}$ denotes the traditional RPN loss that includes a binary cross-entropy loss for anchor classification and a smooth L₁ loss for anchor regression.

Considering the small size of UAV targets in infrared images, we adopt focal loss [30] as the spatial location loss to predict the anchor location. The focal loss is expressed as

$${\mathcal{L}}_{loc}\left(p_{},p_{}^{*}\right)=\frac{1}{N_p}{\displaystyle \sum _{i=1}^{N_p}-\left[\beta p_i^{*}{(1-p_i)}^{\gamma }\log (p_i)+(1-\beta )(1-p_i^{*})p_i{}^{\gamma }\log (1-p_i)\right]},$$

(16)

where $N_p$ denotes the total number of pixels in the location map; $p_i$ indicates the probability for spatial localization; $p_i^{*}$ denotes the corresponding truth label generated by reference [26]; and $\beta$ and $\gamma$ are the adjustment factors, which are set as $\beta =0.25,\gamma =2$ in this study.

The matching strategy in [26] is used to associate each anchor with the ground truth, and bounded IOU loss [31] is used for anchor shape prediction.

$${\mathcal{L}}_{shape}\left(w,h,w^{*},h^{*}\right)={\mathcal{L}}_1\left(1-\min \left(\frac{w}{w^{*}}-\frac{w^{*}}{w}\right)\right)+{\mathcal{L}}_1\left(1-\min \left(\frac{h}{h^{*}}-\frac{h^{*}}{h}\right)\right),$$

(17)

where $\left(w,h\right)$ represents the predicted anchor shape, $\left(w^{*},h^{*}\right)$ denotes the matched ground truth, and ${\mathcal{L}}_1$ represents the smooth L1 loss.

The ID-RCNN part comprises two losses: metric loss and RCNN loss. The overall loss is calculated as

$${\mathcal{L}}_{ID-RCNN}={\lambda }_3{\mathcal{L}}_{metric}+{\mathcal{L}}_{RCNN},$$

(18)

where ${\lambda }_3$ is the balancing factor; ${\mathcal{L}}_{metric}$ indicates the metric loss; and ${\mathcal{L}}_{RCNN}$ denotes the RCNN loss that is composed of binary cross-entropy loss and smooth L₁ loss for candidates’ classification and regression, respectively.

Regarding the metric loss, traditional triple loss [32] only considers the relative distance between positive and negative samples, without constraining the absolute distance between positive samples. As a result, the embedding distance between positive samples and the template target is not compact enough, which will disturb tracking results. Therefore, a triplet loss with an absolute distance constrain [33] is employed as our metric loss. Here, the template target is set as the anchor sample, while candidate targets with IOUs greater than 0.5 and less than 0.35 are labeled as positive samples (P) and negative samples (N), respectively. Thus, we can construct the triplet sample set as $T=\left\{〈{\mathit{X}}_o^{},{\mathit{X}}_{+}^i,{\mathit{X}}_{-}^j〉\right.$$\left.\left|i\in {ℝ}^P,j\in {ℝ}^N\right.\right\}$. The constrained metric loss can be computed as

$${\mathcal{L}}_{metric}\left(\mathit{X}\right)=\frac{1}{K_p}{\displaystyle \sum _{i=1}^{K_p}\left[\log \left(1+\exp \left(D({\mathit{X}}_o^i,{\mathit{X}}_{+}^i)-D({\mathit{X}}_o^i,{\mathit{X}}_{-}^i)+m\right)\right)+\nu \max \left(0,D({\mathit{X}}_o^i,{\mathit{X}}_{+}^j)-n\right)\right]},$$

(19)

where $K_p$ denotes the total number of triplet samples, ${\mathit{X}}_o^{}$ denotes the anchor sample, ${\mathit{X}}_{+}^{}$ denotes the positive sample, ${\mathit{X}}_{-}^{}$ denotes the negative sample, $D$ represents the distance function (Euclidean distance is used in this study), $m$ indicates the margin between positive and negative samples, $n$ indicates the distance threshold between positive samples, and $\nu$ denotes the balance parameter.

2.5. Tracking Process

Based on the above description of the key modules of our STFTrack, the main tracking flow is presented in Algorithm 1.

Algorithm 1: STFTrack: Spatio-temporal-focused Siamese network for infrared UAV tracking.
1:	Input: Tracking sequence of length L, $\left\{{\mathit{I}}^1,{\mathit{I}}^2,\dots ,{\mathit{I}}^L\right\}$;             Initial target bounding box ${\mathit{B}}^1=\left(x_c^1,y_c^1,w^1,h^1\right)$.
	Output: Target bounding box for each frame $\left\{{\mathit{B}}^1\left|i=1,2,\dots ,L\right.\right\}$.
2:	for f = 1 to L do
3:	if f == 1 then
4:	Construct multi-scale template image features;
5:	Extract multi-scale template target features using ROIAlign;
6:	Initialize the Kalman filter.
7:	if 1 < f < L + 1 then
8:	Compute the multi-scale search feature map;
9:	Compute the target location map by Equations (2), (6) and (7), and set anchor points in the region above threshold t;
10:	Predict the anchor shape of each anchor point and obtain focused anchor boxes;
11	Obtain adapted feature by Equation (9);
12:	Using the focused anchor boxes and adapted feature, obtain ROIs with candidate features through RPN;
13:	Calculate modulated feature by Equation (10);
14:	Using the modulated feature, obtain the classification scores and regression parameters of each candidate through the RCNN;
15:	Calculate the similarity between each candidate and template target by Equation (12);
16:	Obtain the discriminant scores by Equation (13);
17:	Choose the candidate with highest discriminant score as the tracking target and output the refined bounding box ${\mathit{B}}^f=\left(x_c^f,y_c^f,w^f,h^f\right)$;
18:	Update the Kalman Filter.
19:	End for

3. Experiments

3.1. Datasets and Evaluation Metrics

3.1.1. Datasets

We evaluate our proposed approach using two datasets: Anti-UAV [34] and LSOTB-TIR [35]. The Anti-UAV contains over 580,000 manually annotated boxes, covering 318 high-quality infrared UAV flight videos of about 1000 frames length each. It presents a realistic and dynamic tracking environment with challenging attributes such as thermal crossover, similar distractors, and frequent disappearance–reappearance. We evaluate the performance of the proposed method for infrared UAV tracking on the Anti-UAV dataset. The LSOTB-TIR dataset contains 1400 infrared video sequences with 47 different target types and more than 600,000 frames. It is a large and diverse general infrared target tracking dataset with 12 challenging attributes including non-rigid deformation, scale change, motion blur, etc. We use the LSOTB-TIR training set to pre-train the proposed method to improve its adaptation to infrared images, as well as the test set to validate the tracking performance of the approach for generic infrared targets.

3.1.2. Evaluation Metrics

Tracking stability is a crucial factor that significantly impacts the efficacy of visual tracking methods. This term encompasses both the precision of tracking results and the robustness to environmental interferences and occlusions. Improving tracking stability is essential for ensuring reliable and accurate tracking performance. In our study, we use the one-pass evaluation (OPE) test mode with three quantitative indicators: precision, success rate, and average overlap accuracy. The precision is defined as the percentage of frames in which the center location error (CLE) between the predicted target and the ground truth is less than a certain threshold. The precision can be calculated as

$$precision=\frac{1}{M}{\displaystyle \sum _{t=1}^M\delta \left(C_t<T_c\right)},$$

(20)

where M denotes the total number of video frames, $C_t$ represents the CLE between the t-th predicted target and ground truth, $T_c$ denotes the certain threshold, $\delta$ indicates an indicator function, where $\delta \left(C_t>T_c\right)=1$ when $C_t>T_c$, and $\delta \left(C_t>T_c\right)=0$ otherwise. The CLE is defined as

$$CLE={‖L_p-L_g‖}_2,$$

(21)

where ${‖·‖}_2$ denotes the L2-norm; $L_p$ and $L_g$ represent the center positions of the predicted target and ground truth, respectively.

The success rate is the proportion of successful frames with an IOU greater than the set threshold between the predicted target box and the ground truth. The success rate can be calculated as

$$success=\frac{1}{M}{\displaystyle \sum _{t=1}^M\delta \left(IO{U}_t>T_o\right)},$$

(22)

where $IO{U}_t$ represents the IOU between the t-th predicted box and its corresponding ground truth; $T_o$ denotes the set threshold.

To quantify the proposed approach, we calculate the precision and success rate with different thresholds to generate the precision plot and success rate plot. We obtain the AUC (area under curve) of the success rate plot and record the precision at the 20-pixel threshold as representative metrics. In addition, we adopt the average overlap accuracy defined in [34] to evaluate the long-term tracking performance of trackers. The average overlap accuracy is defined as

$$accuarcy=\frac{1}{M}{\displaystyle \sum _{t=1}^{M}IO{U}_t\times \delta \left(v_t>0\right)+p_t\times }\left(1-\delta \left(v_t>0\right)\right),$$

(23)

where $v_t$ is the ground-truth visibility flags of the target, and $\delta$ indicates an indicator function. When the target is truly visible, $v_t=1$ and $\delta \left(v_t>0\right)=1$. When the target is invisible, $v_t=0$ and $\delta \left(v_t>0\right)=0$. $p_t$ denotes the prediction indicator function. When the prediction is empty, $p_t=1$, otherwise $p_t=0$.

3.2. Implementation Details

Experimental platform: We conduct algorithm training and evaluation on the same high-performance workstation. The workstation specifications are:

• CPU: Intel Core [email protected] GHz
• GPU: 4× NVIDIA RTX 2080Ti graphics cards, each with 12 GB GDDR6 memory
• Storage: 1 T solid state drive
• Memory: 64 GB DDR4
• Operating system: Windows 10
• Programming environment: Python 3.7 and PyTorch 1.10

Parameters. Our STFTrack is built on Globaltrack and utilizes its trained weights for the backbone and FPN. We adopt ResNet-50 as the backbone. The input image size is set to 640 × 512. During training, STG-RPN generates 512 candidate targets for ID-RCNN training. The weights of each loss are set to ${\lambda }_1=1,{\lambda }_2=0.1,{\lambda }_3=0.1$. The metric loss parameters are set as $m=n=0.35,\nu =1$. For testing, STG-RPN generates 32 high-quality candidate targets, while ID-RCNN outputs the highest-scoring target. The threshold for determining the existence of UAV is set to 0.1, and the location probability threshold is set to 0.1. Additional parameter settings can be found in [20,26].

Training details. For model training, we use SGD to optimize our model for four NVIDIA 2080Ti GPUs. The momentum and weight decay are set to 0.95 and 1 × 10⁻⁵, respectively. The batch size on each GPU is set to two. In a given batch, the template image remains consistent across all image pairs. Image pairs are augmented in the same way as GlobalTrack. We first train STG-RPN on LSOTB-TIR for 32 epochs at a learning rate of 1 × 10⁻³. In rounds 16 and 28, the learning rate decayed by 0.1. Then, the entire model is trained at the same epoch and learning rate strategy using the Anti-UAV.

3.3. Quantitative Evaluation

We compare STFTrack with eight advanced tracking methods, including GlobalTrack, SiamRCNN, STARK [36], SiamRPN++LT [10], ATOM [9], SiamFC, ECO [37], and DSST [38]. For a fair comparison, each tracker uses default parameter settings and public trained weights, and is trained to converge on the Anti-UAV dataset on our platform.

The precision plots and success rate plots for the above nine trackers on the Anti-UAV dataset are shown in Figure 5. It can be observed intuitively that long-term trackers significantly outperform short-term trackers in terms of the precision and success rate. This is because long-term trackers, with their ability to perform re-capture, are better suited for the infrared UAV tracking task. In addition, among these long-term trackers, STFTrack achieves both the highest precision and success rate.

Table 1. Comparison results our STFTrack and state-of-the-art trackers on the Anti_UAV dataset. “↑” indicates that the larger the value, the better the performance.

Method	Precision ↑	Success ↑	Accuracy ↑	FPS ↑
DSST	0.490	0.349	0.354	31.2
SiamFC	0.510	0.369	0.375	60.2
ECO	0.618	0.437	0.444	7.5
ATOM	0.711	0.484	0.490	28.7
SiamRPN++LT	0.756	0.501	0.509	26.3
STARK	0.843	0.588	0.607	33.5
GlobalTrack	0.889	0.639	0.642	9.2
Siam-RCNN	0.900	0.653	0.658	6.1
Ours	0.912	0.666	0.677	12.4

shows the detailed comparison results of different tracking methods in terms of the precision, success rate and average overlap accuracy. STFTrack exhibits considerable improvement in performance metrics, achieving a precision of 91.2%, a success rate of 66.6%, and an average overlap accuracy of 66.7%. Specifically, STFTrack yields improvements of 2.3%, 2.7%, and 3.5% over the baseline GlobalTrack, and 1.2%, 1.3%, and 1.9% over SiamRCNN, respectively. Moreover, with fewer anchor boxes and candidates used during testing, STFTrack runs at 12.4 FPS, which is a 5.3 FPS increase over SiamRCNN. These results demonstrate that STFTrack has achieved significant improvements in both speed and accuracy for infrared UAV tracking.

3.4. Attribute-Based Evaluation

To comprehensively evaluate the proposed method, we conduct extended experiments on six challenging attributes such as out-of-view (OV), thermal crossover (TC), scale variation (SV), object occlusion (OC), fast motion (FM), and low resolution (LR) using the Anti-UAV dataset. Precision plots and success rate plots for each tracker on these six attributes are presented in Figure 6 and Figure 7, respectively. STFTrack obtains a competitive precision and success rate on all attributes in relation to the SiamRCNN, especially SV and TC, with precision scores of 3.8% and 2.5% and success rates of 3.8% and 2.4%, respectively. This demonstrates that with the help of the spatio-temporal focusing mechanism, STFTrack possesses a strong ability to adapt to target scale change and discriminate the infrared UAV from the infrared clutter background. Furthermore, Figure 8 reports the average overlap accuracy of each tracker on different attributes. In these results, STFTrack ranks first in average overlap accuracy for all attributes and encloses the largest polygon area, which indicates a superior performance over the other trackers.

In summary, the comprehensive rankings of the algorithms in terms of precision, success rate, and average overlap accuracy on different challenge attributes demonstrate that STFTrack can better address various challenges in infrared UAV tracking. In particular, the proposed method shows significant performance advantages when coping with target scale changes and thermal crossover interferences.

3.5. Qualitative Evaluation

To visually verify the superiority of the proposed STFTrack, we present the tracking results and IOU-Frames plots of the nine tracking methods mentioned above in several challenging scenarios on Figure 9.

Figure 9a displays the comparison results of each tracker on a fast motion scenario. When the UAV undergoes significant range displacement between consecutive frames, e.g., frame 53 to frame 56, only SiamRCNN, GlobalTrack, STARK, and STFTrack are able to track the target due to their large search approach. Among these trackers, STFTrack exhibits the highest success rate (the number of IOU > 0.5 frames) and the mean IOU over the whole sequence. This is attributed to its guided anchoring strategy, which is particularly effective in handling target scale changes.

Figure 9b demonstrates the performance of each tracker in scenarios involving occlusion and similar distractors. In this sequence, the infrared UAV disappears at frame 39 due to occlusion, and reappears at frame 102. Moreover, in frame 438, other airborne interfering objects appear. The tracking results reveal that STFTrack is particularly effective at recapturing the UAV quickly and accurately after it reappears following occlusion at frame 102. In contrast, other trackers exhibit suboptimal performance in such situations. Especially when the infrared UAV disappears during frame 39 to frame 101, STFTrack realizes a reliable judgment of the existence of the UAV, while other tracking methods are poor. Moreover, in scenarios where other aerial objects appear in the background, e.g., frame 438, SiamRPN++LT and GlobalTrack tend to incorrectly track the interfering objects due to their inadequate discrimination capabilities. However, STFTrack utilizes a two-level spatio-temporal-focused mechanism that leverages spatio-temporal cues to resist interference from distractors, which effectively mitigates this problem and leads to superior tracking results.

Figure 9c,d show the tracking results under complex thermal crossover interference. In this scenario, the contrast between the UAV and the background is substantially reduced, posing a significant challenge to the discriminative ability of trackers. Upon observing the tracking effect of these two video sequences, we can find that almost all the trackers fail to track the UAV when subjected to thermal crossover. However, STFTrack exhibits outstanding accuracy and stability under such challenging scenarios. These results effectively verify that STFTrack has a stronger discriminatory ability than other trackers.

In summary, the proposed STFTrack is capable of handling various challenging tracking scenarios such as fast motion, similar distractors, and thermal crossover. Compared with other algorithms, STFTrack is better suited for infrared UAV tracking tasks.

3.6. Ablation Analysis

To investigate the contributions of each component of STFTrack, we conduct ablation analysis on the critical sub-modules, target location prediction, and metric loss on the Anti-UAV testset. By removing and replacing the essential parts of the baseline model, different variants are constructed. We adopt the precision, success rate, and average overlap accuracy as the evaluation metrics.

Table 2. Comparison of different variants of sub-modules. “↑” indicates that the larger the value, the better the performance. “√” means that the element is selected.

Num.	Main Modules				Evaluation Metrics
	QG-RPN	STG-RPN	QG-RCNN	ID-RCNN	Precision ↑	Success ↑	Accuracy ↑
1	√				0.838	0.586	0.594
2		√			0.865	0.623	0.631
3	√		√		0.889	0.635	0.642
4	√			√	0.887	0.643	0.653
5		√	√		0.903	0.654	0.663
6		√		√	0.916	0.666	0.677

shows the ablation results of the STG-RPN module and the ID-RCNN module. Each row of the table represents a set of model variants. Specifically, the QG-RPN and QG-RCNN are the original modules from GlobalTrack, while both the QG-RPN and STG-RPN represent the single-stage models that directly adopt the highest-scoring candidate of RPN as the tracking result.

Effects of STG-RPN. By comparing the evaluation metrics of variant 1 with variant 2, variant 3 with variant 5, and variant 4 with variant 6, we can observe that the variants based on the STG-RPN exhibited a greater performance improvement than those based on the QG-RPN. Furthermore, our analysis revealed that the STG-RPN module outperformed the QG-RPN module with notable improvements of 3.3%, 3.7%, and 3.9% in terms of precision, success rate, and average overlap accuracy, respectively. These results provide compelling evidence that our spatio-temporal information-guided strategy is highly effective at enhancing performance by suppressing distractions in the Anti-UAV scenario.

Effects of ID-RCNN. By comparing variant 3 with variant 4 and variant 5 with variant 6, we can find that the variants with the ID-RCNN outperformed those with the QG-RCNN, particularly in terms of the success rate and average overlap accuracy. These results suggest that the ID-RCNN performs better than the QG-RCNN by further enhancing the discriminability of the model through metric learning.

Effects of target location prediction. The ablation results of the target location prediction strategies are presented in

Table 3. Comparison of different target location strategies. “↑” indicates that the larger the value, the better the performance. “√” means that the element is selected.

Spatial Location Prediction	Temporal Location Prediction	Precision ↑	Success ↑	Accuracy ↑
√		0.890	0.645	0.658
	√	0.723	0.484	0.491
√	√	0.916	0.666	0.677

, which compared the performance of three strategies, namely spatial location prediction, temporal location prediction, and spatio-temporal fusion location prediction. The results demonstrate that the spatio-temporal fusion location prediction strategy outperforms the other two strategies, with improvements of 2.6%, 2.1%, and 1.9% in the three evaluation metrics, respectively, compared to spatial location prediction. Conversely, the performance of the temporal location prediction strategy is the weakest among the three. This is due to the fact that temporal localization prediction is essentially a local strategy that struggles to handle the sudden movements and disappearance–reappearance of infrared UAVs. Conversely, spatial localization prediction is a global strategy that is vulnerable to thermal crossover and similar object interference. However, spatio-temporal fusion location prediction cleverly integrates the advantages of both a local and global search, such as resistance to background interference and target recapture, making it the most suitable method for infrared UAV tracking.

Effects of DML.

Table 4. Comparison of different metric loss functions. “↑” indicates that the larger the value, the better the performance.

Loss Function	Precision ↑	Success ↑	Accuracy ↑
No metric loss	0.903	0.654	0.663
Naive triplet loss	0.907	0.658	0.667
Hard triplet loss	0.913	0.662	0.674
Constrained triplet loss	0.916	0.666	0.677

provides a comparison of different metric loss functions, including no metric loss, naive triplet loss, hard triplet loss, and constrained triplet loss. The results demonstrate that utilizing triplet loss can improve the tracking performance, with constrained triplet loss exhibiting the greatest improvement. In particular, constrained triplet loss outperforms no metric loss by 1.3%, 1.2%, and 1.4% in the three evaluation metrics.

3.7. Tracking Generalizability Evaluation

In order to assess the tracking generalization of STFTrack to general infrared targets, we conduct a comparison study with nine other tracking methods, including GlobalTrack, ECO-deep-TIR, ECO_TIR, MDNet [39], ATOM, SiamRPN++, SiamMask [40], Siamese-FC-TIR, and MMNet [41], on the LSOTB-TIR testset. The tracking performance comparison of each tracker is illustrated in Figure 10. Based on these comparison results, it is evident that GlobalTrack, which performs exceptionally well in anti-UAV tracking, exhibits poor performance in LSOTB. This is due to the relatively smooth target motion and the presence of a wider variety of targets in LSOTB, which demands a more discriminative tracker with less emphasis on re-detection and more focus on local target search. In such cases, short-term trackers are more effective. However, STFTrack outperforms the other nine trackers with a precision of 77.2% and a success rate of 63.4%, achieving comparable performance to best short-term trackers, i.e., ECO-deep-TIR. Additionally, compared to the baseline algorithm, i.e., GlobalTrack, STFTrack achieves a 6% improvement in precision and 4.5% in success rate. These results suggest that STFTrack’s global-to-local focusing mechanism applies to both long-term and short-term infrared target tracking, substantially enhancing tracking performance.

4. Conclusions

In this study, we propose a novel spatio-temporal-focused Siamese network for infrared UAV tracking. It seamlessly integrates spatio-temporal constraints into a Siamese-based global tracking framework, employing a novel two-level global-to-local focusing strategy. At the first level, we constrain the search region by implementing a soft region selection mechanism using the proposed STG-RPN. This mechanism utilizes template features and motion characteristics to focus the target location and generate high-quality candidate targets. At the second level, we focus on refining candidate targets by introducing ID-RCNN. It further magnifies the discriminability between the target UAV and similar distractors, thereby improving target discrimination accuracy. Experiments on the Anti-UAV dataset demonstrate that our proposed STFTrack achieves 89.5% precision, 64.9% success rate, 65.6% average overlap precision, and 12.4 FPS. Compared to other tracking methods, STFTrack exhibits superior performance. STFTrack also demonstrates robust tracking capability in complex scenarios such as rapid motion, thermal crossover, and comparable distractors. Experiments on LSOTB-TIR dataset show that STFTrack achieves 77.2% precision and 63.4% success rate, comparable to the best short-term tracker. This indicates STFTrack applies to both long-term and short-term infrared target tracking. In conclusion, STFTrack deftly integrates the anti-background interference capability of the local search and the re-detection capability of global search, enhancing the precision and stability of infrared target tracking.

Despite achieving promising results, our proposed STFTrack still suffers two limitations. One limitation is that, as a Siamese-network-based tracking method, STFTrack cannot adaptively update the template features over time, leading to an error accumulation in long-term tracking scenarios. The other is that the maneuver detection method based on pseudo-innovation used in STFTrack is not robust enough against camera jitter, which may result in the inaccurate prediction of target motion states. In our future work, we will first explore lightweight template feature updating strategies, which adaptively integrate target appearance features at different times, to enhance the algorithm’s adaptability to target shape variations. Then, we plan to introduce an image stabilization module to reduce the interference of camera jitter, further enhancing the tracking stability. Finally, we will investigate the reproduction of other advanced tracking methods for a more in-depth study of infrared UAV tracking.