Spectral-Spatial Feature Enhancement Algorithm for Nighttime Object Detection and Tracking

Abstract

Object detection and tracking has always been one of the important research directions in computer vision. The purpose is to determine whether the object is contained in the input image and enclose the object with a bounding box. However, most object detection and tracking methods are applied to daytime objects, and the processing of nighttime objects is imprecise. In this paper, a spectral-spatial feature enhancement algorithm for nighttime object detection and tracking is proposed, which is inspired by symmetrical neural networks. The proposed method consists of the following steps. First, preprocessing is performed on unlabeled nighttime images, including low-light enhancement, object detection, and dynamic programming. Second, object features for daytime and nighttime times are extracted and modulated with a domain-adaptive structure. Third, the Siamese network can make full use of daytime and nighttime object features, which is trained as a tracker by the above images. Fourth, the test set is subjected to feature enhancement and then input to the tracker to obtain the final detection and tracking results. The feature enhancement step includes low-light enhancement and Gabor filtering. The spatial-spectral features of the target are fully extracted in this step. The NAT2021 dataset is used in the experiments. Six methods are employed as comparisons. Multiple judgment indicators were used to analyze the research results. The experimental results show that the method achieves excellent detection and tracking performance.

1. Introduction

Target tracking is an application of visual algorithm research with great practical significance [1]. The long-term single-target tracking algorithm is an important direction in this field [2]. Video target detection is one of the hot spots, which is a technology for locating and classifying targets from video scenes [3]. With the popularity and continuous iteration of UAVs, the range, payload and survivability of UAVs have been further improved, which can provide longer-term surveillance and higher-precision reconnaissance [4]. A rich library of object-tracking datasets that are publicly available has been built for researchers [5,6,7].

Previously, the solutions of related algorithms were mainly based on traditional filtering [8,9,10]. In recent years, the performance of long-term single-target tracking algorithms based on deep learning is gradually catching up with traditional filtering methods [11,12,13]. With the rise of deep learning and the rapid improvement of GPU (Graphics Processing Unit) computing power, existing target tracking algorithms have shown high success rates and accuracy rates [14,15]. Industrial products are used in many practical scenarios and show good performance [16]. However, if the captured image dataset is directly applied to target detection, it will encounter the problem of decreased detection accuracy due to motion blur, video out-of-focus, etc. [17]. Moreover, although these methods have shown their positive effects in related studies, they focus on tracking targets based on favorable lighting conditions [18,19]. However, nighttime and bad weather account for a considerable proportion of the whole year, and target tracking at night also has strong practical application significance. Therefore, it is necessary to further study the tracking algorithm for targets at night.

Compared with daytime-based target recognition, detecting and tracking nighttime targets is somewhat difficult. Mainly at night, the brightness of the target image is low, and the image is blurred. Detection targets are affected by low contrast, low illumination, low color saturation, and noise in the image [20,21]. These differences lead to differences in feature distributions between daytime and nighttime images.

To enhance the features of night objects, a spectral-spatial feature enhancement algorithm for nighttime object detection and tracking (SFDT) is proposed in this paper. The features of nighttime targets are enhanced by integrating the features of daytime targets. However, due to cross-domain differences, current trackers generalize poorly to nighttime scenes [22,23], which seriously hinders the performance improvement of nighttime tracking algorithms. To improve this cross-domain gap and the difficulty of performance degradation, this algorithm is devoted to solving the cross-domain object tracking problem. Then the detection and tracking model can adapt to the working environment at night under the general conditions of the day.

There are three main contributions of this work:

• A novel algorithmic framework is proposed for nighttime object detection and tracking tasks. We perform a feature-enhancing preprocessing operation on nighttime object images. Object features in images at night are more prominent to improve the accuracy of object detection.
• Introduce the concept of domain adaptation in transfer learning to create a day-night discriminator, which can align the target features of day and night and narrow the domain gap between them.
• Low-light enhancement and Gabor filtering are performed on the dataset to enhance the features, and the spectral and spatial features are fully utilized to improve the tracking performance.

The major parts of this paper are concluded as follows. Section 1 introduces the development of object detection methods and transfer learning. Section 2 describes the proposed methodology in detail. Section 3 shows the results of the experiments. Section 4 analyzes the advantages and disadvantages of the algorithm. Section 5 draws the conclusion.

2. Related Work

2.1. Target Detection

Object detection has been a research hotspot in the field of intelligent algorithms [24]. According to the contents of different studies, we can divide the target detection field into the target detection of single-frame static images and the target detection with video timing information [25]. From the perspective of algorithm development, the target detection field can be divided into target detection based on traditional methods and target detection based on deep learning [26]. The following focuses on object detection based on deep learning algorithms.

In the Large-Scale Visual Recognition Challenge (ILSVRC) [27] competition in 2012, Krizhevsky et al. [28] built a deeper convolutional neural network, and the image classification accuracy came out on top in the competition. Subsequently, the method of selecting regions by sliding window, which is commonly used in algorithms, is gradually replaced by advanced candidate region generation methods, such as constrained parametric min-cuts (CPMC) [29], Selective Search [30] and Multiscale combinatorial grouping (MCG) [31]. The target detection algorithm has entered a new stage. Compared with the traditional sliding window method, the emerging candidate region generation method can make full use of the texture, color and other information of the image, and merge regions of different scales by calculating the similarity [32]. While the number of candidate regions is reduced, the quality of the candidate regions is improved. Ross Girshick et al. [33] combined the Region Proposal with the convolutional neural network (CNN) [34] and proposed a region-based convolutional neural network Region-Convolutional Neural Network (R-CNN) detection algorithm. The algorithm achieves leading results on the ImageNet dataset. However, this algorithm has the disadvantage of low detection speed. In order to improve the detection efficiency, He et al. [35] proposed the Spatial Pyramid Pooling Network (SPP-Net). The algorithm performs feature extraction on the data only once. Ross Girshick et al. [36] proposed Fast R-CNN. In this algorithm, a Region of Interest (RoI) pooling layer is adopted to implement feature mapping. The multi-task loss function is also introduced, which further improves the accuracy of target detection. Ren et al. [37] proposed a Faster R-CNN on the basis of Fast R-CNN. The innovation lies in the use of a Region Proposal Network (RPN).

The above target detection method based on candidate regions is called a two-stage detection method. Joseph Redmom et al. [38] proposed the You Only Look Once (YOLO) algorithm, which is called a one-stage detection method because the candidate region does not need to be generated. Instead, the classification and regression of candidate boxes are performed directly on the original image [39]. Target detection using YOLO is a current hotspot. Papers [40,41] all employed YOLO to track the targets.

A Siamese network is a special type of neural network and is one of the simplest and most commonly used one-shot learning algorithms [42]. Compared with CNN networks, Siamese networks require less training data. Therefore, Siamese networks can perform better in detection if there is insufficient prior data. This method was proposed by Sint [43] and SiamFC [44]. B.Li et al. [37] added the Region Proposal Network (RPN) to the Siamese framework for the purpose of object detection. A deeper backbone network and feature aggregation structure are applied by the SiamRPN++ algorithm [45], and the tracking accuracy is further improved. To alleviate the problem of algorithmic complexity caused by the introduced hyperparameters, the AnclFree method [46,47,48] computes a per-pixel regression to predict the offset at each pixel. To further improve detection and tracking, transformer is introduced into the Siamese framework [48,49] to model global information.

2.2. Domain Adaptation

Domain adaptation is a branch of Transfer Learning. Introducing domain adaptation into the algorithm can achieve the purpose of narrowing the differences in the characteristics of different domains [50]. The principle is to map data features from different domains (such as two different datasets) to the same feature space. Then data from other domains (source domains) are leveraged to enhance training in the target domain. Usually, there is rich labeled prior information in the source domain, which contains samples different from the test samples; while the samples with the same nature as the test samples are in the target domain, which does not contain any information or contains a small amount of prior information [51]. The source domain and the target domain often belong to the same type of domain, but the properties of the samples in them are different. Domain adaptation is widely used in the field of object classification. Y. Chen et al. [52] constructed a domain-adaptive object detection algorithm that improved the domain movement problem. Yu et al. [53] proposed a new model Faster Multi-Domain Net. Domain adaptation components are designed to obtain more general characteristics. An adaptive spatial pyramid pooling layer is implemented to reduce model complexity and speed up tracking. The framework proposed by Jihoon Moon et al. [54] employs an Incremental Mean Subspace Computation (ICMS) technique to solve the Online Unsupervised Domain Adaptation (OUDA) problem. Debaditya Acharya et al. [55] combined hierarchical edge maps and semantic segmentation for domain adaptation to achieve the purpose of single-image localization of 3D models.

3. Proposed Method

Spectral characteristics state that any object in nature has its own law of electromagnetic radiation. Spatial features are texture features obtained by Gabor filtering [56]. We use the method of spectral-spatial features to further improve the accuracy of detection and tracking. The SFDT algorithm is dedicated to enhancing the features of nighttime targets for easy extraction. Daytime target features and nighttime target features are modulated to enhance the detectability of nighttime features. The method consists of four steps. First, the target domain images without labels are preprocessed, which includes low-light enhancement, object detection and dynamic programming. Then the patches at nighttime and the target images at daytime pairs are obtained. Secondly, a feature extractor is applied to obtain daytime and nighttime features. In this step, adversarial learning is introduced to reduce the difference between them. Third, the Siamese network is trained on the feature sets, as the object detector and tracker (SDT), which is inspired by symmetry theory. Fourth, the test data are preprocessed (including low-light enhancement and Gabor filtering) and used as the input of SDT to obtain the final detection and tracking results. To clearly illustrate the algorithm flow, the overall description of the SFDT is shown in Algorithm 1.

Algorithm 1 Spectral-spatial feature enhancement algorithm for nighttime object detection and tracking

Input: Target dataset (datasets for the target domain, the source domain, and the test) 1. The target domain data is preprocessed, including low-light enhancement, object detection, and dynamic programming with (1)–(3). 2. Source and target domain data features are extracted with (4)–(5) 3. Source and target domain data features are modulated by domain adaptive structure with (6)–(9). 4. Siamese network is trained to get the tracker head, and the loss function is obtained. 5. The test data is preprocessed, including low-light enhancement and Gabor filter with (1)–(3) and (11)–(12). 6. The feature-enhanced test data is detected and tracked by SDT with (13). Output: Object detection maps and location data

3.1. Preprocessing

3.1.1. Low Light Enhancement

Images acquired at night generally suffer from low light and low visibility. To avoid the phenomenon that the images cannot provide useful information for subsequent object detection, we apply the low-light enhancement method to “illuminate” the objects [57]. This method achieves illumination enhancement by adjusting the pixmap curve.

First, the pixel values of all points are normalized. Then the red, green and blue three-channel pixels of the image are mapped to the enhancement curve (EC) as follows:

$$EC\left(I\right(x);\gamma )=I\left(x\right)+\gamma I\left(x\right)(1-I(x\left)\right)$$

(1)

where x represents the coordinate point of a pixel, and I(X) is the pixel value of the point; $\gamma$ is a variable with a value range of [−1, 1], which is used to control the exposure level of the image and adjust the tone; EC(I(x);$\gamma$) is the output enhancement pixel.

Second, to deal with challenging low-light conditions, the higher-order EC curve with an iterative function is defined as follows:

$$E{C}_k\left(x\right)=E{C}_{k-1}\left(x\right)+{\gamma }_{n}E{C}_{k-1}\left(x\right)(1-E{C}_{k-1}\left(x\right))$$

(2)

where k is the order, which is the number of iterations to control the curvature. The exact value of k is 8 in the research.

Finally, to obtain the mapping curve with optimal effect, a deep learning-based parameter estimation network is proposed. An image is input, and the network will generate a pixel-based curve parameter map as output.

3.1.2. Video Object Detection

An algorithm based on salient object detection is applied to detect the aforementioned low-light enhanced dataset. In order to take full advantage of the context-sensitive information, every three consecutive frames are grouped as input. At the same time, dense salient features can be obtained. ResNet-101 [58], as a feature extractor, can generate feature maps with four spatial resolutions. To preserve the spatial structure, spatial pyramid layers are introduced to replace the last two layers of the network. Finally, the decoder aggregates temporal and spatial features into spatiotemporal features to generate salient predictions.

3.1.3. Dynamic Programming

To generate motion sequences of objects in videos, a dynamic programming method is introduced [59]. We constructed bounding rectangles for the above targets as candidate boxes. Based on the fact that the trajectories of moving objects in adjacent frames are smooth, the key of the proposed method is to encourage smooth trajectories. Therefore, the reward of the trajectory between candidate boxes in two adjacent frames is computed to remove unreliable boxes. The reward for dynamic programming $\mathcal{R}$ is defined as follows:

$$\mathcal{R}={(\frac{l_{i,m}-l_{j,n}}{w_{j,n}})}^2+{(\frac{t_{i,m}-t_{j,n}}{h_{j,n}})}^2+{(log(\frac{w_{i,m}}{w_{j,n}}))}^2+{(log(\frac{h_{i,m}}{h_{j,n}}))}^2$$

(3)

where [$l_{i,m}$,$t_{i,m}$,$w_{i,m}$,$w_{i,m}$] and [$l_{j,n}$,$t_{j,n}$,$w_{j,n}$,$w_{j,n}$] represent the information of the two boxes; i and j represent the labels of the i-th and j-th frames; m and n indicate the box indexes; l is the left coordinate, t is the top coordinate; w and h denote the width and height of the box.

3.2. Dat-Net

To modulate the features of the source domain and target domain for good detection of nighttime objects, we apply an adaptive-based nighttime tracking network (DAT-Net) [60]. It contains the following modules.

3.2.1. Feature Extractor

The applied Siamese network contains two sub-networks and two branches. One branch is the template branch, and the input is the cropped image T of the previous frame; the other is the search branch, and the input is the data S of the current frame. Then feature maps $\varpi \left(T\right)$ and $\varpi \left(S\right)$ are generated, represented as follows [60]:

$$\varpi \left(T\right)=\sum _{k=p}^N{F}_k\left(T\right)$$

(4)

$$\varpi \left(S\right)=\sum _{k=p}^N{F}_k\left(S\right)$$

(5)

where $F_k$ indicates the extracted features from the k-th block of N backbones in all. The data used by the tracker are usually the features of the last p to N blocks. This backbone in the Siamese Neural Network is recorded as FTS-101 [61].

3.2.2. Transformer Adaptive Structure

Aiming at the difficulty that object features are inconspicuous and hard to be detected at nighttime, we introduce a domain-adaptive structure in transfer learning. By modulating the characteristics of the daytime target and the nighttime target, nighttime tracking will be more effective. Specifically, since the transformer structure is good at learning long-range target relations [62], an adaptive layer based on the transformer structure is constructed in this paper.

The transformer applies global information, the timing information in the video cannot be reflected. Therefore, we add the sequence code P to the features obtained above. The transformer structure consists of an encoder and a decoder; these two parts have multiple self-attention structures. Multihead self-attention (MSA) is conducted as:

$${\widehat{\varpi \left(T\right)}}^{\prime }=MSA(P+\varpi \left(T\right))+P+\varpi \left(T\right)$$

(6)

$${\widehat{\varpi \left(S\right)}}^{\prime }=MSA(P+\varpi \left(S\right))+P+\varpi \left(S\right)$$

(7)

$$\widehat{\varpi \left(T\right)}=LN(FFN\left(Mod\left(LN\left({\widehat{\varpi \left(T\right)}}^{\prime }\right)\right)\right)+{\widehat{\varpi \left(T\right)}}^{\prime })$$

(8)

$$\widehat{\varpi \left(S\right)}=LN(FFN\left(Mod\left(LN\left({\widehat{\varpi \left(S\right)}}^{\prime }\right)\right)\right)+{\widehat{\varpi \left(S\right)}}^{\prime })$$

(9)

where ${\widehat{\varpi \left(T\right)}}^{\prime }$ and ${\widehat{\varpi \left(S\right)}}^{\prime }$ are the intermediate variables, $FFN$ represents the feedforward network, and $LN$ represents the normalization operation. $Mod$ is a modulation layer in [18].

3.2.3. Tracker Head

The above-mentioned feature sets $\widehat{\varpi \left(T\right)}$ and $\widehat{\varpi \left(S\right)}$ modulated by the transformer bridge layer are subjected to a cross-correlation operation. Then the similarity map is generated as input to the tracker head to predict object locations.

3.2.4. Feature Discrimination Structure

For the above-modulated feature set $\widehat{\varpi \left(T\right)}$, we need to correctly distinguish the features from daytime or nighttime to achieve the alignment of day/night features. The day-night feature discriminator D consists of two transformer layers and a gradient reversal layer (GRL) [63]. $\widehat{\varpi \left(T\right)}$ is first performed as a Softmax operation and then put into the GRL layer; finally, it goes through two transformer layers. In this way, the function of distinguishing features is realized and the final output is the predicted class c.

3.2.5. Loss Function

The loss of the entire network consists of three parts: classification loss ${\ell }_{clc}$ [64], regression loss ${\ell }_{reg}$ and domain adaptive loss ${\ell }_{adp}$. The application of classification loss and regression loss functions guarantees the superior tracking performance of the tracker; here the tracking loss is consistent with the baseline tracker. The adaptive loss function is used to ensure the modulation effect of source and target domain features. The total loss function [60] is:

$$\ell ={\ell }_{clc}+{\ell }_{reg}+\mu {\ell }_{adp}$$

(10)

where $\mu$ is a weighting coefficient with a value of 0.01.

3.2.6. Gabor Filter

Increasing the number of features in the image is beneficial for better detection of the target. In this algorithm, we introduce texture features as representatives of spatial features. Therefore, the combination of spatial features and spectral features is used as the basis for detecting targets, which will help improve the accuracy of detection. While local features can well represent local changes and capture the existence of small objects better than global features, the Gabor wavelet feature based on biological characteristics is one of the more successful local features [56,65]. The two-dimensional Gabor filter is a complex exponential function modulated by a Gaussian function with strong spatial position and direction selectivity, which can effectively extract the direction features of multiple scales of the image. Therefore, the Gabor filter has significant advantages in extracting the texture and orientation of local features of the image and plays a vital role in the subsequent analysis, processing and identification. To enhance data features, the data are preprocessed by low-light enhancement and Gabor filter based on paper [60]. Gabor filters are adopted to extract texture features from low-light enhanced test data. However, too many features may cause the opposite effect and increase the complexity of the algorithm. In this paper, only one texture feature is adopted. The size of the Gabor filter is set to 9, and the angle is set to $\frac{\pi }{4}$ clockwise.

$$Tes{t}^{\prime }=EC\left(Test\right)$$

(11)

$$TF=Gabor\left(Tes{t}^{\prime }\right)$$

(12)

$$Map=SDT\left(TF\right)$$

(13)

where $Test$ is the original test set. $Tes{t}^{\prime }$ is the dataset after low-light enhancement. $Gabor$ means Gabor filter. $Map$ is the final result.

The effect of the images enhanced by low light and the Gabor filter is shown in Figure 1.

4. Experimental Results

4.1. Datasets

The training sets of the Siamese network are the VID dataset [66], GOT-10K dataset [6] and NAT2021 benchmark [60].

The VID dataset contains a total of 5354 video sequences with 30 target categories set. The training set contains 3862 video sequences. Each frame of an image in the training set corresponds to a manual annotation file in xml format, which contains information such as the target ID, target category and target frame.

GOT-10K is a target tracking dataset released by the Chinese Academy of Sciences. The dataset contains more than 10,000 video sequences with more than 1.5 million manually annotated target boxes. The object class contains a total of 563 objects, including five sub-object classes: animal, vehicle, person, passive motion object and object part.

In order to provide an evaluation of long-term tracking performance, we further build a long-term tracking subset, namely NAT2021-L-test, consisting of 23 sequences that are longer than 1400 frames. The NAT2021 dataset is a sequence of nighttime video images containing a test set and an unlabeled training set. It consists of multiple objects (cars, people, buses, trucks, buildings, etc.) or activities (skating, running, cycling, etc.). The training set contains 1400 unlabeled sequences. Different from the training dataset, the NAT2021L of NAT2021 is the test set in these experiments. This is a set of long-tracking datasets.

4.2. Evaluation Metrics

In this paper, we adopt three indices, Intersection over Union (IoU), success rate and precision, as the result evaluation criteria.

• Intersection over Union

Intersection over Union(IoU) [67] represents the degree of overlap between the location of the predicted box and the location of the ground truth box. The larger the IoU, the better the tracker effect. This evaluation criterion can reflect the change in the tracking target scale and directly determine whether the target tracking task fails.

$$IoU=\frac{R_G^k\cap R_O^k}{R_G^k\cup R_O^k}$$

(14)

where $R_G^k$ and $R_O^k$ denote the area contained by the ground-truth tracking box and the area contained by the predicted tracking box in the k-th frame, respectively.

• Success rate (SR)

The success rate of object tracking is defined by IoU. If the IoU of a frame is greater than the threshold ($T_h$= 0.5), the tracking is considered successful. Therefore, the success rate is the percentage of successfully tracked frames out of the total frames. The specific calculation formula is as follows:

$$SR=\frac{Number(IoU\ge T_h)}{N}$$

(15)

where $Number(IoU\ge T_h)$ represents the number of frames detected successfully; N represents the total number of frames.

SR will change as the threshold changes. Therefore, we find the success rate curves by plotting the threshold and its corresponding accuracy into a curve.

• Precision

The D is defined as the distance between the center position of the box output by the model and the center position of the ground truth box. The distance is determined by Euclidean. Precision is the percentage of video frames where D is less than a given threshold. Generally, the threshold is set to 20 pixels.

4.3. Overall Performance

4.3.1. Comparison Algorithms

For a more in-depth analysis of trackers in nighttime tracking, five novel trackers were introduced as comparison algorithms. They are D3S [68], HiFT [18], Ocean [69], UpdateNet [70], SiameseNet [46] and UDAT [60]. They are used for evaluation on the NAT2020L dataset.

4.3.2. Parameter Settings

The Adam optimizer is applied to train the discriminator. The base learning rate is set to 0.005 and decayed according to a multivariate learning rate strategy with a power of 0.8. The base learning rate of the bridging layer is 0.005 and is optimized with a baseline tracker [63].

4.3.3. Experimental Results

As one of the most common scenarios in object tracking, long-term tracking involves several challenging properties. We tested the feature-augmented NAT2021L dataset with the SDT. The SFDT framework is conducted using PyTorch on an Intel(R) Xeon(R) Silver 4216 CPU. The scores of all methods are shown in

Table 1. Performance of trackers on NAT2021L.

Trackers	D3S	HiFT	Ocean	UpdateNet	SiameseNet	UDAT	SFDT
Precision	0.483	0.441	0.460	0.412	0.483	0.506	0.524
SR	0.327	0.292	0.313	0.245	0.372	0.376	0.401

D3S [68], HiFT [18], Ocean [69], UpdateNet [70], SiameseNet [46], UDAT [60].

. In order to show the detection performance of the proposed algorithm more clearly, the detection maps are shown in Figure 2. Here are two groups of consecutive frames taken from the test dataset.

5. Discussion

• The results in Table 1 show that SFDT has the best performance. It demonstrates that the algorithm can achieve competitive long-term tracking performance and significantly improve the tracking performance of the tracker. Figure 2 shows the detection and tracking results of the algorithm proposed in this paper. It can be seen that the proposed method can effectively track the target.
• In particular, compared with the UDAT algorithm without preprocessing the test set, SFDT performed better than UDAT on both precision and SR. The accuracy is improved by 2%, and the SR is improved by 3%. This means that the data enhanced by lighting and texture features are more suitable for tracking. Under this condition, the network can detect the location of the target more accurately.
• Paper [60] points out that the low-light enhanced test set is not conducive to object tracking. As shown in Figure 1, the low-light enhanced images are too bright, and the image’s details are lost. We speculate that overexposure leads to a decrease in tracking performance. The addition of texture feature enhancement made image details more obvious. Moreover, the brightness of the images after Gabor filtering becomes lower, which makes up for the loss of the previous step. However, the image brightness is still brighter than the original image, which is good for detection and tracking.
• As shown in Figure 2, two groups of consecutive frames are detected. In the first group of frames, there is only one vehicle as the target, and it was successfully detected; in the second group of frames, there are multiple vehicles in the background as interference, and the vehicles that appear continuously are still successfully tracked by the proposed algorithm. The tracking performance of the proposed algorithm is well illustrated.
• Although the proposed algorithm is superior to the comparison algorithms in tracking and monitoring, the computational complexity of the algorithm increases because the preprocessing step is improved in this algorithm. A texture feature extraction method based on the Gabor filter is adopted. The processing time for the same dataset is longer than the proposed algorithm in the paper [60]. There is no evaluation for real-time object detection and tracking performance. We will investigate this in future work.

6. Conclusions

In order to enhance the features of nighttime objects and reduce the impact of insufficient lighting, the nighttime training set is preprocessed, including low-light enhancement, object detection and dynamic programming. To make nighttime targets easier to be detected, daytime datasets are processed together through feature extractors. Adversarial learning and domain adaptive structure are used to reduce the gap between features. Low-light enhancement and Gabor filtering are performed on the test data to enhance the characteristics. Finally, the test dataset is processed through the Siamese network to get the results. The experimental results show that the method proposed in this article performs well on multiple evaluation indicators and is superior to the compared algorithms. The proposed algorithm can improve the detection and tracking accuracy of objects. This fully demonstrates the effectiveness of day/night feature modulation and spatial-spectral feature combination and can be extended to other similar application fields.