Improving Ship Detection Based on Decision Tree Classification for High Frequency Surface Wave Radar

Abstract

The traditional constant false alarm rate (CFAR) method, with fixed parameter settings and single noise background calculation, is unable to intelligently catch the current detection background. To improve the performance of the CFAR method, this paper proposes a target detection method based on decision tree classification (DTC) for high-frequency surface wave radar (HFSWR). Firstly, the training sample set and labels are obtained by means of a ship automatic identification system (AIS). Then, feature vector of range dimension, Doppler dimension and range-Doppler (RD) dimension is extracted by way of cell averaging, ordered statistics, censored mean and trimmed mean. Finally, DTC is used to recognize “true” and “false” targets in feature space. Experimental results show that, under the same number of detection targets, the DTC method is superior to traditional CFAR methods, and the accuracy of target detection can be increased by more than 5%.

1. Introduction

HFSWR can realize over-the-horizon monitoring of “soft targets” (wind, wave, current) and “hard targets” (ship and low-altitude aircraft) by using the characteristics of short-wave diffraction propagation. Therefore, it has received significant attention in most countries. At present, HFSWR mainly uses two kinds of antennas: phased antenna array and compact antenna [1,2]. The HFSWR with compact antenna, which has been widely used, has the advantages of low cost, small footprint, and all-weather and large-scale monitoring [3,4,5]. However, the SNR of the ship echo is very low due to the impact of sea and land clutter, the disturbance of the target radar cross section, and external interference, which makes the detection of such targets extremely difficult [6,7]. Therefore, how to improve the detection performance of a ship target in a complex background is one of the major challenges. Currently, there are several representative research methods in HFSWR target detection.

(1) CFAR method based on clutter background model. The CFAR method can automatically adjust the decision threshold to achieve adaptive detection. Based on the mean value of clutter samples on both sides of a cell under test (CUT), a cell averaging (CA)-CFAR method was proposed by Finn [8]. This method has the best detection performance under the uniform background environment. However, ship target detection in HFSWR often suffers complex situations, such as multi-target and clutter edge. Therefore, Hansen and Trunk proposed the greatest of (GO)-CFAR and the smallest of (SO)-CFAR to improve the performance of the CA method for the non-uniform clutter environment. However, they can only solve either multi-target or clutter edge [9,10]. Inspired by image median filtering technology, Rohling et al. [11] proposed an ordered statistics (OS)-CFAR method. OS-CFAR performs better in multi-target detection than CA-CFAR, and its detection ability was moderate under a uniform environment. Subsequently, some improved methods are proposed based on OS-CFAR, such as: censored mean level detector (CMLD)-CFAR [12], trimmed-mean (TM)-CFAR [13], automatic censored mean level detector (ACMLD)-CFAR [14], two dimensional (2D)-OS-CFAR [15], etc. To deal with a complex background, some adaptive methods have also been proposed. Heterogeneous clutter estimate (HCE)-CFAR estimates clutter the intensity by adjusting the separation points of two clutter regions in the reference window, and obtains the adaptive detection threshold [16]. Variability index (VI)-CFAR realizes the adaptive detection by selecting one of CA, GO, and SO to estimate clutter intensity according to the mean value ratio of the clutter in the leading and lagging reference windows and the variable index [17]. In recent years, for non-uniform and heterogeneous clutter environments, some researchers have proposed first order difference (FOD)-CFAR, second order difference (SOD)-CFAR, and adaptive 2D-OS-CFAR according to the statistical characteristics of clutter in the reference window [18,19,20].

(2) Detection method based on geometric feature difference of target and clutter. In a range Doppler (RD) image, the Bragg region along the range dimension is regarded as a linear shape, and ship target as a point. Grosdidier et al. [21,22] uses morphological component analysis to suppress sea clutter and separate target signals. The simulation results show that this method can effectively detect targets and has broad application prospects. However, sea clutter is not strictly of a linear shape, but rather a columnar shape which is broadened and decayed with distance. Therefore, it is difficult for the method to accurately separate sea clutter and easy to remove the true target signal. In an RD image, Jangel et al. [23] regards sea clutter as a low-frequency component and a target as a high-frequency component, and used the wavelet decomposition and reconstruction technology to suppress sea clutter, and then obtained the ship target signal with a single point characteristic. The disadvantage is that it is difficult to select the appropriate wavelet and the number of wavelet decompositions cannot be adjusted adaptively, so the scope of use is limited. For the shortcomings of [23], Li et al. [24] adaptively determined the wavelet transform scale by means of peak SNR and then used the fuzzy set algorithm to enhance the high frequency components of target signals to separate the sea clutter and target. Before using the wavelet decomposition to suppress sea clutter, Lu et al. [25] firstly carried out the principal component analysis (PCA) of the coherent echo signal and then reconstructed the RD spectrum. The experimental results showed that the SNR of the target could be increased by more than 10dB, which improved the detection ability of the method on weak targets. In the time-frequency (TF) domain, the ship target has long duration and high energy concentration, while noise is chaotic and of short duration. Based on the TF characteristics of target signals, Cai et al. [26] separated the TF ridges on the TF plane by the image segmentation method to realize the detection of weak targets. Then, on the basis of [26], Yang et al. [27,28] proposed the TF-CFAR method by combining the advantages of TF and the image processing method, which achieved better detection performance in multiple targets, clutter edge targets and weak targets for HFSWR.

(3) Target detection method based on learning classification. With the popularity of machine learning, the application of the learning classification method to HFSWR target detection has attracted the attention of some researchers. Based on the feature difference of target, clutter and interference in RD images, Zhang et al. [29] established a lightweight faster region-based convolutional neural network to improve the performance of HFSWR. Next, Zhang et al. [30] also proposed a kind of semi-supervised self-distillation algorithm (S3D) to classify and locate targets. It uses a higher false alarm rate to locate candidate targets. Next, the S3D algorithm is used to retain real targets and remove false targets. However, the detection precision of this method is influenced by a number of training parameters. Based on the convolutional neural network-extreme learning machine (CNN-ELM) model, Wu et al. [31] proposed a two-stage cascade detector for HFSWR. After that, on the basis of [31], Wu et al. [32] proposed an intelligent detection algorithm that includes two states: preprocessing and target detection. Firstly, the clutter region is identified and located by preprocessing, and then a two-level classifier is used to distinguish the target signals. This method has a good recognition effect on targets in the clutter region. However, for the target signal with a low SNR, it is difficult to distinguish the target features. Consequently, the detection performance of this method is poor. Ji et al. [33] used a superior deep learning algorithm and an adaptive dictionary learning algorithm to separate a variety of unwanted echo signals. In his work, the target signal obscured by clutter was identified and good detection results were obtained. The results of the above methods prove the feasibility and effectiveness of introducing an intelligent learning method into HFSWR target detection.

One disadvantage of the traditional CFAR method is that the estimation of background noise only adapts to a single scenario. For example, CA is suitable for uniformly distributed background noise and OS is suitable for background noise with large fluctuations. Fortunately, DTC can integrate multiple clutter intensity estimation methods through multi-dimensional feature extraction, such as range dimension, Doppler dimension, RD dimension, sample adjacent units, etc. to obtain a more flexible and stable detection than the traditional CFAR method. By using a supervised machine learning method, this paper proposes a method of ship target detection based on DTC for HFSWR. Compared with traditional CFAR methods, the DTC method has more obvious advantages in target detection performance.

The remainder of this paper is organized as follows. Section II introduces the principle and process of the DTC method. Section III introduces the methods of data sample acquisition and feature extraction. Section IV gives the experimental results. Section V includes the discussion, and Section VI obtains a brief conclusion.

2. Detection Method

The process of the radar-target detector to make “true” and “false” judgements on a signal is similar to the process of a machine learning algorithm for the binary classification of input samples. Based on the tree structure, decision tree can make decisions on the complex problems, which can realize the classification task of HFSWR targets, as shown in Figure 1.

A decision tree consists of a root node, several inner nodes, and leaf nodes. The critical step in the process of generating the decision tree is how to optimistically divide the attributes, that is, the samples contained by the branch nodes belong to the same category as far as possible, and the “purity” of the nodes is getting higher and higher. Information entropy is the most commonly used indicator to measure the purity of a sample set [34]. Suppose that the proportion of the k-th sample in the current training sample set D is p_k (k = 1, 2, …, M), then the information entropy of D is defined as

$$\mathrm{Ent}(D)=-{\displaystyle \sum _{k=1}^M{p}_k{\log }_2{p}_k},$$

(1)

The smaller the Ent(D) indicates the higher purity of D. Assuming that the discrete attribute $a$ has V possible values $\left\{a^1,a^2,\cdots ,a^V\right\}$, is used to divide the sample set D, and V branch nodes are generated. The v-th branch node contains all the samples in D whose value is $a^v$ on attribute $a$, denoted as D^v. According to different branch node including the different number of sample, the weight of the branch node is given by |D^v|/|D|, and the information gain obtained by the division of the attribute $a$ to the sample set D can be calculated.

$$\mathrm{Gain}(D,a)=\mathrm{Ent}(D)-{\displaystyle \sum _{v=1}^V\frac{\left|D^v\right|}{\left|D\right|}}\mathrm{Ent}(D^v),$$

(2)

The greater the information gain, the greater the purity improvement achieved by using attribute $a$ for division. Therefore, information gain is used to select the attributes of the decision tree.

$$a_{*}=\underset{a\in A}{\arg \max }\mathrm{Gain}(D,a),$$

(3)

$a_{*}$ represents the optimal partition attribute of the current node, and A is the set of training sample attributes. Through recursion and attribute division, until the sample set contained by the current node is empty and cannot be divided any further, a decision tree with strong generalization ability is finally generated. In training a classification tree, the operating environment is Matlab 2016a [35]. The values of the tree depth controllers for growing classification trees are the maximum number of splits, minimum leaf size and minimum parent size. They are set to 100, 1, and 10, respectively. The processing flowchart of HFSWR target detection based on DTC is shown in Figure 2.

3. Sample Acquisition and Feature Extraction

3.1. Sample Acquisition

After pulse compression and coherent integration, ship target echo presents as a 2D sinc function in the RD spectrum, and has a small amount of expansion in both range and Doppler dimension, showing a peak shape similar to a cone. Range, speed, heading and azimuth information of a ship obtained by AIS can be converted to the RD spectrum through mapping (see Figure 3a), but the position of the AIS record after mapping may not exactly fall on the peak point (see Figure 3b). The AIS record in the red dashed box in Figure 3a corresponds to that in Figure 3b, and is located at the 17th range bin. Therefore, in order to increase the reliability of positive samples, the intersection of peak detection and CFAR detection is used to match with AIS targets to obtain positive samples labeled as +1. The criterion of matching is no more than one upper and lower range resolution unit, and no more than three Doppler resolution units on the left and right. Negative samples were randomly selected in the region without AIS targets and labeled as −1. In Figure 3a, it can be seen that the radial velocity of some ships similar to ocean current velocity fall into the first-order Bragg region, and are completely shielded. When taking samples, such ship target signals are not considered. Meanwhile, AIS target samples in the zero-frequency region are not considered.

3.2. Feature Extraction

Build a sample set {x_k|y_k}, k = 1, …, M, where x_k is the feature vector of the k-th sample. y_k represents the corresponding sample label, “+1” represents the positive sample, and “−1” represents the negative sample. For each sample, its feature vector contains the following three types of dimensions (see Figure 4).

(a)

Range dimension: take six reference units respectively from outside of the above and below protection unit;

(b)

Doppler dimension: take five reference units respectively from outside of the left and right protection unit;

(c)

RD dimension: select 5 × 5 data centered on the CUT, eight units in the middle layer as protection units, and 16 units in the outer layer as reference units.

Suppose the power of target is S, the sample of reference unit is R_i, i = 1, …, 2n, consisting of the leading and lagging sliding window. For the above three dimensions, the CA, OS, CMLD and TM methods are used to calculate the feature values of target SNR, which are described below.

(1)

The CA method selects the average power of all reference units as the estimation of clutter power and calculates target SNR as

$$x_{ca}=10\lg \frac{S}{1/2n{\displaystyle \sum _{i=1}^{2n}{R}_i}},$$

(4)

(2)

The OS method sorts all reference units according to the intensity of clutter power, and selects the 9th, 7th and 11th as the estimation of clutter power in range dimension, Doppler dimension, and RD dimension, respectively, to calculate target SNR as

$$x_{os}=10\lg \frac{S}{R_i},i=7, 9, 11,$$

(5)

(3)

The CMLD method sorts all reference units according to their power intensity, and then deletes r large reference values starting from the maximum value, where r is 2. The average value of the remaining reference units is used as the estimation of clutter power to calculate target SNR as

$$x_{cmld}=10\lg \frac{S}{\frac{1}{2n-r}\left({\displaystyle \sum _{i=1}^{2n-r}{R}_i}\right)},$$

(6)

(4)

The TM method sorts all reference units according to power intensity, eliminating r₁ smaller from the minimum value and r₂ larger from the maximum value, where r₁ and r₂ are taken as 1. The average value of the other reference units is used as the estimator of clutter power to calculate target SNR as

$$x_{tm}=10\lg \frac{S}{\frac{1}{2n-r_1-r_2}\left({\displaystyle \sum _{i=r_1+1}^{2n-r_2}{R}_i}\right)},$$

(7)

By using four methods to calculate the feature values of target SNR in three dimensions, it is able to integrate clutter samples with different dimensions and various methods to estimate clutter intensity, so as to achieve “true” and “false” target discrimination in the feature space, thereby improving the capability of radar target detection. The cross-loop/monopole (CLM) antenna has three channels to receive echo data, and each channel is traversed for feature extraction. The 36-dimensional feature values can be obtained cumulatively. Finally, the complete training sample set can be obtained by combining x_k with its label.

3.3. Feature Analysis

Figure 5 shows the distribution of the features for different samples collected in one day, in which the ratio of positive and negative samples is 1:1. By adjusting false alarm probability, sample sets of different sizes can be obtained. When the number of samples is equal to 3500, the values of feature 7 and 9, as well as feature 5 and feature 25, can be almost completely separated, and a very small number of positive and negative samples are overlapped together, as shown in Figure 5a,b. When the number of samples is equal to 7000, most negative samples of feature 7 and feature 9, feature 5 and feature 25, overlap with the positive samples, and the degree of feature separation is relatively poor, as shown in Figure 5c,d. It can be inferred that the larger proportion of positive samples with low SNR will (negative samples selected from non AIS targets in long-distance) result in the poor discrimination between positive and negative sample features. Therefore, the trained model will generate more false alarms during prediction in this situation, and the detection probability will decline.

4. Experimental Results

The experiment uses the radar data collected by the Ocean State Monitoring and Analyzing Radar, type SD (OSMAR-SD), designed by Wuhan University, from September to November [36]. The AIS data used in this experiment is synchronized with the radar data. The radar is located in Dongshan, Fujian, China. The radar parameters are shown in

Table 1. HFSWR parameters.

Parameter	Value
Carrier frequency (MHz)	13.15
Sweep band (kHz)	60
Range resolution (km)	2.5
Velocity resolution (m/s)	0.0825
Receive antenna	Cross-Loop/Monopole
Sweep cycle (s)	0.54
Coherent integration time (CIT) (s)	138.24

.

Among CFAR processing methods, mean level, ordered statistics and adaptive CFAR are relatively representative radar target detection methods with good detection performance in practical applications. Therefore, 2D-CA-CFAR, 2D-OS-CFAR, 2D-VI-CFAR, and 2D-FOD-CFAR are chosen for comparison. Compared with OS-CFAR, 2D-OS-CFAR means that the form of the reference window is two-dimensional, namely the rectangular reference window in Figure 4. Other comparison methods also refer to this meaning. Under the condition that the number of detected targets remains the same within one day, AIS targets, 2D-OS-CFAR targets and DTC targets within a field of radar data are marked respectively in Figure 6. DTC and 2D-OS-CFAR do not consider the ship targets, which are submerged by the Bragg region. The DTC method cannot only detect most 2D-OS-CFAR targets, but also detects other suspected targets, which have more matching pairs with AIS targets, as shown in the black dashed box in Figure 6.

4.1. Matching Rate

The matching rate shown in Figure 7 is obtained by classifying the echo data of seven days using the model trained by the data collected on 27 September. Here, the matching rate refers to the ratio of the matched target to the number of detected targets. It can be seen from Figure 7 that the matching rate of the DTC method is more than 5% higher than for the other four CFAR methods, and has same changing trend with them, which indicates that the DTC method can integrate the advantages of multiple estimation methods of clutter intensity by feature extraction, thus improving the ability of radar target detection.

Table 2. Total number of matched targets and average matching rate.

Method	Total Matched Targets	Total Detected Targets	Average Matching Rate (%)
2D-CA-CFAR	28,404	78,883	36.00%
2D-OS-CFAR	28,593		36.24%
2D-VI-CFAR	28,353		35.94%
2D-FOD-CFAR	27,312		34.62%
DTC-Method	33,184		42.06%

shows the summary of the detected and matched targets of the seven-day radar data. For the four CFAR methods, 2D-OS-CFAR has the best detection performance, and 2D-FOD-CFAR has the worst. The average matching rate of the DTC method is 5.82% higher than that of 2D-OS-CFAR, and 7.44% higher than that of 2D-FOD-CFAR. The above results indicate the superiority of the DTC method.

4.2. SNR

When the SNR of target is lower than 10 dB, Figure 8 compares the number of matched targets of the DTC method with the 2D-OS-CFAR method over 7 days. As shown in Figure 8, the DTC method is superior to the 2D-OS-CFAR method in weak target detection, and the number of matched targets under the DTC method is 2.3~3.4 times that of the 2D-OS-CFAR method. The results show that the DTC method can overcome the shortcomings of the single background noise calculation in the CFAR method and improve the performance in weak target detection through the fusion of multiple features.

4.3. Target Association

In HFSWR target detection, the radial velocity of the ship target generally changes slowly. Therefore, in the time of continuous multi field data, the velocity change of the ship target will form a continuous trajectory. For target association, the distance threshold is 2 km, and the speed threshold is 0.4 m/s. If the target cannot be associated for three consecutive fields, the trajectory will be terminated. Figure 9 shows the associated dots of the matched target of the 2D-OS-CFAR and DTC methods on 26 September. Different trajectories in Figure 9 are shown in different colors. The dots of the same color represent a possible target. From the density of trajectories in Figure 9, it can be found that the DTC method has more continuous trajectories than the 2D-OS-CFAR method, in which the DTC method has 245 trajectories with an average length of 17, while the 2D-OS-CFAR method has 204 trajectories with an average length of 16. The results show that the detection performance of the DTC method is better than that of the 2D-OS-CFAR method.

Table 3. The results of the associated trajectories from the matched target using five methods.

Method	Number of Associated Trajectories	Average Length
2D-CA-CFAR	186	15.65
2D-OS-CFAR	205	16.68
2D-VI-CFAR	186	15.30
2D-FOD-CFAR	194	16.88
DTC-Method	245	16.62

shows the results of the associated trajectories of the matched targets for the five methods. Under the same number of detected targets, the number of trajectories from the matched targets by the DTC method is 31.7%, 19.5%, 31.7% and 26.2% more than that of the 2D-CA-CFAR, 2D-OS-CFAR, 2D-VI-CFAR and 2D-FOD-CFAR, respectively, and the average length is longer than 2D-CA-CFAR and 2D-VI-CFAR, and shorter than 2D-OS-CFAR and 2D-FOD-CFAR. The results of the associated trajectories show that the detection performance of the DTC method is superior to these CFAR methods.

4.4. Generalization Ability

In order to further study the generalization ability of the DTC method, the prediction model trained by the radar data acquired on 27 September is applied to the data of 4 October, 5 October, 1 November, and 2 November for target classification. Meanwhile, when obtaining training samples, by adjusting the number of matching targets, the classifier with the function similar to CFAR is realized. The relationship between the number of detection targets and the matching rate is shown in Figure 10. The number of detection targets in the horizontal axis from less to more corresponds to the number of training samples of 2160, 3228, 4072, 4892, respectively. The detection performance is similar over four days, which demonstrates that the DCT method is robust with the measured radar data on different dates. As the number of detection targets increases, the matching rate decreases gradually. The reason for this is that with the big increase in the number of detection targets, the number of false targets is far more than the number of AIS targets, resulting in an overall low matching rate for all methods. As can be seen from Figure 10, the detection performance of the DTC method is better than that of the other four CFAR methods, which indicates that the DTC method has generalization ability.

5. Discussion

In practice, if needed, other estimation methods of clutter intensity can be extended to extract features of training samples, but the more features that are used, the more time that is taken to train the classifier and predict new data. Therefore, robust features can be selected for model training by field test results. Using an Intel i5-9400 CPU, the proposed DTC method takes 281.47 s to process the first 30 range bins of the 617 field measured data obtained on 26 September 2015. The mean time of processing of each field data is about 0.46 s. The detection time of the DTC method refers to the time taken by the prediction stage after model training was completed. In addition, a large proportion of low SNR targets included in the training samples will result in low feature discrimination, which further leads to the increase in false alarms in prediction results. The experimental results show that the number of prediction targets corresponding to the number of training samples of 2000–5000 is 9000–65,000, and the appropriate number of training samples can be selected for adjusting false alarms in a real-time application. When the number of training samples is equal to 2160,

Table 4. Experimental results of the DTC method with PCA and without PCA.

Time (Day/Month)	26 September			27 September
Method	Detected Number	Matched Number	Matching Rate (%)	Detected Number	Matched Number	Matching Rate (%)
DTC with PCA	26,198	7292	27.83	21,345	5196	24.34
DTC without PCA	13,014	5509	42.33	8351	3285	39.33

shows the experimental results of the DTC method with PCA and without PCA over two days. After PCA keeps enough components to explain 95% variance, the number of targets detected by the DTC method increased by more than half, and the matching rate decreased by nearly 15%. One possible reason is that the PCA reduction dimension will lose part of the information which can precisely distinguish the sample space. After losing the information, the sample space cannot be sufficiently distinguished and the accuracy of the detection results declines.

6. Conclusions

By using prior AIS information to obtain training samples set and calculating the SNR feature of target in the range dimension, Doppler dimension and R-D dimension based on the CA, OS, TM, CMLD methods, the DTC method is applied to HFSWR target detection. By adjusting the number of training samples, the DTC method can realize target detection similar to CFAR. The matching rate of the DTC method is more than 5% higher than that of the other four CFAR methods, the number of weak targets matched with AIS is more than 2.3 times higher than that of the four CFAR methods, and the number of associated trajectories is 19% more than that of the four CFAR methods. The experimental results show that the DTC method has good generalization ability by integrating multiple features and that it can improve the detection accuracy of a single CFAR method and the ability to detect weak targets.