A Vehicle Target Recognition Algorithm for Wide-Angle SAR Based on Joint Feature Set Matching

Abstract

Target recognition is an important area in Synthetic Aperture Radar (SAR) research. Wide-angle Synthetic Aperture Radar (WSAR) has obvious advantages in target imaging resolution. This paper presents a vehicle target recognition algorithm for wide-angle SAR, which is based on joint feature set matching (JFSM). In this algorithm, firstly, the modulus stretch step is added in the imaging process of wide-angle SAR to obtain the thinned image of vehicle contour. Secondly, the gravitational-based speckle reduction algorithm is used to obtain a clearer contour image. Thirdly, the image is rotated to obtain a standard orientation image. Subsequently, the image and projection feature sets are extracted. Finally, the JFSM algorithm, which combines the image and projection sets, is used to identify the vehicle model. Experiments show that the recognition accuracy of the proposed algorithm is up to 85%. The proposed algorithm is demonstrated on the Gotcha WSAR dataset.

1. Introduction

Synthetic Aperture Radar (SAR) has excellent imaging ability, which can obtain high-resolution image under bad weather conditions. The research on SAR includes imaging, target detection, target recognition, time-frequency analysis, and many other aspects [1,2,3,4,5,6,7,8]. The research on Automatic Target Recognition (ATR) of SAR started around 1990 [9,10,11]. ATR is still one of the most challenging research topics. Many researchers have conducted excellent research on SAR ATR [12,13,14].

Wide-angle Synthetic Aperture Radar (WSAR) is a type of the Synthetic Aperture Radar (SAR). Franceschetti et al. introduced the acquisition model and algorithms for the precision processing of SAR data with a large slant range dimension in the early 1990s [15,16]. Different researchers have studied the imaging methods and scattering center models of WSAR [17,18,19,20,21,22,23,24]. Moses et al. gives a definition of WSAR, which refers to any synthesized aperture whose angular extent exceeds the sector that is required for equal resolution in range and cross-range [25]. In 2007, Air Force Research Laboratory (AFRL) released the first Gothca dataset and challenge problem [26]. Subsequently, Dungan et al. proposed several discrimination algorithms for the challenge problem [27,28,29]. In 2012, Dungan et al. worked with AFRL to process the Gothca dataset. They tailored and separated the small data of 33 civilian vehicles from the big scene, and numbered them according to vehicle type and radar detection height, respectively. They published Target Discrimination Research Challenge and the corresponding data subset [30]. Subsequently, different researchers have studied WSAR target recognition [31,32,33,34,35].

The idea of combining imaging and target detection has been studied in the field of infrared target detection [36,37,38,39,40]. This paper introduces an imaging method that is more beneficial in target recognition. Based on the backprojection imaging algorithm, the modulus stretch transform function is added to the imaging process. The image after processing has thinner contour edges than that obtained from the traditional backprojection algorithm. After obtaining contour image, the gravitational-based speckle reduction algorithm is used to obtain a clearer contour image. Subsequently, the image is rotated to obtain a standard orientation image. Accordingly, the image and projection feature set can be extracted from these processed images. Finally, the joint feature set matching (JFSM) algorithm is used to identify the vehicle model. Figure 1 displays an outline of the proposed algorithm.

2. Contour Thinning Imaging

The contour thinning imaging algorithm is an improved algorithm based on the backprojection algorithm. The principle and steps of contour thinning imaging algorithm are described in detail in Reference [35]. In this paper, the main steps of the algorithm are given below.

According to the principle of SAR detection, the radar transmits a linear continuous frequency modulation wave to the target and it receives the echo by matched filter. Radar echo data are functions of receiving frequency f and slow time $\tau ,$ which is denoted by $S(f,\tau )$. In practice, echo data are discretized data, so they are often rewritten as $S(f_k,{\tau }_n)$, where $f_k$ denotes the k-th sampling frequency and ${\tau }_n$ denotes the n-th sampling slow time.

• The $N_{fft}$-point IFFT transform is performed on the echo data $S(f_k,{\tau }_n)$ received at each slow time ${\tau }_n$ to obtain an M-point inverse transform sequence, and it is then cyclically shifted with the FFTshift (•) function.

  $$s_0(m,{\tau }_n)=\mathrm{FFT}\mathrm{shift}\left\{\mathrm{IFFT}\left[S(f_k,{\tau }_n)\right]\right\}$$

  (1)
• Let $\mathbf{r}=(x,y,z)$ denote the target coordinates in the detection scene. Let $\Delta R(\mathbf{r},\tau )$ denote the difference between the distance from the sensor to the origin and the distance from the sensor to the target with the coordinate r at time $\tau .$ Linear interpolation is performed on $s_0(m,{\tau }_n)$ to obtain the estimated values of the corresponding IFFT transformation at all $\Delta R(\mathbf{r},{\tau }_n)$ points and the sub-imaging data at each time ${\tau }_n$ is obtained by multiplying the data with the compensation term.

  $$s_{int}(\mathbf{r},{\tau }_n)=\mathrm{Interp}[s_0(m,{\tau }_n)]\left(\frac{+j4\pi f_0\Delta R(\mathbf{r},{\tau }_n)}{c}\right),$$

  (2)

  $$S_{int}({\tau }_n)=\left\{s_{int}({\mathbf{r}}_{x,y},{\tau }_n)\right\}|x,y\in [1,L],$$

  (3)

  where L denotes the real side length of the imaged scene. Let $d_a(\mathbf{r},{\tau }_n)$ denotes the distance of the antenna from the origin of the ground at coordinate r and the time ${\tau }_n$. Let $d_{a0}(\mathbf{r},{\tau }_n)$ denotes the distance of the antenna from the target in the ground. $\Delta R(\mathbf{r},\tau )$ is calculated, as follows.

  $$\{\begin{array}{l}\Delta R(\mathbf{r},{\tau }_n)=d_{a0}(\mathbf{r},{\tau }_n)-d_a(\mathbf{r},{\tau }_n)\\ d_{a0}(\mathbf{r},{\tau }_n)=\sqrt{{(x_a({\tau }_n)-x)}^2+{(y_a({\tau }_n)-y)}^2+{(z_a({\tau }_n)-z)}^2}\\ d_a(\mathbf{r},{\tau }_n)=\sqrt{x_a^2({\tau }_n)+y_a^2({\tau }_n)+z_a^2({\tau }_n)}\end{array}$$

  (4)
• Let $\theta$ denote the value of the synthetic aperture. The sub-images of all ${\tau }_n$ in the range of the synthetic aperture are superimposed, and the modulus is then stretched for each sub-aperture image by using the $\psi (·)$ function. Finally, all of the sub-aperture images are superimposed to obtain the final image I.

  $$I={\displaystyle \sum _{{\theta }_i}\psi \left({\displaystyle \sum _{{\tau }_n\in \theta }{S}_{int}({\tau }_n)}\right)}$$

  (5)

  $$\psi (x)=\{\begin{array}{lll}{k}_{1}x,& & \left|x\right|\ge T\\ k_{2}x,& & \left|x\right|<T\end{array},$$

  (6)

  where T is the threshold value, $k_1$ is the enhancement coefficient, and $k_2$ is the inhibition coefficient.

Figure 2 shows an example of imaging results for four different models. Figure 2a–d are the imaging results of using the traditional backprojection algorithm for four models of vehicles, namely Chevrolet Impala LT, Mitsubishi Galant ES, Toyota Highlander, and Chevrolet HHR LT. Figure 2e,f correspond to Figure 2a–d for the contour thinning imaging result. The synthetic aperture $\theta$ is 10°. According to Reference [35], the parameters of the modulus stretch function were $k_1$ = 1.2, $k_2$ = 0.1, and T = 0.9. It can be seen that the image of all models of vehicles after modulus stretch greatly strengthens the contour features of the target.

3. Speckle Reduction and Rotation Correction

3.1. Speckle Reduction

Although the image after contour thinning is clearer than before, there are still some noise and artifacts. We use the speckle reduction algorithm commonly used in SAR target detection in order to reduce these noises and artifacts. There are many different speckle reduction algorithms that work well in certain scenarios. The gravitation-based speckle reduction algorithm [41,42] has excellent denoising and anti-artifact effects, so it is used to process the above images.

The algorithm treats the relationship between the target pixel and the speckle pixel as a gravitation relationship. Pixels that are near the target pixel are also more likely to be target pixels, which is proportional to their own brightness and the number of target pixels around that pixel, and inversely proportional to the distance between the pixel and other target pixels. Therefore, the concept of gravitation is introduced to describe the interaction between pixels. The two-dimensional SAR image is regarded as a two-dimensional gravitational field. Different pixels represent different objects in the gravitational field. The gravitation-based speckle reduction algorithm is calculated, as follows:

$$G(i,j)={\displaystyle \sum _{r\le R}m\frac{I(i,j)I(k,l)}{r^2}}$$

(7)

$$F(i,j)=mI{(i,j)}^2$$

(8)

$$I^{\prime }(i,j)=G(i,j)+F(i,j)$$

(9)

where I(i, j) and I(k, l) represent the brightness at coordinates (i, j) and (k, l) on image I. $r=\sqrt{{(i-k)}^2+{(j-l)}^2}$, represents the distance between (i, j) and (k, l). R is the radius of gravitational action. The value of R is related to the size of the target. If R is too small, the target will not be enhanced, while, if R is too large, the noise will also be enhanced. From the experimental results, the value of R can be taken as 1/10 of the target size. m is the coefficient of gravitation. In general, the value of m is taken as 1. G(i, j) represents the gravitational pull of the surrounding pixels that are received by the pixel (i, j). F(i, j) represents the internal stress of the pixel (i, j) itself. The final enhanced pixel brightness I’(i, j) is the sum of the gravitation around the pixel and its internal stress.

Figure 3 shows the example of results after the gravitation-based speckle reduction algorithm of Figure 2e–h. In this case, the radius of gravitational action R is 10 and the coefficient of gravitation m is 1.

As can be seen from Figure 3, as compared with the image before processing in Figure 2, the contour of the processed image remains unchanged, but the noise and artifacts are greatly reduced.

3.2. Rotation Correction

In the actual scene, there is no limit to the parking direction of the vehicle, so the vehicle direction angle on the SAR image is also uncertain. The image obtained by backprojection and other traditional imaging algorithms have blurred contour, which is complex and difficult to automatically detect the rotation angle of the vehicle. In the matching algorithm used in Reference [26,27,28,29], vehicle rotation angle is considered as a variable to be added to the equation together with other parameters, and the specific optimization algorithm solves all unknown variables. However, after using the contour thinning imaging and speckle reduction algorithm, clear vehicle contour lines can be obtained, and Hough transform algorithm can be used to detect the contour line, so as to calculate the vehicle rotation angle. The Hough transform is a very common algorithm that is used to detect straight lines in the field of image processing, which is described in detail in Reference [43], and it will not be repeated in this paper.

The contour image is firstly processed by edge detection algorithm to obtain the binarized edge image in order to accurately detect the contour line. Subsequently, the hough transform is used to detect the straight line to obtain the rotation angle of the vehicle. Edge detection operators can use simple operators such as Sobel, which is calculated, as follows:

$$G_x=\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right],G_y=\left[\begin{array}{ccc}-1& -2& -1\\ 0& 0& 0\\ 1& 2& 1\end{array}\right]$$

(10)

$$I_e=\sqrt{{(I*G_x)}^2+{(I*G_y)}^2}$$

(11)

where $G_x$ and $G_y$ are horizontal and vertical Sobel operators, I is the original image, and $I_e$ is the edge image. The symbol * represents a convolution operation.

After obtaining the vehicle rotation angle θ, the image can be corrected by the rotation transformation equation. The point in the original image whose coordinates are (x, y) is changed to (x′, y′) after transformation. The transformation equation is as follows:

$$\left[\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right]=\left[\begin{array}{ll}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right]\times \left[\begin{array}{l}x\\ y\end{array}\right]$$

(12)

It should be noted that the origin is not in the center when the image is taken as matrix data. Therefore, the origin should be moved to the center before rotation and then restored after rotation. Figure 4 shows the three steps of rotation correction while using Figure 3a as an example.

4. Feature Set Extraction and Matching

After the previous processing, the obtained image is used for feature set extraction. It can be seen from Figure 2a–d that the image obtained by the traditional backprojection algorithm is fuzzy, and it is difficult to directly extract the shape, contour, and other features of the vehicle. After contour thinning and speckle reduction, the vehicle contour is much clearer than before, and it is convenient to detect the contour line, calculate the rotation angle of the vehicle, and other contour data. In this paper, a vehicle recognition algorithm that is based on combined matching of contour image feature set and orthogonal projection feature set is proposed.

4.1. Feature Set Extraction

Figure 5 shows an example of the imaging results for four different models of vehicle at different positions and detection heights. These images have been pre-processed by contour thinning, speckle reduction, and rotation correction. Figure 5a–d are images of the vehicle model Chevrolet Impala LT. Figure 5e–h are images of the vehicle model Mitsubishi Galant ES. Figure 5i–l are images of the vehicle model Toyota Highlander. Figure 5m–p are images of the vehicle model Chevrolet HHR LT.

It can be seen from Figure 5 that different contour sizes of the same model will appear when imaging in different detection locations and different detection heights. Moreover, the contours of different models are sometimes very similar and, in some cases, are more similar than the contours of the same model. Therefore, if only a simple size feature, such as the length or width of the contour, is used, it is difficult to recognize the model of the vehicle. Accordingly, we consider establishing a feature set for each model for recognition. The feature set consists of two parts, one is the selected image data set and the other is the projection data set.

4.1.1. Selected Image Set

Template matching is a simple and direct method for image classification. The matching value can be calculated between the contour image to be recognized and the image in the existing image set and then the category of the image with the largest matching value can then be taken as the vehicle recognition result. Obviously, the more images in the image set and the more representative the images are, the higher the recognition accuracy is. In extreme cases, if the image set contains all of the known images, then the highest recognition accuracy should ideally be achieved. However, in reality, the image set cannot contain all the images, because the actual images are endless, and there will always be new images that have not appeared. Moreover, the image set containing all images loses the significance of studying representative features, and it becomes unnecessarily large and redundant.

Therefore, this paper proposes a simple image set generation algorithm. The algorithm can preserve as many representative images as possible and remove redundant images as possible. Let K denote the total number of known contour images and $I_k$ denote the k-th contour image. Let N denote the number of vehicle models and ${\chi }_i$ indicate the i-th model. For any contour image $I_k$, there is $I_k\in {\chi }_i$, where $k\in [1,K],i\in [1,N]$. Let ${\Phi }_i$ denote the image set corresponding ${\chi }_i$, and the steps for generating the image set are as follows:

• All images belonging to ${\chi }_i$ are added to the image set ${\Phi }_i$, and an image is then randomly selected from ${\Phi }_i$ as the reference image $I_0$.
• The correlation coefficients C of $I_0$ and all other model of images is calculated. The minimum C_min of correlation coefficients C is obtained. The equations are as follows:

  $$C_{\min }=\min (C)$$

  (13)

  $$C=\left\{c_{0,k}|I_0\in {\chi }_i,I_k\notin {\chi }_i\right\}$$

  (14)

  $$c_{i,j}=\max \left\{r_{i,j}(u,v)\right\}$$

  (15)

  $$r_{i,j}(u,v)=\frac{{\displaystyle {\sum }_{x,y}[I_i(x,y)-{\overline{I}}_i]}[I_j(x-u,y-v)-{\overline{I}}_j]}{\sqrt{{\displaystyle {\sum }_{x,y}{[I_i(x,y)-{\overline{I}}_i]}^2}{[I_j(x-u,y-v)-{\overline{I}}_j]}^2}}$$

  (16)

  where $\overline{I}$ is the mean of the I. $r_{i,j}(u,v)$ represents the correlation coefficient of the overlapping region with I_i, when I_j is shift to (u, v), so the mean value is also the mean value of the overlapping region.
• The correlation coefficients of $I_0$ and all of the images in ${\Phi }_i$ are calculated, and the image with the correlation coefficient smaller than $C_{\min }$ is removed from ${\Phi }_i$.
• A new image is randomly selected from ${\Phi }_i$ as the reference image $I_0$, and then Step 2–4 are repeated. If all the images in ${\Phi }_i$ have been reference images, the algorithm ends. Finally, ${\Phi }_i$ is the generated image set.

The image set corresponding to each model can be obtained by following the above steps for all models.

4.1.2. Projection Set

The image set that was obtained after selection process retains some representative images, but many images are also removed, and some features of this model are inevitably lost. In order to compensate for the feature loss, the orthogonal projection feature of the image is saved to increase the recognition accuracy. The horizontal projection and vertical projection have obvious characteristics because the image has been rotated before. Additionally, since the projection data is much smaller than the image data, it is very convenient and cost-effective to store the projection features, and the projection features of all the images can be saved. The horizontal and vertical projection of the image is calculated, as follows:

$$P_y={\displaystyle \sum _{x}I(x,y)}$$

(17)

$$P_x={\displaystyle \sum _{y}I(x,y)}$$

(18)

Figure 6 shows the example of the horizontal projection curve and the vertical projection curve of Figure 5a. The image size is 200 × 200.

The orthogonal projection feature set of each model can be obtained by saving the horizontal and vertical projection data of all images of each model.

4.2. Joint Feature Set Matching

The JFSM algorithm can be carried out after obtaining the image set and projection set. Define the maximum value of the correlation coefficient of two objects as their matching value $\rho$:

$$\rho =\max (\{c_{i,j}\})$$

(19)

Equations (15) and (16) show the calculation of the correlation coefficient $c_{i,j}$. The correlation coefficients between the image to be recognized and each image in the image set of each model was calculated, and the maximum of the correlation coefficients was taken as the image matching value ${\rho }_{img}$ with the model. Similarly, the horizontal and vertical projections of the image to be recognized were calculated, and the correlation coefficients were calculated with the projection sets of each model, and the maximum values of them were respectively taken as the projection matching values ${\rho }_{px}$ and ${\rho }_{py}$. The final overall matching degree ${\rho }_{ov}$ is obtained by the weighted combination of the above three matching values:

$${\rho }_{ov}=\alpha {\rho }_{img}+\left(1-\alpha \right)\frac{{\rho }_{px}-{\rho }_{py}}{2},$$

(20)

where α is the weight, which indicates the importance of image matching value in the overall matching degree. The above equation only calculates the matching degree of one model. In practice, it needs to calculate the matching degree of all models. Finally, the model corresponding to the maximum matching degree is the recognition result of the vehicle.

5. Experimental Results

The Target Discrimination Research subset of the Gotcha dataset that was published by the Air Force Research Laboratory for circular flight observations in a parking lot was used for the experiment. The airborne synthetic aperture radar detects data in a 5 km diameter area on 31 altitude orbits and then extracts phase history data of 56 targets from the large dataset. The targets include 33 civilian vehicles (with repeat models) and 22 reflectors and an open area. The circular synthetic aperture radar provides 360 degrees of azimuth around each target.

In the original data set, some vehicles were moved and rotated, and some of the doors were open during detection. The images of these vehicles were quite different when they were stationary and moving. The imaging of all vehicles in non-stationary state was eliminated in the experiment in order to reduce the imaging interference caused by vehicle changes. Finally, 660 images from 6 models of Chevrolet Impala LT, Mitsubishi Galant ES, Toyota Highlander, Chevrolet HHR LT, Pontiac Torrent, and Chrysler Town & Country were used for recognition experiments. The labels for these six models in the dataset are fcara, fcarb, fsuv, mcar, msuv, and van. The six models of vehicles have 81, 82, 144, 112, 104, and 143 images, respectively.

In the experiment, all of the images were randomly divided into two parts. One part is used as known data to generate model image set and projection set, which is called the training set. The other part is used as unknown data to test the recognition accuracy of the algorithm, called the test set. Let β denote the ratio of the number of images in the training set to the total number of images. Let P denote the recognition accuracy, which is defined as the ratio of the number of images recognized as the correct model in the dataset to the total number of images in the dataset. Let $P_{train}$, $P_{test}$, and $P_{all}$ represent the accuracy of the training set, the accuracy of the test set, and the accuracy of all images, respectively. In the following experiments, each point in the figure is the average of the results of ten randomized trials.

The experiment runs on Matlab. The CPU of the computer is Intel i7 and the memory is 8 GB. The JFSM algorithm takes tens of milliseconds to several seconds to complete a target match, depending on the image size and the number of images in the feature set.

5.1. Results Analysis

In the JFSM algorithm, the model of the recognized vehicle depends on the total matching degree ${\rho }_{ov}$, and the model corresponding to the maximum ${\rho }_{ov}$ is used as the recognition result of the vehicle image. The value of ${\rho }_{ov}$ is calculated by Equation (20) and it is related to the weight α. Figure 7 shows the curves of recognition accuracy P versus weight α. Each point in subsequent figures is the average of 10 randomized experimental results.

As can be seen from Figure 7a, when α is between 0.2 and 0.4, the proposed algorithm has the best recognition accuracy. Therefore, in subsequent experiments, α is 0.3 will be used to calculate ${\rho }_{ov}$.

Figure 8 shows the curve of overall recognition accuracy changing with the training images ratio β at different image size L × L. Figure 9 shows the accuracy of different models changing with β at L = 100. It can be seen from Figure 8 that the recognition accuracy of the test set is between 70% and 85%, the recognition accuracy of the training set is grater than 99%, and the total recognition accuracy is greater than 80%. The recognition accuracy generally increases as β increases. However, the number of images in the test set is too small when β is greater than 0.9, so the accuracy of the test set has a large deviation.

Figure 10 and Figure 11 show the curve of the accuracy changing with the size of the image. Figure 10 shows the overall accuracy of all models. Figure 11 shows the respective accuracy for different models. As can be seen from the figures, when the image size L is as low as 20, the test accuracy rapidly decreases. When L is greater than 40, the accuracy increases slowly. Figure 10 shows that, when β is equal to 0.8, the accuracy reaches a maximum. Figure 9 and Figure 11 show that the mcar model has the highest accuracy and the fcarb model has the lowest accuracy. This experimental result shows that the proposed algorithm can be implemented at a lower image resolution, which can reduce the calculation time.

Figure 12 shows the correctly recognized and incorrectly recognized images of fcara and fcarb in one experiment. Figure 12a–d show the images of fcara. Figure 12e–h show the images of fcarb. As can be seen from the figures, the quality of the imaging results is uneven, some are good, some are poor. Figure 12b,f show the normal images. Figure 12c,g show the imaging distortion that might be caused by the position error of the radar. Figure 12d,h show incomplete imaging due to less than one circle of radar flying around the target.

5.2. Algorithm Comparison

At present, there are few researches regarding vehicle model recognition on the Gotcha dataset. Only a few papers address vehicle target recognition for wide-angle SAR. Dungan et al. proposed a vehicle discrimination algorithm based on point pattern matching, which achieved a recognition accuracy higher than 95% [27,29]. However, the data used by Dungan et al. is different from that in this paper. Dungan et al. used eight groups of altitude orbit data for the same vehicle at the same location for recognition. In this paper, 31 groups of altitude orbit data of different vehicles in different locations are used for recognition. The data that were used in this paper are much more difficult for recognition analysis.

Gianelli et al. proposed an imaging algorithm called data-adaptive Sparse Learning via Iterative Minimization (SLIM), and used the algorithm to achieve 90% recognition accuracy [32]. Gianelli et al. used basically the same dataset as in this paper, but they eliminated all of the flawed data and used only 540 images in total for the recognition experiment. In this paper, only the moving and changing image data of the vehicle is excluded, and a total of 660 images are used for the recognition experiment. No comparative experiment was conducted on the same dataset in this paper since it was impossible to know which image data were excluded from the Reference [32].

6. Conclusions

This paper presents a vehicle target recognition algorithm for WSAR. Based on the backprojection imaging algorithm, the modulus stretch transform function is added to the imaging process. The image after processing has thinner contour edges than that obtained from the traditional backprojection algorithm. After obtaining the contour image, the gravitational-based speckle reduction algorithm is used to obtain a clearer contour image. Subsequently, the image is rotated to obtain a standard orientation image. Hence, the image and projection feature set can be extracted from these processed images. Finally, the image and projection joint feature set matching algorithm is used to identify the vehicle model. The experiment analyzed the relationship between test accuracy, training accuracy and total accuracy with the variation of the parameter α, training images ratio and image size. The experiments show that the recognition accuracy is up to 85%, and the proposed algorithm can be implemented with low image resolution.