Rethinking Representation Learning-Based Hyperspectral Target Detection: A Hierarchical Representation Residual Feature-Based Method

Abstract

Representation learning-based hyperspectral target detection (HTD) methods generally follow a learning paradigm of single-layer or one-step representation residual learning and the target detection on original full spectral bands, which, in some cases, cannot accurately distinguish the target pixels from variable background pixels via one-round of the detection process. To alleviate the problem and make full use of the latent discriminate characteristics in different spectral bands and the representation residual, this paper proposes a level-wise band-partition-based hierarchical representation residual feature (LBHRF) learning method for HTD with a parallel and cascaded hybrid structure. Specifically, the LBHRR method proposes to partition and fuse different levels of sub-band spectra combinations, and take full advantages of the discriminate information in representation residuals from different levels of band-partition. The highlights of this work include three aspects. First, the original full spectral bands are partitioned in a parallel level-wise manner to obtain the augmented representation residual feature through level-wise band-partition-based representation residual learning, such that the global spectral integrity and contextual information of local adjacent sub-bands are flexibly fused. Second, the SoftMax transformation, pooling operation, and augmented representation residual feature reuse among different layers are equipped in cascade to enhance the learning ability of the nonlinear and discriminant hierarchical representation residual features of the method. Third, a hierarchical representation residual feature-based HTD method is developed in an efficient stepwise learning manner instead of back-propagation optimization. Experimental results on several HSI datasets demonstrate that the proposed model can yield promising detection performance in comparison to some state-of-the-art counterparts.

1. Introduction

Hyperspectral remote sensing systems combine imaging and spectral perception technologies to simultaneously obtain abundant spatial and spectral information of the ground objects [1,2,3,4,5]. In practice, the hyperspectral image (HSI) is imaged with 10 s to 100 s of continuous and narrow spectral bands, such as the ultraviolet, visible, and near-infrared bands, which can provide abundant discriminate information for distinguishing ground objects of different materials [6,7,8,9,10,11]. Hyperspectral target detection (HTD) refers to the process of identifying the target pixels from the complex and variable non-target background pixels based on subtle spectral or spatial–spectral combined information [12]. Due to its characteristics, numerous HTD approaches have been developed and have achieved significant performance in many civil and military applications, such as mineral exploration and camouflaged military target detection [13].

The HTD approaches can be generally divided into structured and unstructured background-based approaches. The representative methods for the former category usually characterize HSI with a linear mixture model (LMM) [14], which assumes that each pixel can be approximately represented as a linear combination of several spectral end-members with different fractional abundances. The typical methods include the probability density-based, subspace-based, and linear spectral mixing-based methods, the most classical method of which is the linear mixing-based model [14], such as the constrained energy minimization (CEM) [15,16] and orthogonal subspace projection (OSP) [15]. Different from the structured background-based detection methods, the unstructured background-based detectors characterize the background with a statistical model, such as the multivariate Gaussian distribution, and then build a target detector using binary hypothesis testing. Some representative methods include generalized likelihood ratio test (GLRT) [17], the adaptive coherence/cosine estimator (ACE) [18], and adaptive matched filter (AMF) [19].

To characterize the complex variations of target and background spectra, some representation-based target detection methods have been widely studied. For example, the sparse representation-based target detector (SRD) proposes to represent a testing pixel as a sparse linear combination of limited samples from the target and background spectra combined dictionary [20]. Whether the testing pixel is a target pixel or not is examined through comparing the representation residuals yielded by the target and background sub-dictionaries. Some variations of SRD have also been developed, such as the binary-class collaborative representation-based target detector (BCRD) [21], and the sparse and dense hybrid representation-based target detector (SDRD) [22]. By comparison, the collaborative representation-based methods, which finds a minimum l₂-norm regularized representation coefficient solution, have lower computational efficiency due to the existence of closed-form analytical solution, while the spare representation learning is solved as a l₀-norm regularized minimization problem with complex computation.

Existing representation learning-based HTD methods, such as collaborative or sparse representation-based methods, are rooted in the linear subspace theory, which is flexible to characterize and model the variability of HSI spectra [23,24,25]. Therefore, the representation residual contains some key discriminate information to indicate the label of a query pixel. However, these methods all perform in a learning paradigm of single-layer or one-step detection on the original full spectral bands, and the subtle discriminant information in the representation residual might not be fully discovered through such a one-round shallow learning strategy, which will restrict the detection performance. Recently, hierarchical learning has achieved great success in HSI applications. A key reason is that hierarchical models can allow the layered nonlinear transformation of data to reveal the potential subtle and discriminate features therein. Useful discriminate information that is beneficial for the final learning purposes will be successively discovered and accumulated through such a hierarchical learning strategy [26,27,28]. Inspired by the core ideas of multi-layer hierarchical learning and representation residual learning-based HTD methods, this paper proposes to discover and augment the discriminate target detection information from multiple levels and layers of representation residuals for hierarchical representation residual feature learning. Accordingly, a level-wise band-partition-based hierarchical representation residual feature (LBHRF) learning method is developed in this paper for HTD, and two key modules are carefully devised, including the augmented representation residual feature based on level-wise band-partition (ARRFLB) and the augmented representation residual feature reuse and relearning (ARRFRR). Figure 1 illustrates the diagram of the proposed LBHRF learning method with L levels of band-partition and K layers of hierarchical representation residual feature for HTD. The augmented representation residual feature of a test pixel is first obtained using the ARRFLB module from Level 0 to Level L. Then, several ARRFRR modules are incorporated in cascade to obtain the re-augmented representation residual features from Layer 1 to Layer K, and the residual discrimination information will be successively enhanced via representation residual feature reuse and relearning. In this way, the re-augmented representation of the residual feature in the last layer is fed into the terminal representation-based target detector (RTD) to calculate the hierarchical representation residual and detection value to determine whether the test pixel is a target or not. The implementation details for ARRFLB, ARRFRR, and RTD will be introduced in the following sections. Different from the existing representation-based HTD approaches, the main contributions of this paper are summarized as follows.

(1)

A Level-wise band-partition-based hierarchical representation residual feature (LBHRF) learning method for HTD is devised in a stepwise training manner without the need for back-propagation optimization. The SoftMax transformation, pooling operation, and augmented representation residual feature reuse and relearning among different layers are incorporated with cycle accumulation to enhance the nonlinear and discriminate feature learning capability of the method;

(2)

To flexibly integrate the global spectral integrity as well as the local contextual information of adjacent sub-bands, the original full spectral bands are partitioned into different levels with bands overlapping, and the augmented representation residual feature is then obtained by concatenating different levels of representation residual features;

(3)

Due to computational efficiency, the collaborative representation with the minimization of the l₂-norm regularized representation coefficient is used in the experiments, and the results on several HSI target detection tasks show that the proposed method can yield overall superior detection performance.

The rest of this paper is structured as follows. Section 2 describes related work on HTD. Our LBHRF model for HTD is presented in Section 3, including the key modules ARRFLB, ARRFRR, and RTD. The effectiveness of the proposed method is demonstrated by experimental results presented in Section 4. Conclusions are made in Section 5.

2. Related Work

Consider a hyperspectral image having N pixels with $B$ spectral bands, and all the spectra of an HSI are arranged in a B × N matrix as $\mathbf{X}=[{\mathbf{x}}_i, {\mathbf{x}}_2,\dots , {\mathbf{x}}_N]\in {\mathfrak{R}}^{B\times N}$ containing the targets of interest for detection. The combined dictionary with known target and background pixels is denoted as $\mathbf{A}=[{\mathbf{A}}_t,{\mathbf{A}}_b]$, where ${\mathbf{A}}_t$ indicates the prior target spectra, and ${\mathbf{A}}_b$ is the estimated background spectra. The purpose of HTD is to distinguish a label-unknown test pixel $\mathbf{y}\in {\mathfrak{R}}^B$ as a target pixel or not, with the help of target and background dictionary $\mathbf{A}$. With the idea of linear spectral mixing, it is assumed that the test pixel $\mathbf{y}$ can be represented as a linear combination of the pixels in the target and background dictionary $\mathbf{A}$ with representation coefficient vector $\mathit{\alpha }=[{\mathit{\alpha }}_t,{\mathit{\alpha }}_b]$ as follows.

$$\mathbf{y}={\mathbf{A}}_t{\mathit{\alpha }}_t+{\mathbf{A}}_b{\mathit{\alpha }}_b$$

(1)

The prestigious sparse representation-based target detector (SRD) seeks a spare representation coefficient vector $\mathit{\alpha }$, which means the test pixel $\mathbf{y}$ is approximately represented by very few pixels from the combined dictionary $\mathbf{A}$, and the recovered representation residuals can be used for detection. The mathematical formulation for SRD is expressed as follows,

$${\mathit{\alpha }}^{*}=argmin{ ‖\mathit{\alpha }‖}_0\hspace{1em}s.t.\hspace{1em}\mathbf{y}=\mathbf{A}\mathit{\alpha }$$

(2)

where ${‖·‖}_0$ means the l₀-norm defined as the number of nonzero entries of a vector. The nonzero entries of $\mathit{\alpha }$ can help reveal the category of a test pixel $\mathbf{y}$. The l₀-norm minimizing problem (2) is NP-hard, and can be approximately solved by some greedy pursuit algorithms such as orthogonal matching pursuit (OMP) or subspace pursuit (SP) [22]. If the solution is sufficiently sparse, the NP-hard problem can be relaxed into a linear programming one by replacing the l₀-norm with l₁-norm, which can be solved by convex programming techniques. Considering the existence of noise in data, the equality constraint in (2) is relaxed to the following inequality one.

$$\underset{\mathit{\alpha }}{\mathit{argmin}}{ ‖\mathit{\alpha }‖}_0\hspace{1em}s.t.\hspace{1em}{‖\mathbf{A}\mathit{\alpha }-\mathbf{y}‖}_2\le \theta$$

(3)

where $\theta$ is the upper bound for representation error. The above problem can also be formulated as the minimization of the representation error under a certain sparsity level.

$$\underset{\mathit{\alpha }}{\mathit{argmin}}{ ‖\mathbf{A}\mathit{\alpha }-\mathbf{y}‖}_2\hspace{1em}s.t.\hspace{1em}{‖\mathit{\alpha }‖}_0\le K_0$$

(4)

where $K_0$ is a given upper bound on the sparsity level. After solving (3) or (4), the optimal representation $\mathit{\alpha }=[{\mathit{\alpha }}_t,{\mathit{\alpha }}_b]$ will be obtained. Then, the background pixels ${\mathbf{A}}_b$ and target pixels ${\mathbf{A}}_t$ in dictionary $\mathbf{A}$ are used to reconstruct the test pixel by combining their corresponding sub-coefficients ${\mathit{\alpha }}_t$ and ${\mathit{\alpha }}_b$, by which the representation residuals can be obtained as follows:

$$r_b\left(\mathbf{x}\right)={‖\mathbf{y}-{\mathbf{A}}_b{\mathit{\alpha }}_b‖}_2$$

(5)

$$r_t\left(\mathbf{x}\right)={‖\mathbf{y}-{\mathbf{A}}_t{\mathit{\alpha }}_t‖}_2$$

(6)

The smaller the representation residuals under each category of sub-dictionary, the more likely that the test pixel belongs to this category. Therefore, the difference between these two residuals under the two sub-dictionaries is used to calculate the detection value.

$$D_{SRD}\left(\mathbf{y}\right)=r_b\left(\mathbf{y}\right)-r_t\left(\mathbf{y}\right)$$

(7)

Whether the pixel belongs to the background class or target class is determined according to the relationship between the detection value with specific threshold value. If the detection value $D_{SRD}\left(\mathbf{y}\right)$ is greater than specific threshold value, the test pixel will be claimed as target, otherwise it is more likely to be background. The core idea behind the optimization problem (4) is that the test pixel will be represented by limited pixels similar to it under the constraint of representation error tolerance and sparsity. However, it is argued that it is the collaborative representation mechanism, but not the sparsity, that plays the essential role, as formulated below.

$$\underset{\mathit{\alpha }}{\mathit{argmin}}{ ‖\mathit{\alpha }‖}_2\hspace{1em}s.t.\hspace{1em}{‖\mathbf{A}\mathit{\alpha }-\mathbf{y}‖}_2\le \delta$$

(8)

In addition, the l₂-norm regularized collaborative representation mechanism has significantly lower computational complexity. Many variants of SRD based on collaborative representation mechanism have been studied, such as SDRD [22] and sparse representation-with binary hypothesis detector (SRBBH) [23].

It is not difficult to draw the conclusion that the above HTD methods can be summarized as class-specific representation residuals learning and comparison, by using different representation learning strategies. The derived representation residuals contain valuable discriminate information for HTD. However, these methods all perform one-round of representation learning and representation residuals calculation on the original full spectral bands. As a result, it is unable to make full use of the identification ability of different spectral bands for different materials, and the discriminant information implied in the representation residual cannot be sufficiently explored.

3. Level-Wise Band-Partition-Based Hierarchical Representation Residual Feature Learning for HTD

3.1. Parallel Level-Wise Band-Partition

In practice, HSI is obtained by a spectrometer in response to the electromagnetic wave reflected or emitted from the ground object material surface. Different materials have their unique responses to the electromagnetic wave. As a result, using the same full spectral bands to detect targets of different materials cannot make full use of discriminate information of different spectral bands and inevitably suffer from spectral redundancy. To alleviate the problem, this paper proposes to jointly use sub-band combinations based on level-wise band-partition together with the original full bands. For the l-th (l = 0, 1, 2, …, L) level, a total of $2^l$ sub-band combinations will be obtained. For example, the 2-th level of band-partition will have $2^2=4$ sub-band combinations. Level 0 band-partition is the original full spectral bands themselves. For a test pixel $\mathbf{y}$, the 1-th level band-partitions are denoted as ${\mathbf{y}}^1=\left[{\mathbf{y}}_1^1,{\mathbf{y}}_2^1\right]$, and similarly the l-th band partitions are illustrated as ${\mathbf{y}}^l=\left[{\mathbf{y}}_1^l,{\mathbf{y}}_2^l,\dots ,{\mathbf{y}}_{2^l}^l\right]$. ${\mathbf{y}}_j^i$ means the j-the sub-band combination of $\mathbf{y}$ in i-th level. As for the target and background dictionary, ${\mathbf{A}}_j^i$ collects the j-the sub-band combinations of $\mathbf{A}$ in the i-th level. Accordingly, the level l band-partition for the k-th sample of $\mathbf{A}$ is denoted as ${\mathit{a}}^{\left(k\right)l}=[{\mathit{a}}_1^{\left(k\right)l},{\mathit{a}}_2^{\left(k\right)l},\dots ,{\mathit{a}}_{2^l}^{\left(k\right)l}]$. Figure 2 shows an example process of Level 1 double-partition of a HSI pixel. For this level of band partition, all the pixels are partitioned into two parts with several bands overlapping, and all the pixels are partitioned in the same way to construct the band-partitioned target and background dictionaries.

3.2. Augmented Representation Residual Feature Based on Level-Wise Band-Partition (ARRFLB)

All the pixels in the target and background dictionary $\mathbf{A}$ and the test pixel $\mathbf{y}$ are first band-partitioned in the same way. Afterwards, for the j-th band-partition combinations of the i-th level, the corresponding band-partitioned test pixel ${\mathbf{y}}_j^i$ is represented using ${\mathbf{A}}_j^i$ by solving the following problem.

$${\mathit{\beta }}^{*}=\underset{\mathit{\beta }}{\mathit{argmin}}{‖{\mathbf{A}}_j^i\mathit{\beta }-{\mathbf{y}}_j^i‖}_2^2+{\lambda }_1{‖\mathit{\beta }‖}_p$$

(9)

where ${‖·‖}_p$ is the p-norm used to regularize the representation coefficient $\mathit{\beta }$, and p = 0, 1, or 2 is usually adopted for l₀, l₁, or l₂-norm minimization. ${\lambda }_1>0$ is used to balance two terms in the objective function. After solving (9), the target and background sub-dictionaries ${\mathbf{A}}_{b·j}^i$ and ${\mathbf{A}}_{t·j}^i$ are used to calculate the representation residuals of ${\mathbf{y}}_j^i$ as below.

$${\mathrm{r}}_{b·j}^i={‖{\mathbf{y}}_j^i-{\mathbf{A}}_{b·j}^i{\mathit{\beta }}_b‖}_2$$

(10)

$${\mathrm{r}}_{t·j}^i={‖{\mathbf{y}}_j^i-{\mathbf{A}}_{t·j}^i{\mathit{\beta }}_t‖}_2$$

(11)

Afterwards, the above two representation residual values are concatenated as a 2-dimensional representation residual feature, and used to encode the test pixel ${\mathbf{y}}_j^i$ in the j-th band-partition of the i-th level as in (12).

$${\mathbb{y}}_j^i=[{\mathrm{r}}_{b·j}^i;{\mathrm{r}}_{t·j}^i]$$

(12)

To strengthen the discrimination of representation residual feature, the SoftMax operation is used to recompute a 2-dimensional representation residual feature as follows:

$${\mathbb{y}}_j^i=[\frac{e^{-{\mathrm{r}}_{b·j}^i}}{e^{-{\mathrm{r}}_{b·j}^i}+e^{-{\mathrm{r}}_{t·j}^i}};\frac{e^{-{\mathrm{r}}_{t·j}^i}}{e^{-{\mathrm{r}}_{b·j}^i}+e^{-{\mathrm{r}}_{t·j}^i}}]$$

(13)

For the l-th level band-partition, 2^l representation residual feature with SoftMax, i.e., ${\mathbb{y}}_j^l \left(j=1,2,\dots ,2^l\right)$, will be obtained, which is then transformed into one representation residual feature using the max pooling operation.

$${\mathbb{y}}^l=max\left\{{\mathbb{y}}_1^l;{\mathbb{y}}_2^l;\dots ;{\mathbb{y}}_{2^l}^l\right\}$$

(14)

Alternatively, the average pooling operation can be adopted to fuse the information from the $2^l$ band-partitions in l-th level.

$${\mathbb{y}}^l=average\left\{{\mathbb{y}}_1^l;{\mathbb{y}}_2^l;\dots ;{\mathbb{y}}_{2^l}^l\right\}$$

(15)

Afterwards, all the representation residual features processed by the SoftMax, and pooling operations from all the L levels of band-partitions are augmented and concatenated together to obtain a 2(L + 1)-dimensional augmented representation residual feature as follows.

$$\mathbb{y}=\left[{\mathbb{y}}^0;{\mathbb{y}}^1;\dots ;{\mathbb{y}}^L\right]\in {\mathfrak{R}}^{2\left(L+1\right)\times 1}$$

(16)

In addition to encoding the test pixel $\mathbf{y}$ and get its augmented representation residual feature $\mathbb{y}$ based on band-partitioned target and background dictionary as above, all the pixels of target and background dictionary are also encoded on the target and background dictionary itself to get their corresponding augmented representation residual features. Figure 3 shows an example for encoding augmented representation residual feature based on three levels of band-partition, all the pixels are encoded as a 6-dimensional representation residual feature using three levels (L = 2) of target and background dictionary. Therefore, the subtle discriminate information from different sub-band combinations and representation residuals is fused by the 6-dimensionl augmented representation residual feature.

3.3. Augmented Representation Residual Feature Reuse and Relearning (ARRFRR)

After parallel L levels of band-partition, all the pixels in the target and background dictionary A and the test pixel y are represented as a 2(L + 1)-dimensional augmented representation residual feature, which are formulated as $\mathbb{A}\in {\mathfrak{R}}^{2\left(L+1\right)\times N}$ and $\mathbb{y}\in {\mathfrak{R}}^{2\left(L+1\right)\times 1}$, respectively. It is known that the test pixel can be categorized as the class with minimum representation residual in a sparse representation-based target detector. In other words, the representation residuals contain significant discrimination information to distinguish targets from the background. However, the subtle discriminant information in representation residual might not be sufficiently exploited via such a one-round shallow learning strategy. In the following section, an augmented representation residual feature reuse and relearning (ARRFRR) method will be developed to serve as a key module for learning re-augmented representation residual feature.

In Layer 1, the input augmented representation residual feature is acquired as ${\mathcal{A}}^{\left(0\right)}=\mathbb{A}$ and ${\mathcal{y}}^{\left(0\right)}=\mathbb{y}$. The representation coefficient learning of ${\mathcal{y}}^{\left(0\right)}$ on ${\mathcal{A}}^{\left(0\right)}$ is formulated as follows:

$${min ‖{\mathcal{A}}^{\left(0\right)}{\mathit{\varphi }}^{\left(0\right)}-{\mathcal{y}}^{\left(0\right)}‖}_2^2+{\lambda }_2{‖{\mathit{\varphi }}^{\left(0\right)}‖}_p$$

(17)

where ${ ‖·‖}_p$ is the p-norm used to regularize the corresponding representation, and p can be 0, 1, or 2 for l₀, l₁, or l₂-norm minimization. Different value of p will lead to different optimization problem. For example, when p = 2, a closed-form solution can be achieved, as follows.

$${\mathit{\varphi }}^{\left(0\right)}={\left({\left({\mathcal{A}}^{\left(0\right)}\right)}^{-1}{\mathcal{A}}^{\left(0\right)}+{\lambda }_2\mathbf{I}\right)}^{-1}{\left({\mathcal{y}}^{\left(0\right)}\right)}^{-1}{\mathcal{y}}^{\left(0\right)}$$

(18)

After solving (16), the target and background sub-dictionaries ${\mathcal{A}}_t^{\left(0\right)}$ and ${\mathcal{A}}_b^{\left(0\right)}$ are used to calculate the representation residuals of ${\mathcal{y}}^{\left(0\right)}$ as below.

$${\mathrm{R}}_t^{\left(0\right)}={‖{\mathcal{A}}_t^{\left(0\right)}{\mathit{\varphi }}_t^{\left(0\right)}-{\mathcal{y}}^{\left(0\right)}‖}_2$$

(19)

$${\mathrm{R}}_b^{\left(0\right)}={‖{\mathcal{A}}_b^{\left(0\right)}{\mathit{\varphi }}_b^{\left(0\right)}-{\mathcal{y}}^{\left(0\right)}‖}_2$$

(20)

The corresponding representation residual feature with SoftMax is calculated as follows:

$${\mathcal{s}}_{{\mathcal{y}}^{\left(0\right)}}^{\left(1\right)}=[\frac{e^{-{\mathrm{R}}_b^{\left(0\right)}}}{e^{-{\mathrm{R}}_b^{\left(0\right)}}+e^{{\mathrm{R}}_t^{\left(0\right)}}};\frac{e^{{\mathrm{R}}_t^{\left(0\right)}}}{e^{-{\mathrm{R}}_b^{\left(0\right)}}+e^{{\mathrm{R}}_t^{\left(0\right)}}}]$$

(21)

Then the original augmented representation feature and the above SoftMax vector are concatenated to obtain the re-augmented representation residual feature in Layer 1 as follows:

$${\mathcal{y}}^{\left(1\right)}=[{\mathcal{y}}^{\left(0\right)};{\mathcal{s}}_{{\mathcal{y}}^{\left(0\right)}}^{\left(1\right)}]$$

(22)

According to the procedures presented in Equations (17)–(21), all the augmented representation residual features in $\mathbb{A}$ are re-augmented and updated. As shown in Figure 4, with level-wise band-partition, all the pixels are processed to obtain the corresponding re-augmented representation residual features, and construct the re-augmented representation residual feature dictionary. Afterwards, as shown in ①, each augmented representation residual feature is further represented using the representation residual feature dictionary to relearn the representation residual feature. The relearned representation residual feature is recomputed with SoftMax and concatenated with the original augmented representation residual feature by identity map, as shown in ②. Finally, the re-augmented representation residual feature is used to update the re-augmented representation residual feature dictionary as in ③.

The procedures shown in Figure 4 execute in cycle for K layers to derive the final K-th layer re-augmented representation residual feature dictionary ${\mathcal{A}}^{\left(K\right)}=\left[{\mathcal{A}}^{\left(0\right)};{\mathcal{s}}_{\mathcal{A}}^{\left(1\right)};\dots ;{\mathcal{s}}_{\mathcal{A}}^{\left(K\right)}\right]\in {\mathfrak{R}}^{\left(2\left(L+1\right)+2K\right)\times N}$ and ${\mathcal{y}}^{\left(K\right)}=\left[{\mathcal{y}}^{\left(0\right)}; {\mathcal{s}}_{\mathcal{y}}^{\left(0\right)},\dots , {\mathcal{s}}_{\mathcal{y}}^{\left(K\right)}\right]\in {\mathfrak{R}}^{\left(2\left(L+1\right)+2K\right)\times N}$, with $L=0,1,2\dots$ as the levels of band-partition and $K=0,1,2\dots$ as the layers for augmented representation residual feature reuse and relearning, which are further fed into the representation-based target detector (RTD) as follows.

$$\underset{{\mathit{\varphi }}^{\left(K\right)}}{\mathit{min}}{‖{\mathcal{A}}^{\left(K\right)}{\mathit{\varphi }}^{\left(K\right)}-{\mathcal{y}}^{\left(K\right)}‖}_2^2+{\lambda }_2{‖{\mathit{\varphi }}^{\left(K\right)}‖}_p$$

(23)

With the optimal representation coefficient ${\mathit{\varphi }}^{\left(K\right)}$ in K-th layer, the final hierarchical detection value (HDV) in the output of the K-th layer is calculated as shown below.

$${\mathrm{H}\mathrm{D}\mathrm{V}}^{\left(K\right)}={‖{\mathcal{A}}_b^{\left(K\right)}{\mathit{\varphi }}_b^{\left(K\right)}-{\mathcal{y}}^{\left(K\right)}‖}_2-{‖{\mathcal{A}}_t^{\left(K\right)}{\mathit{\varphi }}_t^{\left(K\right)}-{\mathcal{y}}^{\left(K\right)}‖}_2$$

(24)

If the detection value is bigger than a predefined threshold, the original test pixel $\mathbf{y}$ will be labeled as a target pixel; otherwise, the test pixel is a background pixel. The details for hyperspectral target detection using LBHRF are summarized in Algorithm 1.

Algorithm 1: The proposed LBHRF learning method for hyperspectral target detection.

Input: Test pixel $\mathbf{y}$. Target prior spectra and background dictionaries $\mathbf{A}$. The band-partition level L. The layer K. 1: Learn the L levels augmented representation residual feature for all the target and background pixels $\mathbb{A}$ and test pixel $\mathbb{y}$ via ARRFLB presented in Equations (8)–(14); 2: Initialize k = 1; 3: Repeat; 4: Learn the k-th layer re-augmented representation residual features ${\mathcal{A}}^{\left(k\right)}$ and ${\mathcal{y}}^{\left(k\right)}$ for the target and background dictionary pixels and the test pixel based on the augmented representation residual feature $\mathbb{A}$ and $\mathbb{y}$ by ARRFRR presented in Equations (17)–(22); 5: k = k + 1; 6: Until k > K; 7: Calculate the hierarchical detection value by RTD as presented in Equations (23) and (24). Output: The hierarchical detection value ${\mathrm{H}\mathrm{D}\mathrm{V}}^{\left(K\right)}$ for target detection.

4. Experimental Results and Analysis

4.1. Hyperspectral Data Set

The first data set was collected by the HYDICE sensor [21] with a spatial resolution of 2 m and 210 spectral bands. After removing the low SNR, water absorption and bad bands (1–4, 76, 87, 101–111, 136–153, and 198–210), 162 bands remained. The HYDICE data set and its ground-truth information are shown in Figure 5, which has 150 × 150 pixels, and the vehicles were selected as the targets to detect, which have 21 pixels.

The second and third data sets are collected by the AVIRIS from San Diego with a spatial resolution of 3.5 m. After removing the low SNR, water absorption, and bad bands (1–6, 33–35, 97, 107–113, 153–166, and 221–224), 189 bands were remained. The AVIRIS I data set and AVIRIS II data set and their ground-truth information are shown in Figure 6 and Figure 7, which are 60 × 60 pixels and 100 × 100 pixels in size, respectively.

The proposed LBHRF HTD method is evaluated in comparison to the several advanced HSI target detectors, including constrained energy minimization (CEM) [15], sparse representation-based target detector (SRD) [20], sparse representation-with binary hypothesis detector (SRBBH) [23], binary-class collaborative representation-based detector (BCRD) [21], sparse and dense hybrid representation-based detector (SDRD) [22], and single-spectrum-driven binary-class sparse representation target detector (SSBRTD) [29]. In summary, the comparing methods include classic target methods, i.e., CEM and the state-of-the-art representation learning based methods, i.e., SRD, SRBBH, BCRD, SDRD, and SSBRTD. For a fair comparison between different detectors, the widely used dual concentric window strategy is adopted to estimate the background characteristic around each test pixel. For our LBHRF HTD method, the l₂-norm is used to regularize the representation coefficient in the two basic residual feature learning modules, i.e., p = 2 in Equations (9) and (10).

To evaluate the performance of different detection methods, the probability of false alarm (P_F), and probability of detection (P_D) under different threshold variables τ are calculated [30]. In addition, the ROC curves w.r.t. (P_F, P_D) and (τ, P_F) are used to reveal the target detectability and background suppression of each detection method. The area under curve (AUC) corresponding to the two ROC curves are used to quantitatively evaluate the performance of different detection methods. A detection method with a higher ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\mathrm{P}\mathrm{F},\mathrm{P}\mathrm{D})}$ approaching 1 (→1) and a lower ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\tau ,{\mathrm{P}}_{\mathrm{F}})}$ close to 0 (→0) is judged to have better performance for target detection as well as background suppression [30]. Without good background suppression ability, a HSI detector cannot guarantee a sufficiently low false alarm rate. Additionally, the ratio between the two AUC values, i.e., ${\mathrm{A}\mathrm{U}\mathrm{C}\left(\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\right)=\mathrm{A}\mathrm{U}\mathrm{C}}_{({\mathrm{P}}_{\mathrm{F}},{\mathrm{P}}_{\mathrm{D}})}/{\mathrm{A}\mathrm{U}\mathrm{C}}_{(\tau ,{\mathrm{P}}_{\mathrm{F}})}$, can be calculated to comprehensively consider the balance ability of a detector in both target detectability and background suppression [30].

4.2. Experimental Results

First, the qualitative detection results of different detection methods are demonstrated. The 2D visualization detection results of different HSI target detectors on the three data sets are shown in

Table 1. Visualization of detection results of different HTD methods on the three datasets.

Data Sets	The Comparing HSI Detectors
	CEM [15]	SRD [20]	SRBBH [23]	BCRD [21]	SDRD [22]	SSBRTD [29]	LBHRF	Ground-Truth
HYDICE
AVIRIS I
AVIRIS II

. From the qualitative visualization results, one can observe that LBHRF method can generate a more conspicuous target distribution when comparing with the ground-truth and detection results of the comparing counterparts. In detail, the targets detected by the proposed LBHRF method have a clearer shape with a purer background, which means a better ability for both target detection and non-target background suppression. By contrast, the detection results of the comparing methods cannot clearly distinguish the targets, and the background component has strong interference in the detection results.

It is generally known that for a HTD task, a high-performance detector should have significantly strong responses for the target pixels and simultaneously possesses strong inhibition ability for non-target background pixels, which can lead to a higher probability of detection rate by preserving lower false alarm rate. For a quantitative evaluation, the ROC curves w.r.t. (P_F, P_D) and (τ, P_F) are drawn and shown in Figure 8 and Figure 9 to further demonstrate the target detectability and background inhibition characteristics of different detectors. In addition, the AUC values, i.e., AUC(P_F, P_D) and AUC(τ, P_F) that correspond to the two kinds of ROC curves are reported in

Table 2. AUC(P_F, P_D)value comparison of different HTD methods on the three HSI datasets. The best result is in bold with the second-best result underlined.

Methods	Data Sets
	HYDICE	AVIRIS I	AVIRIS II
CEM [15]	0.9139	0.6549	0.6983
SRD [20]	0.9840	0.9561	0.9331
SRBBH [23]	0.8909	0.8913	0.7224
BCRD [21]	0.9175	0.9805	0.9899
SDRD [22]	0.9973	0.9583	0.9921
SSBRTD [29]	0.9597	0.9051	0.9689
LBHRF	0.9978	0.9844	0.9987

and

Table 3. AUC(τ, P_F) comparison of different HTD methods on the three HSI datasets. The best result is in bold with the second-best result underlined.

Methods	Data Sets
	HYDICE	AVIRIS I	AVIRIS II
CEM [15]	0.3218	0.3236	0.4573
SRD [20]	0.4459	0.1876	0.3421
SRBBH [23]	0.1234	0.0936	0.2558
BCRD [21]	0.7926	0.8888	0.8635
SDRD [22]	0.4488	0.3812	0.3028
SSBRTD [29]	0.5106	0.4287	0.7326
LBHRF	0.0310	0.0388	0.0037

, where the best results are indicated in bold and the second-best results are underlined.

As Figure 8 shows, the ROC curve of the proposed LBHRF method w.r.t. (P_F, P_D) is closer to the upper left part of each figure when comparing with the other methods. The corresponding quantitative AUC(P_F, P_D) value given in Table 2 also validates this point, and the proposed method can generally achieve a higher AUC(P_F, P_D) value, which means a better overall detection performance. Moreover, the ROC curves of the proposed LBHRF method w.r.t. (τ, P_F) shown in Figure 9 approach nearer to the lower left part of each figure than that of the other methods. Similarly, the AUC(τ, P_F) values listed in Figure 9 correspond to Figure 9 also indicate that the proposed LBHRF can yield lower AUC(τ, P_F) values. When comprehensively considering the balance between AUC(P_F, P_D) and AUC(τ, P_F), i.e., the AUC(ratio) reported in

Table 4. AUC (ratio) comparison of different HTD methods on the three HSI datasets. The best result is in bold and the second-best result is in underline.

Methods	Data Sets
	HYDICE	AVIRIS I	AVIRIS II
CEM [15]	2.8400	2.0238	1.5270
SRD [20]	2.2068	5.0965	2.7276
SRBBH [23]	7.2196	9.5224	2.8241
BCRD [21]	1.1576	1.1032	1.1464
SDRD [22]	2.2221	2.5139	3.2764
SSBRTD [29]	1.8796	2.1113	1.3225
LBHRF	32.1871	25.3711	269.9189

, our LBHRF method presents an incomparable obvious advantage over the other methods. In general, the benefits of the LBHRF method can be attributed to the multi-level and multi-layer residual feature learning and augmentation, and the discriminate information is accumulated in the obtained augmented representation residual feature for improved overall detection performance.

4.3. Parameters Sensitivity Analysis

In Section 3.2, the augmented representation residual feature learning based on level-wise band-partition is presented, and there is an adjustable trade-off parameter ${\lambda }_1$ in Equation (9). In the experiments, ${\lambda }_1$ is set and remain unchanged for all the different band-partition combinations under different levels. In addition to ${\lambda }_1$, trade-off parameter ${\lambda }_2$ in Equations (17) and (22) also needs to be set for augmented representation residual feature reuse and relearning. For simplicity, ${\lambda }_2$ is also assigned and remains unchanged for different layers of residual feature learning. As a result, this section will study the influences of ${\lambda }_1$ and ${\lambda }_2$ on the final detection performance. Specifically, the AVIRIS I HSI data set is used as an example, and ${\lambda }_1$ and ${\lambda }_2$ are both selected from the parameter candidate set $\left\{{10}^{-5},{10}^{-3},{10}^{-1},1,10,{10}^3,{10}^5\right\}$ when L = 0, 1, 2 with varying K.

In Section 3.3, the final re-augmented representation residual feature for target and background dictionary and test pixel, i.e., the resulting ${\mathcal{A}}^{\left(K\right)}\in {\mathfrak{R}}^{\left(2\left(L+1\right)+2K\right)\times N}$ and ${\mathcal{y}}^{\left(K\right)}\in {\mathfrak{R}}^{\left(2\left(L+1\right)+2K\right)\times N}$ are fed into the RTD for HDV calculation and target detection as in Equations (22) and (23). Therein, L means the levels of band-partition and K indicates the layers for augmented representation residual feature reuse and relearning. The target detectability and background suppressible in terms of the ROC curves w.r.t. (P_F, P_D) and (τ, P_F) of our LBHRF detector are studied under different settings of L and K. To be specific, L = 0, 1, 2 is considered when K varies from 0 to the deepest layer of 30. It is worth noting that L = 0 means that only the original full spectral bands are utilized without band-partition for LBHRF detector construction. Also, K = 0 means that the augmented representation residual feature with $2\left(L+1\right)$ dimensions obtained from the L levels of band-partition is directly fed into the ultimate RTD for category judgement of a test pixel.

Figure 10 shows the AUC(ratio) performance variation of the proposed LBHRF method in terms of varying ${\lambda }_1$ and ${\lambda }_2$ when band-partition level L = 0, 1, 2. For each individual figure, when ${\lambda }_1$ and ${\lambda }_2$ fall in the scopes of $[{10}^{-5},1]$ and $[{10}^{-5},{10}^{-3}]$, respectively, the proposed LBHRF method tends to achieve a better and more stable AUC(ratio) performance. In addition, when L increases from 0 to 2, our LBHRF method can gradually yield better AUC(ratio) performance, which shows the band-partition strategy used for the residual feature learning and augmentation helps to improve the target detection and background suppression performance.

In addition, the changes for the ${\mathrm{A}\mathrm{U}\mathrm{C}}_{({\mathrm{P}}_{\mathrm{F}},{\mathrm{P}}_{\mathrm{D}})}$, ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\tau ,{\mathrm{P}}_{\mathrm{F}})},$ and AUC(ratio) performance of the proposed LBHRF method when L = 0 and K varies from 0 to 30 are displayed in Figure 11. From the figures, it is not difficult to conclude that ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\mathrm{P}\mathrm{F},\mathrm{P}\mathrm{D})}$,${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\tau ,\mathrm{P}\mathrm{F})},$ and AUC(ratio) values of the proposed LBHRF method show a similar change trend with varying band-partition level L. Specifically, the overall performance indicator ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\mathrm{P}\mathrm{F},\mathrm{P}\mathrm{D})}$ is inclined to decrease when K increases from 0 to 30, and at the same time the ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\tau ,\mathrm{P}\mathrm{F})}$ values also tend to decrease. In addition, it is noteworthy that the decreasing ratios of ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\tau ,{\mathrm{P}}_{\mathrm{F}})}$ values are larger than that of ${\mathrm{A}\mathrm{U}\mathrm{C}}_{({\mathrm{P}}_{\mathrm{F}},{\mathrm{P}}_{\mathrm{D}})}$ values. For example, when L = 0, the ${\mathrm{A}\mathrm{U}\mathrm{C}}_{({\mathrm{P}}_{\mathrm{F}},{\mathrm{P}}_{\mathrm{D}})}$ value decreases from about 0.975 to around 0.96 with a decreasing ratio of 1.53%. Meanwhile, the ${\mathrm{A}\mathrm{U}\mathrm{C}}_{(\tau ,{\mathrm{P}}_{\mathrm{F}})}$ value decreases from about 0.06 to around 0.03, and the decreasing ratio reaches about 50%. The results demonstrate that when the residual feature learning and augmentation layer goes deeper, the overall target detectability decrease slightly and the background inhibition ability improves significantly, which can lead to a better overall AUC(ratio) performance, as demonstrated in Figure 11. In practice, a smaller layer K can be set when a higher overall target detectability is preferred. When a stronger background suppression ability is desired, a larger layer K will be preferred for parameter setting.

4.4. Discussion

When designing a target detector for HSI, there are two key characteristics that should be generally considered, i.e., the target detection ability as well as the background suppression capacity. A fine target detector should ensure a high true positive rate as well as a strong background suppression ability, which can reduce the false positive rate. The experimental results show that the existing representation learning-based HTD methods with original full spectral bands underperform comparatively on the three HSI target detection tasks. The reason of which is that the one-round residual feature learning strategy cannot make full use of the rich spectral information to detect target from background.

By comparison, the proposed LBHRF method aims to sufficiently utilize the global full spectra as well as the contextual information of adjacent local sub-band combinations. In addition, the SoftMax transformation, pooling operation, and representation residual feature reuse and re-augmentation are equipped in cascade to enhance the nonlinear learning ability of the LBHRF method. With the learning methodology, our LBHRF method can progressively discover and converge the underlying discriminant information in both global full bands and the local sub-bands combinations, and the finally obtained augmented representation residual feature with RTD can well distinguish the highly-mixed target and background pixels by highlighting target and inhibiting background, which are verified by the above qualitative and quantitative analysis.

5. Conclusions

This paper revisits the prestigious representation learning-based HTD methods, and finds that the detection performance of which will be restricted by the one-round shallow residual feature learning on the original full spectral bands. To alleviate the problem, this paper proposes to divide and congregate different levels of sub-band spectral bands combinations for multi-level and multi-layer residual feature learning and augmentation, and thus the discriminate information that are beneficial for distinguishing targets from background can be accumulated in the obtained augmented residual feature. Experimental results on different HSI target detection tasks show that the proposed LBHRF method can not only achieve an leading overall target detection performance, i.e., a higher AUC(P_F, P_D) value, but also obtain a significant improvement in background suppression ability, i.e., a lower AUC(τ, P_F) value, in comparison to some representative state-of-the-art representation learning-based HTD methods. Future work will consider introducing spatial information into our method for further improvement. In addition, the construction of a more efficient universal background dictionary will be studied in future.