TRP-Oriented Hyperspectral Remote Sensing Image Classification Using Entropy-Weighted Ensemble Algorithm

Abstract

The problem that the randomly generated random projection matrix will lead to unstable classification results is addressed in this paper. To this end, a Tighter Random Projection-oriented entropy-weighted ensemble algorithm is proposed for classifying hyperspectral remote sensing images. In particular, this paper presents a random projection matrix selection strategy based on the separable information of a single class able to project the features of a certain class of objects. The projection result is measured by the degree of separability, thereby obtaining the low-dimensional image with optimal separability of the class. After projecting samples with the same random projection matrix, to calculate the distance matrix, the Minimum Distance classifier is devised, repeating for all classes. Finally, the weight of the distance matrix is considered in ensemble classification by using the information entropy. The proposed algorithm is tested on real hyperspectral remote sensing images. The experiments show an increase in both stability and performance.

1. Introduction

With the continuous improvement of sensor technology, the spectral resolution of remote sensing images is getting higher and higher, usually including dozens to hundreds of bands [1,2]. Hyperspectral remote sensing image classification is essentially the process of dividing the image domains into non-overlapping sub-regions according to the feature information of the images and assigning a specific class to each sub-region [3,4]. While providing rich spectral information, it also greatly increases the computational cost required for hyperspectral remote sensing image classification. Therefore, dimensionality reduction is usually required before classification [5,6,7].

Traditional dimensionality reduction methods can be roughly divided into two types: band selection [8,9,10] and feature extraction based on data transformation [11,12,13]. The first type of method is generally based on a certain evaluation criterion function to perform a band combination search to achieve the purpose of dimensionality reduction. The second type of method is to map hyperspectral remote sensing images to a low-dimensional space through linear or nonlinear transformation, thereby obtaining a low-dimensional representation of the original data set. For low-configuration hardware, these methods cannot reduce the dimensionality in an acceptable timeframe, which greatly limits the application of hyperspectral remote sensing images [14,15]. Random Projection (RP) has the characteristics of being independent of high-dimensional data and simple to calculate, which is a dimensionality reduction algorithm with little information loss [16,17]. This algorithm provides a feasible mapping way for the Johnson-Lindenstrauss lemma [18,19], which has been widely used in biology, environmental monitoring, and disaster monitoring fields [20,21,22].

The classification task has attracted many researchers to conduct research in this field, and many algorithms of dimensionality reduction and classification have been proposed [23,24,25]. According to the conditions of prior information, it can be divided into supervised classification algorithms and unsupervised classification algorithms. Representative supervised classification algorithms include Minimum Distance (MD), Support Vector Machines (SVM), and Convolutional Neural Networks [26,27,28]. Zhou et al. [29] proposed a radical algorithm through a self-organizing pixel entanglement neural network. This network used pixel entanglement coefficient to mine the quantum entanglement relationship on the array space. Zheng et al. [30] proposed a spectrum interference-based two-level data augmentation method in deep learning for automatic modulation classification. This algorithm is the first time radio signals were used to help modulation classification by considering the frequency domain information. Zhao et al. [31] proposed a Tighter RP based on Minimum Intra-class Variance (TRP-MIV) algorithm for hyperspectral remote sensing image classification. This algorithm selects the random projection matrix to generate low-dimensional images and uses the MD classifier [32] for subsequent classification. Zhao and Mao [33] proposed a semi-random projection method, which uses Linear Discriminant Analysis (LDA) to calculate each column vector in the projection matrix and calculates the projection matrix by repeating it multiple times. Fuzzy C-Means (FCM) clustering algorithm is one of the representative unsupervised classification algorithms [34]. Fowler et al. [35] proposed a Compressive-Projection Principal Component Analysis (CPPCA) algorithm for hyperspectral images. The CPPCA algorithm first uses the RP algorithm to project the hyperspectral images into the low-dimensional space to reduce the computational complexity of image processing. Next, Principal Component Analysis (PCA) is used to further process the dimensionality reduction results of the RP algorithm in the projection direction with the smallest mean square error. Finally, the final low-dimensional image is obtained. Pasunuri et al. [36] combined the PCA, RP, and K-means algorithms to classify the high-dimensional data. Alshamiri et al. [37] proposed a classification algorithm combining Extreme Learning Machine (ELM) and RP, which uses ELM to transform high-dimensional data and keeps the linear class separability of high-dimensional data in the ELM feature space. The RP is used to project the transformation result of ELM into a low-dimensional space. Rathore et al. [38] proposed a new Cumulative Agreement Fuzzy C-Means (CAFCM) algorithm, which uses the clustering effectiveness index to sort all the membership matrices and accumulates all the membership matrices to get the final similarity measure matrix. Anderlucci et al. [39] proposed a model-based clustering algorithm for high-dimensional data, which obtains the final segmentation result through consensus aggregation. Although the RP algorithm can achieve simple and fast dimensionality reduction results, since the random projection matrix is randomly generated, it may generate low-dimensional images that are not conducive to subsequent classification tasks and have the disadvantage of high randomness.

To solve the above problems, this paper exploits TRP-oriented hyperspectral remote sensing image classification using an entropy-weighted ensemble algorithm. The proposed algorithm can effectively improve the classification accuracy of hyperspectral remote sensing images. First, based on the TRP algorithm, a distance matrix suitable for a certain class is generated by combining the random projection matrix selection strategy based on the separable information of a single class and the MD classifier. The above steps are repeated for all classes and the information entropy of the distance matrix of all classes is calculated as weights to generate the final similarity measure matrix, thereby realizing the classification of hyperspectral remote sensing images. The structure of this paper is organized as below. Section 2 and Section 3 give the materials and the proposed algorithm, respectively. The results and discussion are provided in Section 4. Finally, this paper is concluded in Section 5.

2. Materials

This paper uses four publicly available datasets with validation data, namely, the real LongKou [40,41], Salinas, Pavia University, and Pavia Centre images. Among them, the LongKou images are obtained by the UAV-borne hyperspectral system, and the images of Salinas, Pavia University, and Pavia Centre are obtained by the airborne hyperspectral platform. Compared with the airborne hyperspectral platform, the hyperspectral images obtained by the UAV-borne hyperspectral system have higher spatial resolutions. In addition, the standard classification data are included in the experimental data, which can effectively measure the effectiveness of the proposed algorithm. Figure 1 is an experimental image, where Figure 1(a1–d1) are false-color images, Figure 1(a2–d2) are standard classified images, Figure 1(a3–d3) are mean spectral curves for each class, and Figure 1(a4–d4) are legends for all classes of four real images. The experimental image parameters are shown in

Table 1. Parameters of experimental images.

	LongKou	Salinas	Pavia University	Pavia Centre
Size	250 × 400	265 × 107	260 × 340	1096 × 515
Class	6	7	6	9
False-color bands	130, 65, 18	34, 18, 11	68, 21, 2	68, 21, 2
Number of bands	270	204	103	102
Spatial resolution	0.463 m	3.7 m	1.3 m	1.3 m
Sensor	Nano-Hyperspec	AVIRIS	ROSIS	ROSIS

.

As some data points in the four experimental images do not contain any information, these data points are regarded as background and discarded before application. The numbers of spectral vectors of these hyperspectral remote sensing images are 93083, 14879, 11915, and 107352, respectively.

3. The Proposed Algorithm

First, the TRP algorithm is used to reduce the dimensions of hyperspectral remote sensing images containing all bands. Then, the MD classifier is used to classify the low-dimensional images to obtain the distance matrix. Finally, they are included in the ensemble classification framework to obtain the final classification results.

3.1. TRP Algorithm

Given a hyperspectral remote sensing image A = {a_j, j = 1, …, J}, where j is the pixel index, J is the number of pixels, and a_j = (a_jd, d = 1, …, D) is the spectral measure vector of pixel j, d is the band index, D is the number of bands, and a_jd is the spectral measure of the band d of pixel j. Taking the spectral vector a_j (j = 1, …, J) as the row vector, a hyperspectral remote sensing image can be expressed as a J × D matrix. For the convenience of description, A is still used to refer to a hyperspectral remote sensing image matrix without confusion, that is, A = [a₁, … a_j, …, a_J]^T, where T is the transpose operation.

The TRP algorithm can project hyperspectral remote sensing images into a low-dimensional subspace and make any vector pair in the low-dimensional subspace satisfy the distance relative invariance with a high probability. It is possible for the TRP algorithm to reduce the dimensionality of hyperspectral remote sensing images. The TRP algorithm is as follows [31].

Theorem 1.

The D dimensional feature space can be randomly projected into the K_TRP dimensional space, where K_TRP is a positive integer and satisfies,

$$K_{\mathrm{TRP}}\ge K_{\mathrm{TRP}}^0=\lceil \frac{320+160\beta }{\epsilon +20{\epsilon }^2}\ln J\rceil$$

(1)

where $K_{TRP}^0$ is the intrinsic dimensionality, ⌈ ⌉ is the round up character, ε ∈ [0.7, 1.5], and β > 0 are the projection parameters that control the range of distance preservation and the success rate of projection, respectively. Let R_TRP = [r_dk]_{D × KTRP} be a tighter random projection matrix, and r_dk are independent random variables subject to standard normal distribution, that is, r_dk ~ N(0, 1). For a given hyperspectral remote sensing image A, the low-dimensional image B projected to K_TRP dimensionality by R_TRP is,

$$B=\frac{1}{\sqrt{K_{\mathrm{TRP}}}}A{R}_{\mathrm{TRP}}$$

(2)

where B = [b₁, …, b_j, …, b_J]^T = [b_jk]_{J × KTRP}, and b_j is the low-dimensional vector of pixel j. For the spectral vectors a_j and a_j′ in the hyperspectral remote sensing image A, let the corresponding two low-dimensional vectors in the low-dimensional image B be b_j and b_j′, respectively. b_j and b_j′ satisfy the distance relative invariance, if

$$(1-\epsilon ){\Vert {\mathbf{\alpha }}_j-{\mathbf{\alpha }}_{j^{\prime }}\Vert }_2^2\le {\Vert {\mathit{b}}_j-{\mathit{b}}_{j^{\prime }}\Vert }_2^2\le (1+\epsilon ){\Vert {\mathbf{\alpha }}_j-{\mathbf{\alpha }}_{j^{\prime }}\Vert }_2^2$$

(3)

where ${\Vert \cdot \Vert }_2$ represents 2-norm. So, ${\Vert {\mathbf{\alpha }}_j-{\mathbf{\alpha }}_{j^{\prime }}\Vert }_2^2={\displaystyle \sum _{d=1}^D{\left(a_{jd}-a_{j^{\prime }d}\right)}^2}$ Any two low-dimensional vectors in the K_TRP dimensional space obtained by the TRP algorithm satisfy the distance relative invariance at least with the probability P_TRP, where P_TRP = 1 − J^–β.

It is worth noting that the distance relative invariance is not that the square of the distance between the vectors before and after projection is equal, but ${\Vert {\mathit{b}}_j-{\mathit{b}}_{j^{\prime }}\Vert }_2^2$ remains at interval $\left[(1-\epsilon ){\Vert {\mathbf{\alpha }}_j-{\mathbf{\alpha }}_{j^{\prime }}\Vert }_2^2,(1+\epsilon ){\Vert {\mathbf{\alpha }}_j-{\mathbf{\alpha }}_{j^{\prime }}\Vert }_2^2\right]$. The b_j and b_j′ with ${\Vert {\mathit{b}}_j-{\mathit{b}}_{j^{\prime }}\Vert }_2^2$ out of the interval $\left[(1-\epsilon ){\Vert {\mathbf{\alpha }}_j-{\mathbf{\alpha }}_{j^{\prime }}\Vert }_2^2,(1+\epsilon ){\Vert {\mathbf{\alpha }}_j-{\mathbf{\alpha }}_{j^{\prime }}\Vert }_2^2\right]$ are considered to have little similarity of the vector pair before and after projection. Under the constraint of the same probability P_TRP, the intrinsic dimensionality of the TRP algorithm is lower than that of the RP algorithm. Therefore, the TRP algorithm can reduce the number of bands in hyperspectral remote sensing images to a greater extent, while ensuring that the vector pairs satisfy the distance relative invariance with probability P_TRP. As distance reflects the structure of the dataset, the TRP algorithm indicates that low-dimensional images in a low-dimensional space can maintain the structure of hyperspectral remote sensing images with a high probability.

The detailed process of the TRP Algorithm 1 can be summarized as follows.

Algorithm 1. The detailed process of the TRP algorithm.

Input: test hyperspectral remote sensing image A.

Output: low-dimensional image B.

Step 1. Calculate $K_{\mathrm{TRP}}^0$ ← Equation (1), and set the dimensionality KTRP.

Step 2. Generate rdk according to the standard normal distribution, that is rdk ~ N(0, 1).

Step 3. Form R.

Step 4. Calculate the low-dimensional image B ← Equation (2).

3.2. Random Projection Matrix Selection Strategy

The random projection matrix is randomly generated without considering the class information of hyperspectral remote sensing images. Different random projection matrices will produce different low-dimensional images. Therefore, the selection of the random projection matrix directly affects the subsequent classification accuracy of hyperspectral remote sensing images. To make low-dimensional images have stronger class separability and achieve accurate classification of hyperspectral remote sensing images, this subsection uses a random projection matrix selection strategy based on the separable information of a single class to obtain low-dimensional images with the best separability of a single class. It is worth noting that the benefit of subsequent classification refers to greater differences between classes and smaller differences within classes. This means that there may be overtraining effects and generalization may degenerate, but it is more meaningful for classification tasks.

First, the sample matrix of all classes is defined as F = [F¹; …; F^l; …; F^L], where l is the class index, F^l is the sample matrix of the lth class, and L is the number of classes, which is a priori. It can be specifically expressed as

$$F^l=\left[\begin{array}{cccc}{f}_{11}^l& f_{12}^l& \cdots & f_{1D}^l\\ f_{21}^l& f_{22}^l& \cdots & f_{2D}^l\\ ⋮& ⋮& \ddots & ⋮\\ f_{H1}^l& f_{H2}^l& \cdots & f_{HD}^l\end{array}\right]=\left[\begin{array}{cccc}{F}_1^l& F_2^l& \cdots & F_D^l\end{array}\right]$$

(4)

where $F_d^l$ is the sample vector of the lth class and the dth (d = 1, 2, …, D) band, and H is the number of samples of the lth class. It is worth noting that this paper sets the number of samples to be the same for all classes. Then, the TRP algorithm is used to reduce the dimensionality of the samples. Through the random projection matrix R, the sample matrix F of all classes can be projected into the K_TRP dimensional space, thereby obtaining the low-dimensional sample matrix S = [S₁; …; S_l; …; S_L] of all classes. It is calculated as follows,

$$S=\frac{1}{\sqrt{K_{\mathrm{TRP}}}} F{R}_{\mathrm{TRP}}$$

(5)

S_L in the S matrix is the low-dimensional sample matrix of the lth class obtained by using the TRP algorithm for dimensionality reduction. The specific expansion is as follows:

$$S^l=\left[\begin{array}{cccc}{s}_{11}^l& s_{12}^l& \cdots & s_{1K_{\mathrm{TRP}}}^l\\ s_{21}^l& s_{22}^l& \cdots & s_{2K_{\mathrm{TRP}}}^l\\ ⋮& ⋮& \ddots & ⋮\\ s_{H1}^l& s_{H2}^l& \cdots & s_{H{K}_{\mathrm{TRP}}}^l\end{array}\right]=\left[\begin{array}{cccc}{s}_1^l& s_2^l& \cdots & s_{K_{\mathrm{TRP}}}^l\end{array}\right]$$

(6)

where $s_{H1}^l$ is the low-dimensional measure of the first dimensionality of Hth sample of lth class, and $s_k^l$ is the low-dimensional sample vector of the kth dimensionality in the K_TRP dimensional space,

$$\begin{array}{cc}\hfill s_k^l& =\left[\begin{array}{c}{s}_{1k}^l\\ s_{2k}^l\\ ⋮\\ s_{Hk}^l\end{array}\right]=r_{1k}\left[\begin{array}{c}{f}_{11}^l\\ f_{21}^l\\ ⋮\\ f_{H1}^l\end{array}\right]+r_{2k}\left[\begin{array}{c}{f}_{12}^l\\ f_{22}^l\\ ⋮\\ f_{H2}^l\end{array}\right]+\cdots +r_{Dk}\left[\begin{array}{c}{f}_{1D}^l\\ f_{2D}^l\\ ⋮\\ f_{HD}^l\end{array}\right]\hfill \\ & =r_{1k}{F}_1^l+r_{2k}{F}_2^l+\cdots +r_{Dk}{F}_D^l={\displaystyle \sum _{d=1}^D{r}_{dk}{F}_d^l}\hfill \end{array}$$

(7)

According to Equation (7), $s_k^l$ is related to the vector r_k of the kth column in the random projection matrix. Thus, each element r_dk in the kth column of the random projection matrix can be limited by constraining each cumulative sum of $s_k^l$. The random projection matrix with the best class separability of the dimensionality reduction result can be selected by means of multiple sampling.

The measurement of the random projection matrix selection strategy based on the separable information of a single class is the large intra-class variance of a single class and the small distance from other classes. Each dimensionality in the random projection matrix is separately selected, to select the random projection matrix that is extremely conducive to the classification of this class.

As each element r_dk of the random projection matrix R obeys the standard normal distribution, multiple random numbers can be generated according to this distribution as the sampling set of the elements r_dk. It is defined as Q_dk = [$Q_{dk}^1$, …, $Q_{dk}^{\psi }$, …, $Q_{dk}^{\Psi }$]. Each random number $Q_{dk}^{\psi }$ is used to calculate the class separability, and the random number that maximizes the degree of separability of a certain class is selected as the element r_dk of the random projection matrix R.

Specifically, the lth class is taken as an example to introduce the random projection matrix selection strategy based on the separable information of a single class. For the random number $Q_{dk}^{\psi }$, the variance of the lth class samples and the distance from other class samples are calculated, respectively. Then, the minimum distance between this class and other classes is divided by the variance of this class to get the lth class final difference value $W_{dkl}^{\psi }$. It can be calculated by

$$W_{dkl}^{\Psi }=\frac{\underset{l^{\prime }=1,\dots ,L}{min}\left(r_{1k}{\Vert F_1^l-F_1^{l^{\prime }}\Vert }_2+\cdots +r_{d-1k}{\Vert F_{d-1}^l-F_{d-1}^{l^{\prime }}\Vert }_2+Q_{dk}^{\Psi }{\Vert F_d^l-F_d^{l^{\prime }}\Vert }_2\right)}{var(r_{1k}{F}_1^l+\cdots +r_{d-1k}{F}_{d-1}^l+Q_{dk}^{\Psi }{F}_d^l)}$$

(8)

According to Equation (8), the difference matrix of this class can be obtained by using all random numbers and is expressed as $W_{dkl}^{\psi }$ = [$W_{dkl}^1$, …, $W_{dkl}^{\psi }$, …, $W_{dkl}^{\Psi }$]. Then, the ψ^*th sampling is obtained by maximizing the final difference value of the lth class, that is,

$${\psi }^{*}=\arg \underset{\psi =1, \dots , \Psi }{\max }\left\{G_{dk}^{\psi }\right\}$$

(9)

Finally, the element r_dk takes the ψ^*th sampling random number that maximizes the final class difference value, that is,

$$r_{dk}=Q_{dk}^{{\psi }^{*}}$$

(10)

3.3. Entropy-Weighted Ensemble Algorithm

The random projection matrix selection strategy based on the separable information of a single class considers the separability of a single object class but does not measure the class separability of low-dimensional images from a global perspective. To this end, based on the idea of the random projection matrix selection strategy of the separable information of a single class and ensemble classification, the classification results of multiple low-dimensional images are combined to construct a classification model, thereby obtaining more stable and accurate classification results.

The main idea of the entropy-weighted ensemble classification algorithm is to use the random projection matrix selection strategy based on the separable information of a single class to select L random projection matrices suitable for L classes, respectively. In addition, the TRP algorithm is used to reduce the dimensionality of the hyperspectral remote sensing images A based on these projection matrices, thereby obtaining L low-dimensional images. The number of ensembles is set as the number of classes in this paper to ensure that each class is considered. For ease of reference, iter is used to represent the index of ensembles, that is, iter = 1, …, L. The low-dimensional image obtained by the iterth ensemble using the TRP algorithm is defined as B^iter.

The MD classifier is used to classify L low-dimensional images to obtain L distance matrices, respectively. Each distance matrix Z^iter is regarded as a similarity measurement matrix between the low-dimensional spectral vector and the mean vector of each class. Finally, the entropy information is used to weigh each matrix to obtain the final similarity measure matrix C.

With the help of the class information of the samples, the TRP algorithm is used to calculate the lower limit of the projection dimension K_TRP, and the random projection matrix R^iter is obtained by the random projection matrix selection strategy. Then, the sample matrix F of all classes can be projected into the K_TRP dimensional space. The result of the dimensionality reduction of the sample matrix F by the random projection matrix R^iter is S^iter = [S^iter1; …; S^iterl; …; S^iterL], which is calculated as follows

$$S^{iter}=\frac{1}{\sqrt{K_{\mathrm{TRP}}}} F{R}^{iter}$$

(11)

Then the feature mean vector $S_{\mathrm{mean}}^{iterl}$ of the low-dimensional samples of the lth class is calculated to obtain distance matrix, which is expressed as

$$S_{\mathrm{mean}}^{iterl}=\frac{1}{H}{S}^{iterl}=\frac{1}{H}\left[\begin{array}{cccc}{s}_{11}^{iter}+\cdots +s_{H1}^{iter},& s_{12}^{iter}+\cdots +s_{H2}^{iter},& \cdots ,& s_{1K_{TRP}}^{iter}+\cdots +s_{H{K}_{TRP}}^{iter}\end{array}\right]$$

(12)

Then, according to Equation (11), the same random projection matrix R^iter is used to reduce the dimensionality of the hyperspectral remote sensing image A to obtain the low-dimensional image B^iter. So far, the feature mean vector and low-dimensional images of all classes of low-dimensional samples can be obtained. The MD classifier builds a classification model on low-dimensional images, and the similarity of the low-dimensional vector is defined by calculating the distance between each low-dimensional vector and the mean vector of each class sample. The distance matrix is defined as Z^iter = [$z_1^{iter}$, …, $z_j^{iter}$, …, $z_J^{iter}$], where $z_j^{iter}$ is the distance between the jth low-dimensional vector and the mean vector of all class samples of the iterth ensemble. Specifically, $z_j^{iter}$ = [$z_{j1}^{iter}$, …, $z_{jl}^{iter}$, …, $z_{jL}^{iter}$], where $z_{jl}^{iter}$ is between the mean vector $S_{\mathrm{mean}}^{iterl}$ of the lth class low-dimensional samples and the low-dimensional vector $b_J^{iter}$. The distance is calculated as follows

$$z_{jl}^{iter}=\Vert S_{\mathrm{mean}}^{iterl}-b_j^{iter}\Vert$$

(13)

In the process of entropy-weighted ensemble classification, to avoid the problem that the distance value in each distance matrix is too large or too small, it is necessary to normalize each distance matrix Z^iter to obtain the matrix Y^iter before processing all the distance matrices, that is, Y^iter = [$y_1^{iter}$, …, $y_j^{iter}$, …, $y_J^{iter}$]. Specifically, $y_j^{iter}$ = [$y_{j1}^{iter}$, …, $y_{jl}^{iter}$, …, $y_{jL}^{iter}$], where $y_{jl}^{iter}$ is calculated as follows

$$y_{jl}^{iter}=\frac{z_{jl}^{iter}-\underset{l=1, \dots , L}{\min }\left\{\underset{j=1, \dots , J}{\min }\left\{z_{jl}^{iter}\right\}\right\}}{\underset{l=1, \dots , L}{\max }\left\{\underset{j=1, \dots , J}{\max }\left\{z_{jl}^{iter}\right\}\right\}-\underset{l=1, \dots , L}{\min }\left\{\underset{j=1, \dots , J}{\min }\left\{z_{jl}^{iter}\right\}\right\}}$$

(14)

The information entropy is used to perform weighted ensemble processing on multiple distance matrices to generate a similarity measure matrix, and the entropy value of the defined matrix Y^iter is calculated as follows

$$E^{iter}=-{\displaystyle \sum _{g=0}^G{p}_g^{iter}\ln p_g^{iter}}$$

(15)

where G represents the total number of unique distance values, $p_g^{iter}$ represents the probability that the distance value in iterth matrix Y^iter is g, and the frequency is not 0. It can be calculated by

$$p_g^{iter}=\frac{1}{J\times L}\#\left\{(j,l),y_{jl}^{iter}=g\right\}$$

(16)

where # represents the quantity, and the (j, l) is the position where the distance value is g. The final similarity measure matrix C = {c_jl, j = 1, …, J, l = 1, …, L} is calculated as follows

$$C=\frac{1}{L}{\displaystyle \sum _{iter=1}^L{E}^{iter}{Y}^{iter}}$$

(17)

Finally, to model the class, the deterministic classification result o = [o₁, …, o_j, …, o_J] can be obtained in the decision-making process of hyperspectral remote sensing images classification, where o_j ∈ {1, …, L}. Simply, the low-dimensional vector b_j belongs to the lth class with the smallest Euclidean distance,

$$o_j=\arg \underset{l=1, \dots , L}{\min }\left\{c_{jl}\right\}$$

(18)

3.4. The Complexity of the Proposed Algorithm

The space and time complexities of the proposed algorithm are analyzed here. This section studies the complexity in three parts: the optimization strategy of the projection matrix, the MD classifier, and the ensemble algorithm.

The main contribution of the complexity of the optimization strategy of the projection matrix is to calculate the projection matrix. To update the projection matrix, it takes O(HLD) space and O(HLDK_TRP) time for the calculation of the low-dimensional sample matrix. Furthermore, O(HK_TRP) space and O(THL²K_TRP) time are required to calculate the class dissimilarity. The main contribution of the complexity of the classification algorithm is to calculate the distance matrix. To update the distance matrix, it takes O(SD) space and O(SDK_TRP) time for the calculation of the low-dimensional images. Furthermore, O(SK_TRP) space and O(SLK_TRP) time are required to calculate the distance. The main contribution of the complexity of the ensemble algorithm is to repeat the above two steps. To obtain the final distance matrix, the overall space complexity of the proposed algorithm is O(LSD), and the overall time complexity of the proposed algorithm is O(L(HLDK_TRP + THL²K_TRP + SDK_TRP + SLK_TRP)).

Figure 2 presents the flow chart for the proposed algorithm. For easier understanding, the blue arrows represent the dimensionality reduction process of samples, and the red arrows represent the dimensionality reduction process of hyperspectral remote sensing images.

Furthermore, the detailed process of the proposed Algorithm 2 can be summarized as follows.

Algorithm 2. The detailed process of the proposed classification algorithm.

Input: samples F, test hyperspectral remote sensing image A.

Output: the classification results o.

Step 1. Calculate $K_{\mathrm{TRP}}^0$ ← Equation (1), and set the dimensionality KTRP.

For iter = 1: L

Step 2. Randomly generate Ψ random numbers according to the standard normal distribution.

Step 3. Calculate the final difference value $W_{dkl}^{\psi }$ ← Equation (8).

Step 4. Calculate the ψ*th sampling ← Equation (9).

Step 5. Form Riter ← Equation (10).

Step 6. Reduce the dimensionality of the hyperspectral image A and samples F ← Equation (11).

Step 7. Calculate the mean vector of each low-dimensional class sample ← Equation (12).

Step 8. Calculate distance matrix Ziter ← Equation (14).

End

Step 9. Obtain the entropy value Eiter ← Equation (15).

Step 10. Obtain the final similarity measure matrix C ← Equation (17).

Step 11. Make a classification decision o ← Equation (18).

4. Results and Discussion

To verify the superiority of the proposed classification algorithm, MATLAB R2018a software is used to classify real hyperspectral remote sensing images on a computer with Intel (R) Core (TM) i5-4460, 3.20 GHz, 8 GB memory. The results are qualitatively and quantitatively evaluated. In this paper, four sets of real hyperspectral remote sensing images with different spectral and spatial resolutions are shown in Figure 1. In addition, this paper mainly considers the amount of calculation and sets the projection parameters as ε = 1.5 and β = 0.5 for the four experimental images. For the sake of fairness, the projection parameters of all experimental images are set uniformly, and their parameters are shown in

Table 2. Projection parameter settings in the experiment of the proposed algorithm.

	LongKou	Salinas	Pavia University	Pavia Centre
Ψ	10	10	10	10
H	10	10	10	10
KTRP	99	83	81	100
KRP	344	289	282	348

. The projection dimensionalities of the TRP algorithm in Table 2 are set as the intrinsic dimensionality, which is based on Equation (1) in the projection parameters ε = 1.5 and β = 0.5. It is worth noting that the projection dimensionality of the TRP algorithm in Table 2 is at least one-third that of the RP algorithm, greatly reducing the computational complexity.

To validate the effectiveness of the random projection matrix selection strategy, the nonlinear projection method [42] was used to visualize class separability. As this method displays the separability of two classes in a two-dimensional space, the class separability of spectral vectors before and after dimensionality reduction in the Pavia University image was analyzed. The first and second lines in Figure 3 show the visualization results of the first and third ensembles, respectively. Among those, the horizontal and vertical coordinates in the visualization results represent the distance from the spectral vector to the mean vector of the two classes of samples, respectively. The blue and red circles represent the distance distribution of the two classes of spectral vectors, respectively. The black dashed line is the dividing line for measuring the class separability, that is a straight line at a 45-degree angle passing through the origin. The more spectral vectors scattered on both sides, the better the class separability of the dimensionality reduction results. From Figure 3, the spectral vectors are almost completely scattered on both sides of the dividing line, and most of them are close to the coordinate axis. This indicates that there are many correctly classified spectral vectors.

To prove the superiority of the proposed algorithm, TRP-MIV, the SVM classification algorithm based on LDA (hereinafter referred to as LDA-SVM) and the CAFCM classification algorithm are used to classify these hyperspectral remote sensing images. The classification results are qualitatively and quantitatively evaluated. The parameters of comparison algorithms are shown in

Table 3. Parameter settings in the experiment of comparison algorithms.

		LongKou	Salinas	Pavia University	Pavia Centre
TRP-MIV	number of random numbers	10	10	10	10
	number of samples of each class	10	10	10	10
LDA-SVM	number of samples of each class	99	83	81	100
CAFCM	number of ensembles	169	41	148	628

. In addition, all experimental images are tested 100 times to effectively evaluate the classification accuracy.

The experimental results are shown in Figure 4. Figure 4 shows the best classification results in 100 trials of the LongKou, Salinas, Pavia University, and Pavia Centre images, respectively. In addition, to qualitatively evaluate the above classification results, the contour lines between the regions in the classification results are extracted and superimposed on the corresponding false color images, and the superposition results are shown in Figure 5.

Through visual evaluation, it can be concluded that the contours of each region in the contour stacking result of the proposed algorithm can coincide well with the boundaries of each class region in the false color images. For images with large intra-class variances (for example, the Salinas and Pavia University images), the proposed algorithm can still obtain better classification results despite misclassification. Since the proposed algorithm can maintain the separability of each class, the algorithm can perform a fine classification of each class. Since the TRP-MIV algorithm selects the random projection matrix based on the separable information of all classes, the features of some classes are always ignored. There are several speckle noises in the classification results of the LDA-SVM algorithm, especially in the Pavia Centre image where there is still misclassification. It can be seen from Figure 4 and Figure 5 that the missed classification of the CAFCM classification algorithm is profoundly serious, and the spectral vectors in the same class are incorrectly classified into other classes. For complex texture images, the CAFCM algorithm is easily disturbed by noise, so it cannot obtain ideal classification results. All information shows that the proposed algorithm can classify hyperspectral remote sensing images well.

In order to quantitatively evaluate the classification results of the proposed algorithm, the confusion matrix of the classification results is obtained by taking Figure 2(a2–d2) as the standard classification data, and the Overall Accuracy (OA), Average Accuracy (AA), Average Precision Rate (APR), and the Kappa coefficient of the whole classification result are calculated according to the confusion matrix. Meanwhile, the runtime of all images was quantitatively analyzed.

Table 4. Accuracy evaluation of classification results for the LongKou image. (The values in brackets are the variance of 100 trials, and those not in are the mean).

	The Proposed Algorithm	TRP-MIV	LDA-SVM	CAFCM
Kappa coefficient	0.84	0.76	0.71	0.51
	(0.01)	(0.03)	(0.01)	(0.10)
OA/%	87.39	81.50	78.43	62.83
	(0.65)	(2.06)	(0.01)	(8.19)
AA/%	82.38	74.29	59.65	50.79
	(1.30)	(2.95)	(1.71)	(5.31)
APR/%	81.15	74.71	74.60	54.15
	(0.47)	(1.55)	(0.89)	(8.60)
Running time/s	5.26	1.59	8.14	1509.77
	(0.13)	(0.06)	(0.41)	(198.36)

,

Table 5. Accuracy evaluation of classification results for the Salinas image. (The values in brackets are the variance of 100 trials, and those not in are the mean).

	The Proposed Algorithm	TRP-MIV	LDA-SVM	CAFCM
Kappa coefficient	0.96	0.94	0.85	0.71
	(0.00)	(0.01)	(0.06)	(0.09)
OA/%	97.04	95.04	87.86	76.66
	(0.08)	(0.76)	(5.06)	(7.08)
AA/%	96.54	94.19	82.63	70.99
	(0.10)	(1.03)	(5.53)	(7.56)
APR/%	96.29	93.74	/	/
	(0.09)	(0.55)
Running time/s	2.01	1.01	1.43	180.51
	(0.07)	(0.10)	(0.05)	(16.17)

,

Table 6. Accuracy evaluation of classification results for the Pavia University image. (The values in brackets are the variance of 100 trials, and those not in are the mean).

	The Proposed Algorithm	TRP-MIV	LDA-SVM	CAFCM
Kappa coefficient	0.89	0.81	0.66	0.41
	(0.00)	(0.04)	(0.06)	(0.03)
OA/%	92.11	86.11	75.00	51.84
	(0.27)	(2.85)	(3.58)	(2.90)
AA/%	95.01	89.28	75.60	50.02
	(0.34)	(2.28)	(7.28)	(7.40)
APR/%	92.71	88.96	/	44.88
	(0.15)	(1.61)		(3.53)
Running time/s	0.84	1.21	1.29	126.76
	(0.03)	(0.77)	(0.15)	(13.17)

and

Table 7. Accuracy evaluation of classification results for the Pavia Centre image. (The values in brackets are the variance of 100 trials, and those not in are the mean).

	The Proposed Algorithm	TRP-MIV	LDA-SVM	CAFCM
Kappa coefficient	0.84	0.80	0.57	0.36
	(0.01)	(0.02)	(0.10)	(0.09)
OA/%	90.27	87.86	73.29	52.03
	(0.58)	(1.10)	(8.59)	(9.87)
AA/%	80.93	74.06	54.94	38.82
	(2.15)	(3.26)	(5.98)	(4.71)
APR/%	78.24	73.02	60.47	34.45
	(1.88)	(3.09)	(7.88)	(5.98)
Running time/s	6.55	1.72	17.04	2291.91
	(0.25)	(0.11)	(0.50)	(409.44)

are the accuracy evaluations of the classification results of the four images, respectively. Additionally, the values in Table 4, Table 5, Table 6 and Table 7 are the mean values of 100 trials, where the unit of running time is seconds.

The mean values of OA, AA, APR, and the Kappa coefficient of the proposed algorithm for the four experimental images are greater than 87%, 80%, 78%, and 0.84, respectively. The OA, AA, APR, and the Kappa values of the comparison algorithms are all lower than those of the proposed algorithm, which means that the proposed algorithm can obtain higher accuracy classification results than the comparison algorithms. Since the TRP-MIV algorithm does not need to perform multiple projections, the running time of the TRP-MIV algorithm is shorter than that of the proposed algorithm. Nonetheless, as can be seen from the running time in Table 4, Table 5, Table 6 and Table 7, the proposed algorithm can obtain high classification accuracy in an acceptable time. In addition, the classification accuracy variance value of the proposed algorithm for 100 trials, which is the value in brackets, is smaller than those of the comparison algorithms, which fully shows that the proposed algorithm is robust. All the classification accuracies show that the proposed algorithm can obtain very good classification results.

Based on the above discussion, by comparing the visual and quantitative evaluations of the proposed classification algorithm and the comparison algorithms, the proposed algorithm outperforms the comparison algorithms in classification performance and stability but is slightly inferior to the comparison algorithms in algorithm running time.

5. Conclusions

Aiming to resolve the problem that the randomness of the RP algorithm may lead to unstable classification results, TRP-oriented hyperspectral remote sensing image classification using an entropy-weighted ensemble algorithm is proposed. Additionally, the classification experiments are conducted on real hyperspectral remote sensing images. The proposed algorithm can effectively improve the classification accuracy and robustness of hyperspectral remote sensing images. The classes with poor separability can also be better distinguished, which basically meets the task of classifying hyperspectral remote sensing images completely and finely. The following points should be completed in the following work. The choice of projection dimensionality in this paper is based on the principle of computational cost. The influence of projection dimensionality on classification accuracy is the research direction.