Full-Process Adaptive Encoding and Decoding Framework for Remote Sensing Images Based on Compression Sensing

Abstract

Faced with the problem of incompatibility between traditional information acquisition mode and spaceborne earth observation tasks, starting from the general mathematical model of compressed sensing, a theoretical model of block compressed sensing was established, and a full-process adaptive coding and decoding compressed sensing framework for remote sensing images was proposed, which includes five parts: mode selection, feature factor extraction, adaptive shape segmentation, adaptive sampling rate allocation and image reconstruction. Unlike previous semi-adaptive or local adaptive methods, the advantages of the adaptive encoding and decoding method proposed in this paper are mainly reflected in four aspects: (1) Ability to select encoding modes based on image content, and maximizing the use of the richness of the image to select appropriate sampling methods; (2) Capable of utilizing image texture details for adaptive segmentation, effectively separating complex and smooth regions; (3) Being able to detect the sparsity of encoding blocks and adaptively allocate sampling rates to fully explore the compressibility of images; (4) The reconstruction matrix can be adaptively selected based on the size of the encoding block to alleviate block artifacts caused by non-stationary characteristics of the image. Experimental results show that the method proposed in this article has good stability for remote sensing images with complex edge textures, with the peak signal-to-noise ratio and structural similarity remaining above 35 dB and 0.8. Moreover, especially for ocean images with relatively simple image content, when the sampling rate is 0.26, the peak signal-to-noise ratio reaches 50.8 dB, and the structural similarity is 0.99. In addition, the recovered images have the smallest BRISQUE value, with better clarity and less distortion. In the subjective aspect, the reconstructed image has clear edge details and good reconstruction effect, while the block effect is effectively suppressed. The framework designed in this paper is superior to similar algorithms in both subjective visual and objective evaluation indexes, which is of great significance for alleviating the incompatibility between traditional information acquisition methods and satellite-borne earth observation missions.

1. Introduction

Optical remote sensing imaging technology is an important branch of remote sensing technology, serving as the “eyes” for human exploration of the universe and the unknown world. Satellite remote sensing image data with “four highs” characteristics of high space, high time, high spectrum and high radiation resolution provides important information and services for human monitoring and research on the earth’s environment and resources [1,2,3,4,5]. The traditional remote sensing satellite image signal acquisition process uses an optical remote sensing camera to perform Nyquist full sampling imaging on the target, then converts the collected analog signal to digital, and finally compresses and encodes the image information through digital processing [6,7]. This method of obtaining image information is similar to the traditional ground camera mode, but contrast to ground working cameras, satellite resources are often tight, while ground receiving end resources are relatively sufficient. Therefore, the working mode with simple encoding and complex decoding is more suitable for spaceborne earth observation missions. The emergence of compressive sensing (CS) theory provides a good solution for this pattern, which directly collects compressed data and senses compressed information, avoiding the redundancy of compressing the data after collecting it, and compression is easier to process than decompression [8,9,10,11].

Traditional compressive sensing observes and reconstructs remote sensing images as a whole, which requires a large amount of space to store the measurement matrices and also increases the difficulty of reconstruction. The hybrid coding framework of block compressed sensing (BCS) divides images into blocks and performs downsampling to alleviate high storage and computational complexity issues through independent measurement and non-overlapping block recovery. To some extent, matrix storage requirements and algorithm computational complexity will be reduced, and the robustness of the transmission process will increase. When using BCS to process images, the images are mostly evenly segmented, and a fixed sampling rate is used for random sampling. However, the texture distribution of each spatial position in each frame of the image is different. When the same observation matrix is used for sampling measurement, the non-stationary characteristics of the image may cause block artifacts, especially at lower sampling rates, which will lead to too many sampling points in smooth areas with fewer detailed textures and too few sampling points in areas with rich texture details, which will affect the reconstruction quality of later image [12,13,14].

To solve these problems, many studies have focused on optimizing the sampling measurement process, by dynamically setting the sampling rates of different image blocks during the sampling process, or classifying image blocks according to different parameters, and then assigning different sample rates to different categories. These methods adaptively select the number of samples from all blocks. The main difference lies in the way of selecting samples or the adaptive measurement extraction process. Common adaptive block compressive sensing (ABCS) methods include the following categories [15]. Adaptive reweighted compressive sensing assigns measurements to each block based on statistical information such as variance, entropy, and number of significant coefficients, in order to effectively recover the image [16]. In reference [17] the standard deviation is used to allocate adaptive sampling rates to each block based on its own data structure, in addition to a fixed sampling rate, to achieve real sampling rate allocation. In this ABCS coding system investigated by Li et al. [18], the structural complexity of blocks can be determined by the error between blocks and sampling rates are allotted according to the error values obtained. The ABCS method based on spatial entropy proposed in reference [19] has similar complexity and reconstruction quality like ABCS method based on error between blocks but differs only in the calculation process. Resources are allocated based on the amount of information, where regions with rich information represent more edges and textures, and higher sampling rates are allocated to blocks with more information and vice versa. All the above methods are subclasses of ABCS and all use some adaptive measurement extraction processes. However, all of them are based on uniform image segmentation without considering the texture feature information of the image. Adaptive sampling is limited by the size of the image block and cannot effectively separate complex and smooth regions. When the sampling rates between adjacent blocks differ greatly, block level artifacts are usually observed, making the reconstruction look highly pixelated.

Based on the above analysis, starting from the general mathematical model of compressed sensing, a theoretical model of segmented compressed sensing is established. Faced with the problem of incompatibility between traditional information acquisition modes and spaceborne ground observation tasks, a fully adaptive encoding and decoding compressed sensing framework for remote sensing images is proposed. Unlike previous semi adaptive or local adaptive methods, the advantages of the adaptive encoding and decoding framework proposed in this paper are mainly reflected in four aspects: (1) the ability to select encoding modes according to the image content; (2) capable of utilizing image texture details for adaptive segmentation; (3) able to detect the sparsity of encoding blocks and adaptively allocating sampling rates; (4) the reconstruction matrix can be adaptively selected based on the size of the encoding block.

2. BCS Mathematical Model

When compressive sensing theory is directly applied to large-scale remote sensing image reconstruction, large-scale imaging signals will lead to long encoding observation time, huge computational complexity in image restoration algorithms, and large storage space required for observation matrices [20,21]. Constructing a mathematical model for block compression perception, which uniformly blocks the image and uses the same observation matrix to sample and measure each sub block separately, can effectively reduce the size of the measurement matrix and the complexity of the image reconstruction algorithm, improve image sampling efficiency and reconstruction performance. The mathematical model of block compression perception observation process is as follows:

Assume that the size of the target image $x$ is $H\times W$, decompose the image into non-overlapping encoding tree blocks (ETB), where $d$ represents the sub-block size, $d=h\times w$, and the entire image is divided into $B=\left(H\times W\right)/\left(h\times w\right)$ blocks, then the original signal $x$ defined on equal block sizes can be expressed as:

$$x={\left[\underset{x[1]}{\underbrace{x_1,\cdots ,x_{d_1}}},\underset{x[2]}{\underbrace{x_{d_1+1},\cdots ,x_{2d_1}}},\cdots ,\underset{x[B]}{\underbrace{x_{(B-1)d_1+1},\cdots ,x_{B{d}_1}}}\right]}^T$$

(1)

For each sub block, the same observation ${\mathrm{\Phi }}_d$ matrix is used for independent sampling, where ${\mathrm{\Phi }}_d$ is a $\widehat{n}\times d$ sized matrix, $\widehat{n}=\left[\frac{nd}{HW}\right]$, $n$ is the number of linear observation values, and the entire image measurement matrix is composed of:

$$\mathrm{\Phi }=\left[\begin{array}{cccc}{\mathrm{\Phi }}_d& 0& \cdots & 0\\ 0& {\mathrm{\Phi }}_d& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& \cdots & 0& {\mathrm{\Phi }}_d\end{array}\right]$$

(2)

The mathematical model of BCS can be expressed as:

$$y=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_B\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{\Phi }}_d& 0& \cdots & 0\\ 0& {\mathrm{\Phi }}_d& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& \cdots & 0& {\mathrm{\Phi }}_d\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_B\end{array}\right]=\mathrm{\Phi }\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_B\end{array}\right]$$

(3)

The set of observation vectors obtained is:

$$y=\left\{y_i\left|y_i={\mathrm{\Phi }}_d{x}_i,i=1,2,\cdots ,B\right.\right\}$$

(4)

Assuming the sparse signal $x=\left\{x_1,x_2,\cdots x_B\right\}$, given the observation matrix ${\mathrm{\Phi }}_d\in R^{\widehat{n}\times d}$, sparse basis matrix $\mathrm{\Psi }\in R^{h\times h}$, the sparse representation coefficient of the block signal after the basis transformation is $\theta$, where $x_i=\mathrm{\Psi }\theta$, then the sensor matrix $\mathrm{\Theta }={\mathrm{\Phi }}_d\mathrm{\Psi }$, compressed observation signal $y_i={\mathrm{\Phi }}_d{x}_i=\mathrm{\Theta }{\theta }_i$. For any block $K$ sparse coefficient ${\theta }_i$, there is a constant ${\delta }_K\left(0<{\delta }_K<1\right)$ that makes the following formula valid.

$$\left(1-{\delta }_K\right){‖{\theta }_i‖}_2^2\le {‖\mathrm{\Theta }{\theta }_i‖}_2^2\le \left(1+{\delta }_K\right){‖{\theta }_i‖}_2^2$$

(5)

Then it is said that matrix $\mathrm{\Theta }$ satisfies the $K$ order Block RIP condition, and the minimum ${\delta }_d$ satisfying the above condition is the block finite isometric constant (Block RIC) of matrix $\mathrm{\Theta }$.

3. Remote Sensing Images Adaptive Encoding and Decoding Framework

The framework of adaptive coding and decoding of remote sensing images is shown in Figure 1, which consists of five parts: mode selection, feature factor extraction, adaptive shape segmentation, adaptive sampling rate allocation, and image reconstruction:

(1)

Mode selection stage: Design the maximum between-class variance (OTSU) method to calculate the optimal threshold between the background and target pixels, and adaptively select image processing modes (complex mode and simple mode) based on the optimal threshold of the image;

(2)

Feature factor extraction stage: Establish a texture roughness saliency model, take edge roughness as the salient feature, use Robert and Prewitt operators to describe the global feature saliency expression, and build the corresponding saliency models for complex and simple patterns in the pattern selection stage;

(3)

Adaptive morphological blocking stage: Introducing the idea of image quadtree partitioning, based on setting the maximum and minimum coding blocks (CB), utilizing feature saliency factors as constraint conditions, and adopting a top-down partitioning strategy to recursively divide the image space into different levels of tree structures;

(4)

Sampling rate adaptive allocation stage: Develop image sparsity judgment criteria, adaptively set sparsity as the information density function of a given coding block based on the different information densities between different coding blocks, and adaptively allocate sampling rates for a given image block;

(5)

Image adaptive reconstruction stage: Propose the adaptive blocked compression- orthogonal matching pursuit algorithm (ABCS-OMP), adaptively select the reconstruction matrix and sparse basis according to the encoding block category, set an iterative threshold based on the known sparsity, and use the OMP algorithm to reconstruct the encoding block.

Firstly, this framework uses the maximum between-class variance method to calculate the image content complexity and adaptively selects the coding mode. By extracting image feature factors, a saliency model of the corresponding mode is established to guide the adaptive morphological segmentation of the coding tree block. Then, by using the information density function to detect the sparsity of the encoding block, the sparsity of the image is explored as much as possible to achieve adaptive sampling rate allocation for the image. Finally, the reconstruction matrix and sparse basis are adaptively selected based on the coding block category, and the iteration threshold is set according to the known sparsity. OMP algorithm is used to reconstruct the coding block.

3.1. Adaptive Mode Selection Stage

The image adaptive mode selection stage proposed in this section mainly includes two parts: first, image content analysis, defining the global image complexity calculation function; second, scene model classification, setting threshold parameters for adaptive mode discrimination.

In view of the image content, this article defines an image complexity calculation method based on the maximum between-class variance. According to the grayscale characteristics of the image, the variance is used to calculate the uniformity of the grayscale distribution, and the image is divided into two parts, the foreground and the background. The larger the inter class variance between foreground and background, the greater the difference between the two parts that make up the image, and the more obvious the contrast between foreground and background. And when part of the background is mistakenly classified into the foreground or part of the foreground is mistakenly classified into the background, the difference between the two parts will be reduced, maximizing the inter-class variance means minimizing the probability of misclassification [22]. Based on the above analysis, by calculating the maximum inter class variance of the encoding tree block, the optimal threshold of the image block is obtained, and the optimal threshold is used for image segmentation. The image complexity $C$ is defined as the number of foreground pixels of the encoding tree block, and the complexity threshold $\alpha$ is set to complete the adaptive mode selection of the encoding tree block.

Based on the maximum between-class variance to evaluate the complexity of image content, the specific mathematical method is as follows: assuming that image pixels can be divided into background and foreground parts according to the threshold, the image with a size of $M\times N$ has a total pixel size of $Sum=M\times N$, $N_1$ and $N_2$ are the number of foreground and background pixels, respectively, then the proportion of background and foreground pixels $w_1$ and $w_2$ can be expressed as:

$$\begin{array}{l}{w}_1=\frac{N_1}{Sum}\\ w_2=\frac{N_2}{Sum}\end{array}$$

(6)

If $P_i$ represents the number of pixels with a grayscale value of $i$ in the background, and ${\mu }_{\mathrm{t}1}$ represents the mathematical expectation of the background pixels relative to the entire image, the average grayscale value of the background pixels can be expressed as:

$${\mu }_1={\displaystyle \sum _{i=0}^{t}i\ast \frac{P_i}{N_1}}={\displaystyle \sum _{i=0}^t\frac{i\ast P_i/Sum}{N_1/Sum}}=\frac{{\mu }_{\mathrm{t}1}}{w_1}$$

(7)

Similarly, ${\mu }_{\mathrm{t}2}$ represents the mathematical expectation of foreground pixels relative to the entire image, and the average grayscale value of foreground pixels can be expressed as:

$${\mu }_2={\displaystyle \sum _{i=t+1}^{M}i\ast \frac{P_i}{N_2}}={\displaystyle \sum _{i=t+1}^M\frac{i\ast P_i/Sum}{N_2/Sum}}=\frac{{\mu }_{\mathrm{t}2}}{w_1}$$

(8)

Then the average gray value of the image within the $0~M$ gray level range is:

$$\mu ={\displaystyle \sum _{i=0}^{t}i\ast \frac{P_i}{N_1}}+{\displaystyle \sum _{i=t+1}^{M}i\ast \frac{P_i}{N_2}}={\mu }_{\mathrm{t}1}+{\mu }_{\mathrm{t}2}=w_1{\mu }_1+w_2{\mu }_2$$

(9)

The between-class variance is:

$$G=w_1{(\mu -{\mu }_1)}^2+w_2{(\mu -{\mu }_2)}^2=w_1\ast w_2\ast {({\mu }_1-{\mu }_2)}^2$$

(10)

Assuming that when the gray level is $i=i_t$, the between-class variance is the largest $G_{\max }$, then the image complexity $C$ can be expressed as:

$$\begin{array}{l}CTB\left[B<i_t\right]=0\\ C=nonzeros(CTB)\end{array}$$

(11)

For the scene model, the main purpose of this section is to adaptively select an appropriate processing method based on the image content and maximize the use of the richness of the image to select an appropriate sampling method. According to the global complexity calculation model, the complexity threshold $\alpha$ is set as the judgment threshold for the image content complexity $C$. The original image is adaptively divided into two categories: simple and complex. The complex type corresponds to scenery such as cities, forests and clouds, simple types correspond to features such as oceans. In addition, considering the great increase in the resolution of remote sensing images, the image content information represented by a single coding tree block is greatly reduced, and there is a significant disparity between the foreground and background of a single coding tree block (for example: the content of the coding tree block is all ocean), resulting in a bimodal or multimodal between-class variance function, and misjudgment of the maximum between-class variance occurs. Based on this, the optimal threshold constraint $i_t$ is added during mode adaptive selection.

The specific steps of coding tree block adaptive mode selection based on the maximum between-class variance are as follows:

Step 1: Divide each frame/scene of the original remote sensing image into $512\times 512$-sized coding tree blocks (CTB).

Step 2: Calculate the histogram of the CTB and normalize it.

Step 3: Set the foreground and background classification thresholds $i,i\in [0,M]$, and iterate from 0.

Step 4: Calculate the between-class variance $G=w_1\ast w_2\ast {({\mu }_1-{\mu }_2)}^2$.

Step 5: $i+1$, return to step 4 until $i>M$.

Step 6: Calculate the maximum between-class variance $G_{\max }$ and the optimal threshold $i_t$.

Step 7: Calculate the image content complexity $C$.

Step 8: Adaptive mode selection, define the adaptive mode selection threshold $\alpha$ and the optimal threshold constraint condition $i_{\alpha }$. When $C<\alpha$, it is in simple mode; When $C>\alpha$ and $i_t<i_{\alpha }$, it is a simple mode; When $C>\alpha$ and $i_t>i_{\alpha }$, it is a complex pattern.

3.2. Feature Factor Extraction Stage

In addition to conventional statistical features such as variance and information entropy, two-dimensional images also have visual saliency features such as texture and edge information. The edges of an image are the most basic features, and feature maps can be extracted based on the mutation of grayscale, color, and texture structure. The saliency model studied in this section is mainly a pure mathematical calculation method. The purpose of establishing the saliency model is to provide a basis for the adaptive segmentation task of images. The edge detection operator is used to extract the edge roughness information of images, and the corresponding saliency models are constructed for the complex and simple modes in the pattern selection stage. By setting the saliency factor threshold of the global feature map, the quadtree is guided to adaptively partition the image in the next stage.

The edges of images have two attributes: direction and amplitude. Edges can usually be detected through first-order derivatives or second-order derivatives, where the first derivative takes the maximum value as the corresponding edge position, and the second derivative takes the zero crossing point as the corresponding edge position. The research object of this article is that remote sensing images contain noise and radiation stripes. The purpose of the research is to alleviate the resource constraints on the satellite and speed up the processing timeliness. Therefore, noise and processing time are the primary considerations when selecting operators. For this purpose, this section mainly discusses the first-order gradient operator, Sobel operator, Roberts operator, Prewitt operator, Kirsch operator and LoG operator [23,24].

Figure 2 shows the edge texture maps of remote sensing images of several typical landmarks such as cities, forests, clouds, and oceans using different operators. It can be observed from the figure that for images with rich content and texture details such as cities, forests, and clouds, the edge details extracted by Sobel operator, Roberts operator, and Prewitt operator are more comprehensive. However, the Sobel operator and Prewitt operator have the phenomenon that the edge is too wide when extracting natural scenes such as forests and clouds, which is not suitable for expressing the saliency of these types of features. For scenes with relatively simple ocean image content, the Sobel operator and Prewitt operator have good performance in extracting sea surface details. Compared to the edge contours of ships, the Prewitt operator is superior. Based on the above analysis, with edge roughness as the salient feature, the Roberts operator and Prewitt operator are used to establish the saliency expressions for complex (such as cities, forests, clouds) and simple (such as oceans) models, respectively.

Based on this, the Roberts operator is used to detect complex CTBs and construct an urban forest pattern saliency model. The correlation between the CTBs and the two Roberts operators can be expressed as:

$$\begin{array}{l}{G}_x=R_x\ast f(x,y)\\ G_y=R_y\ast f(x,y)\\ G=\sqrt{{G_x}^2+{G_y}^2}\approx \left|G_x\right|+\left|G_y\right|\end{array}$$

(12)

Among them, $f(x,y)$ represents the CTB, $R_x$, and $R_y$ are the horizontal and vertical Roberts operators. If the threshold $T_R$ is set, the local feature saliency factor of complex patterns can be expressed as:

$$s=\left\{\begin{array}{c}1, \mathrm{G}>T_R\\ 0, \mathrm{G}<T_R\end{array}\right.$$

(13)

Similarly, use the Prewitt operator to detect simple CTBs and construct a simple pattern saliency model. The correlation $G$ between the CTB and two Prewitt operators can be expressed as:

$$\begin{array}{l}{G}_x=P_x\ast f(x,y)\\ G_y=P_y\ast f(x,y)\\ G=\sqrt{{G_x}^2+{G_y}^2}\approx \left|G_x\right|+\left|G_y\right|\end{array}$$

(14)

$P_x$ and $P_y$ are Prewitt’s two direction operators. If the threshold $T_P$ is set, the local feature salience factor of the ocean model can be expressed as:

$$s=\left\{\begin{array}{c}1, \mathrm{G}>T_P\\ 0, \mathrm{G}<T_P\end{array}\right.$$

(15)

3.3. Adaptive Shape Blocking Stage

The purpose of BCS is to fully utilize image texture features to make block segmentation more accurate, improve the compression sampling effect of each image block, and thus improve the overall image reconstruction effect. In BCS, the image is directly divided into blocks of the same scale. Although the method is simple, it ignores the content and structural texture of the image, inevitably resulting in image reconstruction effects.

In order to improve the reconstruction effect and achieve adaptive image segmentation, this section proposes an image adaptive segmentation strategy based on quadtree, which sets the saliency factor as a constraint condition, fully utilizes the texture structure of the image itself, and guides the quadtree method to adaptively divide the image into encoding blocks of different scales, with large scales corresponding to smooth regions of the image and small scales corresponding to edge regions. The detailed process of dividing $512\times 512$-sized CTBs using the adaptive blocking algorithm is shown in Algorithm 1.

Algorithm 1. Adaptive morphological blocking algorithm based on quadtree images.

$\mathrm{Task}:\mathrm{Adaptive}\mathrm{blocking}\mathrm{of}512\times 512$-sized coding tree blocks

$\mathrm{Input}:512\times 512$ encoding tree block $Y$, blocking threshold $\sigma$$, \mathrm{maximum} \mathrm{encoding} \mathrm{block} l_{\max }$$, \mathrm{minimum} \mathrm{encoding} \mathrm{block} l_{\min }$

$\mathrm{Initialization}:\mathrm{block}\mathrm{level} L=0$$, \mathrm{encoding} \mathrm{block} \mathrm{size} l=0$.

Step:

$\left(1\right) \mathrm{Extract} \mathrm{the} \mathrm{feature} \mathrm{map} \mathrm{of} \mathrm{the} 512\times 512$ CTB and calculate the salient factor $s$ of the CTB feature.

$\left(2\right) \mathrm{Let} L=L+1$, calculate the saliency factors sum of the feature map of the $C$-th coding block (CB):

${\sigma }^{\prime }={\displaystyle \sum _{i=1}^l{\displaystyle \sum _{j=1}^{l}s}}$

$\mathrm{Among} \mathrm{them}, l=\frac{512}{2^L}$.

$\left(3\right) \mathrm{Compare} {\sigma }^{\prime }$ in step (2) with the blocking threshold $\sigma$. If it is less than the blocking threshold $\sigma$$\mathrm{and} l_{\min }\le l\le l_{\max }$$, \mathrm{the} \mathrm{image} \mathrm{division} \mathrm{is} \mathrm{completed}. \mathrm{If} {\sigma }^{\prime }$ is greater than the blocking threshold $\sigma$$\mathrm{and} l_{\min }<l$, return to step (2).

$\mathrm{Output}: \mathrm{Adaptive} \mathrm{Blocked} \mathrm{Image} Y^{\prime }$

3.4. Sampling Rate Adaptive Allocation Stage

After the CB undergoes two-dimensional discrete cosine transform (DCT), most of the energy is concentrated in the low-frequency part in the upper left corner of the coefficient matrix, while the high-frequency coefficients are distributed in the lower right corner. The expression only contains the cosine term, and the CB can be represented by fewer spectral coefficients. Moreover, when the absolute value of the low-frequency coefficients is greater than the absolute value of the high-frequency coefficients, the CB becomes sparser and the greater the proportion of the low frequency part. In this section. $\gamma$ is used as the sparsity judgment threshold of the DCT coefficient $T_{i,j}$. After DCT of the CB, the maximum amplitude coefficient $T_{\max }$ is used to normalize the absolute value $\left|T_{i,j}\right|$ of the coefficient. If the normalized coefficient $\left|T_{i,j}\right|/T_{\max }$ is less than the threshold $\gamma$, the coefficient determined to be smaller is set to 0. Otherwise, the coefficient is regarded as the larger DCT coefficient. $k$ represents the number of larger coefficients among the DCT coefficients of each coding block. In this process, the $\gamma$ threshold is constant across all encoding blocks, but $k$ varies between coding blocks. $k$ is used as a measure of sparsity, defined as the information density function of the coding block.

From the above analysis, it can be seen that the smaller the $k$ is, the sparser the coding block is, and the better image reconstruction can be achieved through fewer sampling points. This section adaptively allocates the sampling rate to the image block by calculating the information density function of the CT. However, in the actual image processing process, an image may contain many $512\times 512$-sized CTBs, and a CTB can be adaptively divided into multiple CBs of different sizes, especially high-resolution and large-width remote sensing images. Assigning a sampling rate to each CB undoubtedly increases the burden of on-board storage and transmission. Based on this, this article adopts clustering method to cluster CBs with the same size in an image into one category, and assign the same sampling rate to the CBs in the category according to their average information density. In fact, for large-scale images, there may be hundreds to thousands of CBs in a class. Considering the compression timeliness issue, if the number of CBs in the category is less than 10, the average information density of all blocks is calculated; while if it is greater than 10, 10 CBs are randomly selected to calculate the average information density.

Furthermore, according to the principle of compressed sensing, image reconstruction relies on both the measurement results and the prior of signal sparsity. Therefore, the number of measurements $m$ depends largely on the sparsity $k$ of the CB, rather than the length $n\times n$ of the CB. For high-quality reconstruction of the CB, the following constraints must be satisfied [25]:

$$ck\log \left(n\times n/k\right)\le m\ll n\times n$$

(16)

where $c$ is a constant. In practical situations, according to the rule of thumb, the number of measured values $m$ must be at least three times greater than the sparsity $k$.

Figure 3 is a flow chart of the sampling rate adaptive allocation method based on information density function and the specific algorithm is shown in Algorithm 2.

Algorithm 2. Sampling rate adaptive allocation algorithm based on information density function.

Task: Adaptive allocation of sample rate to different scale CBs

$\mathrm{Input}: \mathrm{Different} \mathrm{scale} \mathrm{CB} n\ast n,n\in \left[4,8,16,32,64\right]$

Step:

$\left(1\right) \mathrm{CB} \mathrm{clustering}. \mathrm{After} \mathrm{the} \mathrm{adaptive} \mathrm{partitioning}, \mathrm{the} \mathrm{CBs} \mathrm{of} \mathrm{different} \mathrm{sizes} \mathrm{are} \mathrm{grouped} \mathrm{into} Sor{t}_{n\ast n}$ according to the size.

$\left(2\right) \mathrm{The} \mathrm{coding} \mathrm{block} \mathrm{undergoes} \mathrm{discrete} \mathrm{cosine} \mathrm{transform}. \mathrm{Count} \mathrm{the} \mathrm{number} \mathrm{of} \mathrm{CBs} num$$\mathrm{in} \mathrm{each} \mathrm{category}. \mathrm{When} num\le 10$$, \mathrm{calculate} \mathrm{the} \mathrm{DCT} \mathrm{coefficients} coef{s}_i$$\mathrm{of} \mathrm{all} \mathrm{CBs} \mathrm{in} \mathrm{the} \mathrm{category}; \mathrm{when} num>10$, randomly select 10 CBs to calculate the DCT coefficients.

(3) Calculate the CB information density function $k$:

$\begin{array}{l}normalized\_coefs=coefs/\max (coefs)\\ coefs\left[normalized\_coefs<\gamma \right]=0\\ k=nonzeros(coefs)\end{array}$

$\left(4\right) \mathrm{Calculate} \mathrm{various} \mathrm{types} \mathrm{of} \mathrm{adaptive} \mathrm{sampling} \mathrm{rates} Sampl{e}_{n\ast n}$:

$Sampl{e}_{n\ast n}=3\ast {\displaystyle \sum _{i=1}^{num}k}/num$

$\left(5\right) \mathrm{To} \mathrm{prevent} \mathrm{the} \mathrm{difference} \mathrm{between} \mathrm{adaptive} \mathrm{sampling} \mathrm{rates} \mathrm{from} \mathrm{being} \mathrm{too} \mathrm{large}, \mathrm{set} \mathrm{the} \mathrm{upper} \mathrm{bound} \mathrm{of} \mathrm{the} \mathrm{sampling} \mathrm{rate} Sampl{e}_{\max }$$\mathrm{and} \mathrm{the} \mathrm{lower} \mathrm{bound} \mathrm{of} \mathrm{the} \mathrm{sampling} \mathrm{rate} Sampl{e}_{\min }$:

$Sample=\left\{\begin{array}{l}0.2,0<Sample<0.2\\ 0.9,0.9<Sample\end{array}\right.$

$\mathrm{Output}: \mathrm{Adaptive} \mathrm{sampling} \mathrm{rate} Sor{t}_{n\ast n}$$\mathrm{for} \mathrm{various} \mathrm{types} \mathrm{of} Sampl{e}_{n\ast n}$

3.5. Image Adaptive Reconstruction Stage

Most traditional compressed sensing reconstruction algorithms use a fixed observation matrix for the observation samples of all CBs, such as Gaussian random matrix, Bernoulli random matrix, partial Fourier matrix, Toeplitz matrix, Hadamard matrix, etc. This naturally ignores the image content and texture structure, does not fully exploit the sparsity of the image, and the sparsity is different between different CBs. Applying the same sampling rate can easily cause the reconstructed image to be highly pixelated.

Based on the image adaptive coding in Section 3.1, Section 3.2, Section 3.3 and Section 3.4, this section proposes an adaptive block compressed sensing-orthogonal matching pursuit algorithm (ABCS-OMP) in the decoding and reconstruction stage. The reconstruction matrix and sparse basis are adaptively selected for the CB category, the iteration threshold is set according to the known sparsity, and the OMP algorithm is used to reconstruct the image block. Figure 4 is the complete workflow structure diagram of the remote sensing image fully adaptive coding and decoding compressed sensing framework designed in this paper, in which the ABCS-OMP algorithm is shown in Algorithm 3, where the input in Algorithm 3 is the measured value of the image after adaptive sampling and observation, and the output is the adaptive reconstruction algorithm to restore the measured value to the original image.

Algorithm 3. Adaptive decoding-orthogonal matching pursuit algorithm based on CS.

Task: Coding block adaptive decoding reconstruction

Input: observation value $Y$$, \mathrm{measurement} \mathrm{matrix} {\mathrm{\Phi }}_{n\ast n}$$, \mathrm{sparse} \mathrm{basis} {\mathrm{\Psi }}_{n\ast n}$$, \mathrm{coding} \mathrm{block} \mathrm{sparsity} {\overline{k}}_{n\ast n}$

Step:

$\left(1\right) \mathrm{Initialize}. \mathrm{Residual} r_0=y_i$$, \mathrm{number} \mathrm{of} \mathrm{iterations} t=1$$, \mathrm{sparse} \mathrm{representation} \mathrm{coefficient} \theta =0\in R^{n\ast n}$$, \mathrm{encoding} \mathrm{block} \mathrm{index} \mathrm{set} {\mathrm{\Lambda }}_0=\varnothing$.

$\left(2\right) \mathrm{Calculate} \mathrm{the} \mathrm{sensing} \mathrm{matrix}. \mathrm{According} \mathrm{to} \mathrm{the} \mathrm{coding} \mathrm{block} \mathrm{size} n\ast n$$\mathrm{adaptively} \mathrm{selects} \mathrm{the} \mathrm{measurement} \mathrm{matrix} {\mathrm{\Phi }}_{n\ast n}$$\mathrm{and} \mathrm{the} \mathrm{sparse} \mathrm{basis} {\mathrm{\Psi }}_{n\ast n}$$, \mathrm{and} \mathrm{calculates} \mathrm{the} \mathrm{sensing} \mathrm{matrix} T_i$:

$T_i={\mathrm{\Phi }}_{n\ast n}\ast inv({\mathrm{\Psi }}_{n\ast n})$

$\left(3\right) \mathrm{Calculate} \mathrm{the} \mathrm{index} j_t$$\mathrm{with} \mathrm{the} \mathrm{greatest} \mathrm{correlation}, j_t=\arg \underset{j}{\max }{‖T_i\left[j\right]r_{t-1}‖}_2$.

$\left(4\right) \mathrm{Update} \mathrm{index} \mathrm{set} {\mathrm{\Lambda }}_t$$, {\mathrm{\Lambda }}_t={\mathrm{\Lambda }}_{t-1}\cup j_t$.

$(5) \mathrm{Find} \mathrm{the} \mathrm{least} \mathrm{squares} \mathrm{solution} {\widehat{\theta }}_t={\left(T_i{{ }_{{\mathrm{\Lambda }}_t}}^T{T}_i{ }_{{\mathrm{\Lambda }}_t}\right)}^{-1}{T}_i{{ }_{{\mathrm{\Lambda }}_t}}^T{y}_i$.

$(6) \mathrm{When} t\ge {\overline{k}}_{n\ast n}$$, \mathrm{the} \mathrm{algorithm} \mathrm{ends}, \mathrm{return} \mathrm{the} \mathrm{sparse} \mathrm{representation} \mathrm{coefficient} {\widehat{\theta }}_t$$, \mathrm{otherwise} \mathrm{let} t=t+1$ and return to Step 3.

$\left(7\right) \mathrm{Calculate} \mathrm{the} \mathrm{reconstruction} \mathrm{CB} \mathrm{vector} y^{\prime }$:

$y^{\prime }=inv({\mathrm{\Psi }}_{n\ast n})\ast {\widehat{\theta }}_t{ }^{\prime }$

4. Simulation

The operating system used in the test was Windows 10 flagship Edition 64-bit, the processor was 11th Gen Intel(R) Core (TM) i7-1165G7 @ 2.80 GHz, the test tool was MATLAB 2019b, and the timing functions were tic and toc. In order to verify the effectiveness of the full-process adaptive encoding and decoding method for remote sensing images proposed in this article, the aerospace remote sensing data set NWPU VHR-10 [26] and Landsat 7 Cloud Cover Assessment Validation Data [27] were selected as test images. This chapter mainly contains four performance analysis tests: (1) Discuss the adaptive mode selection threshold $\alpha$ in Section 3.1; (2) Verify the establishment of simple and complex modes in Section 3.2 combined with the adaptive morphological blocking in Section 3.3; (3) Discuss the sparsity judgment threshold $\gamma$ of DCT coefficient $T_{i,j}$ in Section 3.4; (4) Performance comparison of different algorithms. In order to verify the performance of the algorithm in this paper, it is compared with various algorithms in terms of subjective reconstruction effects, objective evaluation indicators, and encoding and decoding calculation time.

4.1. Analyze Adaptive Mode Selection Threshold α

In this section, the adaptive pattern selection method based on the maximum between-class variance method proposed in Section 3.1 is simulated and verified, and the judgment criteria of the two constraint conditions of the adaptive pattern selection of image content complexity $C$ and the optimal threshold $i_t$ are discussed.

Firstly, the aerospace remote sensing data set NWPU VHR-10 and Landsat 7 Cloud Cover Assessment Validation Data are divided into blocks to form a set of $512\times 512$-sized CTBs, and the set was selected to construct three complex image content data sets of clouds, cities and forests, as well as a simple ocean image content data set. Each data set contains 30 $512\times 512$-sized CTBs. Each data set contains 30 $512\times 512$-sized coding tree blocks. The image content complexity $C$ and optimal threshold $i_t$ of each coding tree block in the four data sets are calculated according to the adaptive mode selection algorithm of the maximum inter-class difference method.

In this test, the image is quantized by $8bit$, and the maximum image content complexity of a single CTB is 262,144. The larger the complexity $C$, the richer the image content and the more texture details. Figure 5 is a two-dimensional scatter plot of the complexity and optimal threshold of the four datasets. It is observed that the point distribution of the complexity $C$ and the optimal threshold $i_t$ of the cloud, city and forest complex content datasets are relatively stable, and the complexity $C$ almost all are greater than 32,768 (maximum complexity), and the optimal threshold I is also almost all greater than 0.125 (maximum optimal threshold). It was observed that the point distribution of the complexity $C$ and the optimal threshold $i_t$ of the cloud, city and forest complex content datasets is relatively stable. The complexity $C$ is almost all greater than 32,768 (one-eighth of the maximum complexity), and the optimal threshold $i_t$ is also almost entirely greater than 0.125 (one-eighth of the maximum optimal threshold).

Based on the above analysis, $\alpha =32768$ defines the judgment threshold of image content complexity $C$, and $i_{\alpha }=0.125$ is the judgment threshold of the optimal threshold $i_t$. Combining threshold $\alpha$ and $i_{\alpha }$, the maximum between-class variance method is used to adaptively select modes for the four datasets of clouds, cities, forests and oceans. The test results and accuracy are shown in

Table 1. Datasets adaptive mode selection test based on maximum between-class variance method.

Dataset	Oceans	Clouds	Cities	Forests
The number of “1”	3	30	29	29
The number of “0”	27	0	1	1
Accuracy	90%	100%	97%	97%

, where “1” indicates complex modes and “0” indicates simple mode. The test results show that the adaptive mode selection threshold determined in this chapter has a mode selection accuracy of more than 90% for CTBs such as oceans, clouds, cities and forests. The image adaptive mode selection algorithm based on the maximum between-class variance method can be used for remote sensing image adaptive mode selection.

4.2. Image Adaptive Morphological Blocking Test

In order to test the simple and complex saliency models constructed in Section 3.2, as well as the two inn ovations proposed in Section 3.3 using feature saliency factors to guide quadtrees to adaptively divide images into shapes, this section selects cities, airports, forests, and clouds. As well as typical remote sensing image features such as oceans, the effectiveness of the proposed method is tested, and compared with several current typical image adaptive blocking methods, the advantages of the proposed method are analyzed. The test results are shown in Figure 6.

The test results show that compared to the three methods based on difference, mean and variance, and grayscale entropy, the saliency factor method proposed in this paper can effectively perform adaptive image segmentation for complex terrain such as cities, airports, forests, and clouds based on edge textures; secondly, for simple ocean images, sea surface ships and ripples can be detected. The detection of edge details and adaptive morphological division are the advantages of the proposed method. The method based on saliency factors proposed in this article first uses the maximum between-class variance method to construct two adaptive selection modes, simple and complex. Then two feature saliency models are established based on the modes, and finally the feature saliency factor is used to guide the quadtree to carry out adaptive morphological division of images. Whether it is a simple ocean image or a complex city, forest, and cloud image, the image can be partitioned according to the image texture details. The method proposed in this paper has good stationarity.

4.3. Analyze Sparsity Judgment Threshold $\gamma$

In the adaptive allocation method of sampling rate based on the information density function, by setting the sparsity threshold $\gamma$, the sparsity $k$ is adaptively set to the information density function of a given block, and the sampling rate is allocated to the coding block according to the average information density, so as to achieve adaptive downsampling of images. The smaller the judgment threshold $\gamma$, the larger the sparsity $k$, the higher the sampling rate, and the smaller the reconstruction error. However, as the average percentage of coefficients used to reconstruct the CB increases, the compression effect of the algorithm decreases. In order to further balance the relationship between the reconstruction error $e$ and the reconstruction coefficient percentage, the following simulation test is performed:

(1)

Set the sparsity judgment threshold $\gamma$ to 30 points between $\left[{10}^{-4},{10}^{-1}\right]$.

(2)

Calculate the reconstruction error $e$ and the reconstruction coefficient percentage for different $\gamma$ respectively.

(3)

Draw $\gamma$ and reconstruction error curves.

(4)

Draw the reconstruction coefficient percentage and reconstruction error curve.

The test results are shown in Figure 7. As $\gamma$ increases, the reconstruction error increases and the percentage of coefficients used for reconstruction decreases. To further observe the reconstruction effect, the reconstruction effect diagram is displayed at four different positions on the curve. From an empirical perspective, point (B) is the highest error that can be tolerated, and the reconstruction effect is good. To sum up, the sparsity judgment threshold $\gamma =0.02$ is the optimal value (or close enough to the optimal value).

4.4. Performance Comparison of Different Algorithms

In this section, the classic compressed sensing reconstruction algorithm (CS-OMP) [28] based on orthogonal matching pursuit, the widely used uniform block compressed sensing reconstruction algorithm (BCS-OMP) [29] and the BCS-SPL [30] algorithm optimized in two-dimensional compression, block compression and mathematical model are selected for testing and comparison. And analyzed from the aspects of sampling rate, encoding time, decoding time, peak signal-to-noise ratio, structural similarity and subjective reconstruction effect. The whole-process adaptive encoding and decoding framework for images based on compressed sensing proposed in this article has four functions: (1) It can select the encoding mode according to the image content; (2) It can use the image texture details to adaptively block; (3) It can detect the sparsity of the encoding block and adaptively allocates the sampling rate; (4) The reconstruction matrix can be adaptively selected according to the coding block size. The framework does not need manual intervention to set the compression ratio in advance, which effectively avoids the problem of mismatch between the sampling rate and the redundancy of the image, and can explore the sparsity of the image to the maximum extent and allocate the sampling rate reasonably.

The sequence of experiments was as follows: Firstly, the ABCS-OMP algorithm was used for adaptive sampling of the input image to obtain the optimal or near-optimal sampling rate of the image; then, the same sampling rate was set for the three methods of CS-OMP, BCS-OMP and BCS-SPL to compare the reconstructed image quality of different algorithms. Under the same compression ratio and consistent test environment, this section selects five typical remote sensing images with different image complexities, such as cities, airports, forests, clouds, and oceans for performance testing, and compares them from both subjective and objective aspects. Among them,

Table 2. Number of various image blocks and sub-sampling rate.

		City	Airport	Forest	Cloud	Ocean
4 × 4	SubR	0.9000	0.5625	0.9000	0.9000	0.8813
	Num	13732	4132	14596	10524	1460
8 × 8	SubR	0.2000	0.2000	0.2000	0.2000	0.2000
	Num	587	1131	343	665	591
16 × 16	SubR	0.2000	0.2000	0.2000	0.2000	0.2000
	Num	19	483	26	200	269
32 × 32	SubR	0	0	0	0	0.2000
	Num	0	0	0	0	89
64 × 64	SubR	0	0	0	0	0.2000
	Num	0	0	0	0	10
Sampling rate		0.7867	0.2914	0.8263	0.6963	0.2624

shows the adaptive sampling results of 5 different types of images by the method proposed in this paper, where ‘SubR’ represents the subsampling rate of each type of image block, and ‘Num’ represents the number of each type of image block.

Table 3. Objective evaluation index test results of different algorithms.

Test Image	Algorithm	Objective Evaluation Index
		Sampling Rate	Coding Time	Decoding Time	PSNR	SIMM	BRISQUE
City	CS-OMP	0.7867	0.0818	181.2283	26.1894	0.6794	43.4582
	BCS-OMP	0.7867	0.0566	86.31.82	26.3191	0.7448	22.4270
	BCS-SPL	0.7867	0.5851	3.9024	31.5552	0.8226	39.7210
	ABCS-OMP	0.7867	0.8285	2.9680	35.9815	0.9150	15.0426
Airport	CS-OMP	0.2914	0.0824	10.2454	28.1141	0.2486	41.0038
	BCS-OMP	0.2914	0.0505	7.1032	26.6067	0.3481	32.3333
	BCS-SPL	0.2914	0.5697	6.8631	35.9875	0.6513	39.3433
	ABCS-OMP	0.2914	0.4773	3.2485	36.3416	0.8039	28.5709
Forest	CS-OMP	0.8236	0.0853	152.2923	30.4309	0.7029	41.5029
	BCS-OMP	0.8236	0.0580	99.2431	32.9563	0.8465	28.0624
	BCS-SPL	0.8236	0.5935	4.1734	37.8384	0.8236	31.0780
	ABCS-OMP	0.8236	0.8924	3.1422	39.0126	0.9298	25.9349
Cloud	CS-OMP	0.6963	0.0807	55.8707	33.9590	0.6893	20.9542
	BCS-OMP	0.6963	0.0499	60.1737	31.5285	0.7389	26.2792
	BCS-SPL	0.6963	0.5643	5.2329	42.0761	0.8310	38.5014
	ABCS-OMP	0.6963	0.5782	3.4125	40.4628	0.9586	19.7492
Ocean	CS-OMP	0.2624	0.0836	4.7509	43.8920	0.1684	49.5071
	BCS-OMP	0.2624	0.0580	4.5081	35.5093	0.4340	50.3378
	BCS-SPL	0.2624	0.5543	6.0009	40.2244	0.6557	49.1216
	ABCS-OMP	0.2624	0.2922	3.2660	50.8027	0.9925	46.5433

lists the test results of different indicators of the four algorithms for 5 images. The subjective test results are shown in Figure 8.

Analyzed from both subjective and objective perspectives, the traditional CS-OMP algorithm has a slow calculation speed at high sampling rates and serious artifacts in reconstructed images at low sampling rates. The classic BCS-OMP algorithm also has a slow calculation speed at high sampling rates and the block-level artifacts are obvious at low sampling rate, resulting in highly pixelated reconstructed images. Compared with the previous two methods, the optimized BCS-SPL algorithm has greatly improved the operation speed and image reconstruction quality. However, due to multiple Wiener filtering, the edge of the image is not sharp enough, the edge details are not kept intact enough, the structural similarity of objective evaluation indicators is low, and the pixel blocks at the edge are obvious when the simple image content is reconstructed.

Compared with the first three algorithms, the ABCS-OMP algorithm proposed in this article has four main advantages: (1) In the mode selection stage, through the comprehensive analysis of the image, the maximum inter-class variance is used to calculate the image complexity, the parameters are combined with the mode, and the observation mode is selected according to the content to improve the compression efficiency; (2) In the adaptive segmentation stage, the image space is recursively divided into tree structures of different levels based on the edge feature significance factor guided by quadtree, which makes full use of the structure texture of the image itself, divides the image into smooth and edge regions for partition processing, which can alleviate the problem of high pixelation of reconstructed images; (3) In the adaptive sampling rate allocation stage, according to the different information density of the encoded block, the sparsity adaptive is set as the information density function of the given block, and the encoded block is dynamically sampled, which can excavate the compressibility of the image as much as possible and effectively avoid image artifacts; (4) In the adaptive reconstruction stage, OMP calculation is used to accelerate the reconstruction speed by using the prior knowledge of known image block sparsity. The simulation results are consistent with the analysis results. From the subjective and objective evaluation indicators of codec time, PSNR, SIMM and BRISQUE, the fully adaptive coding and decoding compressed sensing framework designed in this paper maintains a peak-to-noise ratio and structural similarity above 35 dB and 0.8 for remote sensing images with complex edge texture, which have good stability. Secondly, especially for ocean images with relatively simple image content, when the sampling rate is 0.26, the peak signal-to-noise ratio reaches 50.8 dB and the structural similarity is 0.99. Compared with the other three methods, the BRISQUE value of the image recovered by ABCS-OMP method is the smallest, with better clarity and less distortion. In the subjective aspect, the reconstructed image has clear edge details and good reconstruction effect, while the block effect is effectively suppressed.

5. Conclusions

In order to solve the problem of incompatibility between traditional information acquisition modes and spaceborne earth observation tasks, this paper draws on the theory of compressed sensing and starts from the general mathematical model of compressed sensing to establish a theoretical model of block compressed sensing and proposes a full process adaptive encoding and decoding framework for remote sensing images. Firstly, this framework uses the maximum between-class variance method to calculate the image content complexity and adaptively selects the coding mode. By extracting image feature factors, a saliency model of the corresponding mode is established to guide the adaptive morphological segmentation of the coding tree block. Then, by using the information density function to detect the sparsity of the encoding block, the sparsity of the image is explored as much as possible to achieve adaptive sampling rate allocation for the image. Finally, the reconstruction matrix and sparse basis are adaptively selected based on the coding block category, and the iteration threshold is set according to the known sparsity. OMP algorithm is used to reconstruct the coding block. Compared with CS-OMP, BCS-OMP, BCS-SPL and other methods, the framework proposed in this article does not require manual preset compression ratio, can select the sampling mode according to the image content and adaptively sets the compression efficiency by detecting sparsity, so as to achieve the requirements of fast compression on the satellite and high fidelity reconstruction on the ground. Under a certain compression ratio, the method proposed in this article has better stability characteristics for images with complex textures, and can effectively alleviate block-level artifacts and enrich image texture details. For simple image content, when the sampling rate is 0.26, the structural similarity increases by 52.3%, the peak signal-to-noise ratio has reached 50.8 dB, and the reconstructed image has better decoding image quality and visual effects.