Multi-View Data-Based Layover Information Compensation Method for SAR Image Mosaic

Abstract

Currently, massive Synthetic Aperture Radar (SAR) images acquired from numerous SAR satellites have been widely utilized in various fields, and image mosaicking technology provides important support and assistance for these applications. The traditional mosaic method selects specific SAR images that can cover the region of interest (ROI) from redundant data to produce “One Map”. However, an SAR image suffers from severe geometric distortion, especially in mountainous areas, which inevitably reduces the utilization of mosaic image. Therefore, a multi-view data-based layover information compensation (MDLIC) method for SAR image mosaic is proposed, aiming to take full advantage of multi-view data to compensate for the missing information in the layover area of the SAR image. This is performed to improve the information content of the mosaic image and realize efficient thematic information extraction and analysis. First, the calculation of the object-space extent of all images and the division of object-space grid are completed on the basis of geometric and radiometric preprocessing. Then, according to the transformation relationship between the object-space and the image-space, the sampling rate image of each image corresponding to the object-space grid is generated, which determines the layover area and the layover degree in each image. Finally, the information compensation strategy is implemented in accordance with the sampling rate image to realize the compensation of the layover information. The feasibility and effectiveness of the MDLIC method are verified by using multiple SAR images from the Chinese Gaofen-3 01 satellite as datasets for experiments. The experimental results indicate that the MDLIC method can obtain mosaic images with richer information compared with the traditional method, while still providing satisfactory results.

1. Introduction

Synthetic Aperture Radar (SAR) is an active Earth observation imaging radar system that can be utilized for all-day and all-weather Earth observation. SAR possesses unique advantages in applications such as resource exploration [1], environmental monitoring [2], disaster assessment [3], ocean monitoring [4], and mapping [5]. It has now become a crucial tool in high-resolution Earth observation and global resource management. In recent years, the number of in-orbit SAR remote sensing satellites has significantly increased. Currently, massive SAR images acquired from numerous SAR satellites are continuously stored in historical databases, providing ample, and sometimes highly redundant, coverage of the region of interest (ROI).

Generally, the coverage of a single SAR image is relatively limited due to the limitations of the SAR sensor design [6], especially when dealing with high-resolution SAR images where the coverage is often smaller. Meeting the diverse needs of various research and applications is increasingly challenging. Consequently, the acquisition of a large-range SAR image can be achieved by mosaicking multiple SAR images. Traditional mosaicking consists of four aspects [7]: SAR image registration, radiometric normalization, seamline generation, and image blending. SAR image registration ensures that multiple SAR images to be mosaicked maintain consistent geometry, which is a fundamental requirement for mosaicking. Radiometric normalization minimizes the radiometric inconsistencies between images in SAR images mosaic, ensuring visually satisfactory mosaic results. To achieve a seamless mosaic, seamline generation is employed to find the optimal seamline location among the images. Based on the generated seamlines, image blending is applied to eliminate slight radiometric differences near the seamlines, further enhancing the visual performance. Image mosaicking is often an essential task for various applications, including geographical mapping, resource monitoring, environmental assessment, and disaster monitoring. Traditional mosaic method selects those specific SAR images covering the ROI from redundant data to produce a unified map, following the aforementioned four steps.

However, due to the terrain undulation and the side-looking and range imaging characteristics of SAR, geometric distortions, including foreshortening, layover, and shadow, are prevalent in an SAR image, especially in mountainous areas [8]. The foreshortening phenomenon can be mitigated by radiometric terrain correction and ortho-rectification methods, while the layover and shadow phenomena cannot be eliminated by conventional processing methods based on a single SAR image. The presence of layover and shadow areas in radar image results in the loss of valuable information. Both layover and shadow can compromise the quality and accuracy of SAR images, impacting their subsequent applications and the overall effectiveness of SAR mosaic images. This, in turn, diminishes the utility of mosaic images.

In recent decades, many scholars have explored methods for detecting layover and shadow areas, including the wavenumber shift method [9], the threshold segmentation method [10,11], the edge sharpening method [12], geometric models combined with the morphological method [13], and the multi-stages detection method [14]. These methods are designed to identify layover and shadow areas, followed by masking, and aim to prevent or minimize errors in the geocoding or interpretation of SAR images. However, they do not address the problem of information loss and also impose limitations on the subsequent application of SAR image.

Currently, scholars have also delved into the compensation of layover and shadow to recover or estimate missing information using other sources of data or prior knowledge. These methods contribute to enhancing the visual quality and quantitative performance of SAR images. One effective method is dual-aspect compensation, utilizing the pixel gray values of normal areas in one image to compensate for layover and shadow areas in another image based on SAR images of two view directions in the same area. TerraSAR-X already provides value-added products such as the ascending and descending merge SAR [15]. This product reduces shadow and layover areas by merging ortho-rectified images from ascending and descending orbit with the same observation angle or images from the same orbit with different incidence angles. Wan et al. [16,17] carried out research on dual-aspect geometric correction for data products like TerraSAR-X and RadarSat-2, and they completed layover compensation experiments. Wang et al. [18] proposed and established a selection strategy and technical process for the optimal combination of incidence angles for dual-aspect compensation, providing the optimal combination of incidence angles for three terrain conditions: gentle, moderate, and steep. Liu et al. [19] also achieved dual-aspect compensation for a Chinese SAR satellite and obtained preliminary results. Wang et al. [8] proposed a layover compensation method for regional spaceborne SAR image without ground control points, which is essentially a dual-aspect compensation method.

The key to realizing dual-aspect compensation for SAR images is to generate layover mask image and shadow mask image based on range-Doppler (RD) model. The RD model, as a rigorous geometric calibration model, demands orbit, altitude, or other information from the satellite platform for its establishment, making formulation of the RD model complex. In contrast, the rational function model (RFM), as a general model, is widely employed in photogrammetric processing of remote sensing images due to its simplicity of implementation and standardization [20]. Despite the advantages of RFM, there are only a few methods for generating layover and shadow mask images. Guindon and Adair [21] calculated the image corresponding digital elevation model (DEM) sampling points and the size of the SAR look angle and slant range to detect the layover area. Chen [22] judged whether each DEM resolution cell belongs to the layover and shadow area to produce the corresponding mask image. Wei [23] proposed a layover mask image and shadow mask image determination method based on the relationship between the look angle and the slant angle in the distance direction. However, the above methods require satellite imaging parameters (including orbit parameters, resolution, slope distance, Doppler frequency, etc.) and are influenced by DEM errors. Some scholars Soergel et al. [10], Ren et al. [11], Han et al. [24] utilized amplitude threshold segmentation as a judgment method for layover mask image and shadow mask image. Due to the random speckle noise inherent in SAR images, it is difficult to extract shadow and layover accurately by threshold segmentation or digital image processing. Wang et al. [8] implemented geometric calibration using RFM instead of the RD model and proposed the RFM-based layover mask area localization method to complete dual-aspect layover compensation. This method required specifying the the master image data and slave image data during the processing, setting the layover threshold, and also taking into account the overlapping rate between images, thus making it less advantageous. The process of dual-aspect information compensation based on the RD model to generate the mask image is complicated and requires the imaging parameter information of an SAR sensor [16,18]. However, the imaging parameters of different SAR sensors vary and obtaining SAR sensor imaging parameter information is challenging, limiting the widespread use of this processing method. The extracted layover mask cannot distinguish the degree of information loss in the layover area. In the area where both master image data and slave image data have layover, optimization is not possible. Dual-aspect information compensation requires filtering the data to achieve the best combination of incidence angles. Even after the dual-aspect information compensation, some missing information areas still exist in the mosaic image [18]. This is because the adopted dual-aspect SAR images have limited information acquisition capacity and are unable to fully compensate for the missing information caused by layover and shadow. Additionally, specifying the master and slave images [8] in this process may result in the wastage of other image information. In general, the application of SAR mosaic images with dual-aspect compensation is also constrained.

The focus of this paper is on the production of SAR mosaic images with rich information using multi-view data. Specifically, the rich information is the information compensated into the layover areas, which restores the information lost in the mosaic due to layover. To achieve this, a multi-view data-based layover information compensation (MDLIC) method for SAR image mosaic is processed. For multiple SAR ascending and descending orbits images of the ROI, the SAR image registration based on RFM and radiometric normalization processing are first completed. Then, a multi-view information compensation model is constructed on the basis of sampling analysis of the SAR multi-view data. The information-compensated SAR images are subjected to seamline generation processing and image blending to finally obtain a mosaic image with rich information.

The main contributions are as follows:

• A MDLIC method is introduced to eliminate the distortions in the layover area and restore the information in SAR mosaic images.
• The method initiates from the essence of the layover phenomenon, which is not affected by speckle noise or pixel gray values, so that the layover area and the degree of layover can be detected quickly according to the sampling rate image.
• The proposed method is independent of terrain parameters and does not necessitate consideration of the overlapping relationships between multi-view data. This characteristic enhances the efficiency and utilization of multi-view data, contributing to the robustness and versatility of the MDLIC method.

The remainder of this paper is organized as follows. The relevant theories and proposed framework are introduced in Section 2. The experimental results are demonstrated and analyzed in Section 3. Conclusions are drawn in Section 4.

2. Materials and Methods

2.1. Overview

The workflow of this paper is shown in Figure 1.

First, registration and radiometric normalization are performed for multiple single look complex (SLC) images of ascending and descending orbits, which can be regarded as the radiometric and geometric preprocessing stage and the necessary step of mosaic. Subsequently, the projection extent of all images in the object-space is computed based on the refined rational function coefficients (RPCs). Following this, utilizing DEM data and the transformation relationship between object-space and image-space, sampling rate images are generated in both image-space and object-space, corresponding to the object-space grids for each image. These images play a crucial role in determining the layover area and its degree in each image. Concurrently, the establishment of the object-space grid size is essential, aligning with the resolution of the SAR image. Subsequent to the grid size determination, the optimal SAR image corresponding to each pixel of every image is identified based on the sampling rate in object-space. The pixel from the optimal SAR image is then considered as the corresponding position in the compensated image. Following these steps, the layover information compensation is completed for each image post-ortho-rectification. Finally, the compensated images undergo seamline generation and image blending processes, culminating in the creation of a mosaic image. This comprehensive approach significantly enhances the information content within the mosaic images, providing considerable advantages for subsequent photogrammetric applications.

2.2. Geometric and Radiometric Preprocessing of SAR Image

The fundamental task of mosaicking, image registration, involves aligning images acquired at different times and under varying imaging conditions. In this paper, a geometry-aware image registration method [25] is employed. This method extracts inherent orientation features and focuses on geometry-invariant areas to ensure consistent geometry among multiple SAR images to be mosaicked.

Ideally, the radiometric intensity in overlapping areas of adjacent images should be consistent after radiometric calibration [26,27]. However, several factors such as radar wave signal attenuation, antenna pattern change, transmission or receiving power gain errors, and imaging processor gain error [26] introduce radiometric differences between images, especially for data from different orbits. Radiometric normalization is implemented to eliminate the differences between images.

The level-1A products of Gaofen-3 01 satellite are SLC images. The radiometric calibration equation for obtaining the backscattering coefficient from the SLC image is as follows [28]:

$${\sigma }^0=10{log}_{10}[P\times {(Qv/32767)}^2]-K_{dB}$$

(1)

where ${\sigma }^0$ is the backscattering coefficient in dB; P is power of complex image and $P=I^2+Q^2$, I represents real channel of image, and Q as the imaginary channel; $Qv$ is the maximum value before image quantification, and $K_{dB}$ is the calibration constant. $Qv$ and $K_{dB}$ can be retrieved from the metadata file in Extensible Markup Language (XML) format of Gaofen-3 01 satellite. In Equation (1), the value of $Qv$ reflects radiometric characteristic of Gaofen-3 01 satellite.

In accordance with Equation (1), a radiometric principle-based radiometric normalization method [29] is employed to eliminate the radiometric differences between images by analyzing the radiometric principle of SAR.

2.3. Geometric Distortions in SAR Image

Due to the terrain undulation and the side-looking and range imaging characteristic of SAR, SAR images suffer from geometric distortions that differ from optical images [30,31,32,33], such as foreshortening, layover, and shadow, especially in mountainous areas. As shown in Figure 2, the undulating terrain can lead to geometric distortion in slant range SAR image.

For slopes oriented towards the SAR sensor, when the local terrain slope angle is smaller than the local incidence angle, the length of such slopes in the SAR image is shorter than that of flat terrain, which means that the range resolution on such slopes becomes worse. Conversely, for the slopes facing away from the SAR sensor, when the local terrain slope angle is smaller than the complementary angle of the local incidence angle, the range resolution on such slopes becomes better in the SAR images than the flat terrain [34]. This phenomenon is known as foreshortening, and the degree of foreshortening is related to the slope and the incidence angle. Foreshortening is most pronounced when the local terrain slope angle is equal to the local incidence angle. As illustrated in Figure 2, the imaging plane distance (slant range) between $d^{\prime }$ and $e^{\prime }$ is smaller than the distance on the actual slope surface from d to e.

When the local terrain slope angle surpasses the local incidence angle, the bottom and the top of such slopes are reverse imaged, which is known as the layover phenomenon. In Figure 2, specifically in area $b^{″}-a-b-a^{″}$, the local terrain slope is larger than the incidence angle and the top of the mountain is closer to the sensor than the bottom, so layover occurs. The result is that multiple signals with the same range and Doppler frequency are integrated within a single resolution cell of SAR image [8,35]. In object-space, the layover area can be classified into active layover area and passive layover area [30,31]. In the range direction, each active layover area corresponds to two passive layover areas. In Figure 2, area $a-b$ is termed the active layover area and area $b^{″}-a$ and $b-a^{″}$ are called the passive layover areas, where area $b^{″}-a$ is near range passive layover area, and area $b-a^{″}$ is far range passive layover area. The near range passive layover area is closer to SAR sensor than the far range passive layover area. Active layover area is the source for layover, and the positions of two passive layover areas in SAR image are completely covered by the active layover area, while active layover areas and passive layover areas can be distinguished in object-space, they cannot be differentiated in SAR image-space. Generally, layover areas appear significantly brighter than the rest of the SAR image.

Nevertheless, when the local terrain slope angle surpasses the complementary angle of the local incidence angle, such steep slopes are completely prevented from receiving the SAR signal by the mountain itself. This gives rise to the shadow phenomenon, resulting in a dark area in the SAR image. The shadow area can be classified into active shadow area and passive shadow area. In contrast to the two passive areas in the layover, each active shadow area has only one passive shadow area located at the end of the active shadow area, unreachable by the radar beam since it is blanked by the active shadow area. As depicted in Figure 2, when the SAR signal cannot reach the Earth’s surface, a shadow occurs, where area $b-c$ is the active shadow area and area $c-d$ is the passive shadow area. The active shadow area and passive shadow area in SAR image-space correspond to different areas in object-space, which differs from the image characteristics of the layover area. This distinction arises because active layover area and passive layover area in object-space correspond to the same image area.

Generally, radiometric and geometric effects resulting from foreshortening phenomenon can be mitigated through radiometric terrain correction and ortho-rectification. However, the challenge lies in the difficulty of extracting meaningful information from layover and shadow areas. This limitation is a critical factor hindering the effective application of SAR in mountainous regions and significantly constrains the usability of mosaic images. It is important to note that traditional single-image-based methods are insufficient in eliminating the adverse effects of layover and shadow.

Wang et al. [18] have utilized DEM data to simulate the impact of different incidence angles on the proportion of layover areas and shadow areas under different topographic conditions. Their findings indicate that when the incidence angle is less than 42°, and layover is the main factor causing information loss. Presently, a multitude of SAR satellites in orbit, such as Gaofen-3 series satellites, TerraSAR-X, COSMO-SkyMed, and RadarSat-2, with incidence angles typically falling within the range of 10°–60°, [36]. Particularly in the high-resolution imaging mode, the incidence angle often falls within the range of 20°–50°. This implies that for high-resolution SAR images, the layover phenomenon tends to be more pronounced and visually impactful than the shadow phenomenon in most scenes. Consequently, the focus of the proposed method in this paper is on accomplishing the compensation of layover information.

2.4. Multi-View Information Compensation Model

2.4.1. Projection Extent in Object-Space

The single look SAR images with radiometric consistency and refined RPCs are obtained after geometric and radiometric preprocessing. Utilizing the image size and positioning parameter file, the projection extent of each image in object-space can be calculated. Assuming that the original size of image i is $W_i$ columns and $H_i$ rows, the quadrangle vertices coordinates in image-space are $(0,0)$, $(W_i,0)$, $(0,H_i)$, and $(W_i,H_i)$, then the corresponding coordinates in object-space can be obtained as ${(Lon1,Lat1)}_i$, ${(Lon2,Lat2)}_i$, ${(Lon3,Lat3)}_i$, and ${(Lon4,Lat4)}_i$ based on the positioning parameter information, as illustrated in Figure 3.

The maximum and minimum values of coordinates in object-space are calculated to obtain the object-space extent of image i as $({[Lo{n}_{min},Lo{n}_{max}]}_i,{[La{t}_{min},La{t}_{max}]}_i)$. The object-space extent of all multi-view images to be processed is calculated and merged to obtain the extent of the final mosaic image as $([Lo{n}_{min},Lo{n}_{max}],[La{t}_{min},La{t}_{max}])$, illustrated in Figure 4. This extent also represents the coverage of the ROI.

2.4.2. Generation of Sampling Rate Image

As previously discussed, the side-looking imaging mechanism of SAR introduces distinctive geometric distortions since SAR measures range from space. This leads to pronounced geometric distortions, including bright layovers caused by multiple scatterers from different parts of the terrain. In the layover area, the value of a specific image pixel is evidently the sum of the backscattered energy from multiple ground units. Conventional correction methods based on a single radar image are unable to eliminate this phenomenon, but it can be addressed effectively by utilizing multiple images. Particularly, images captured from ascending and descending orbits or from left-looking and right-looking directions (referred to as the opposite side) inherently reflect characteristics of opposite aspects, presenting dissimilar yet complementary information, as illustrated in Figure 5.

Once the object-space extent of all images is determined, the object-space grid size is assigned according to the image resolution, which is the basis for dividing the object-space grid. Subsequently, leveraging the DEM data, the corresponding image-space coordinates in different images are calculated grid by grid according to the object-space grids, and the sampling times of the image coordinates are counted. The number of sampling times of pixel at $(x,y)$ of image i and j are $N_{(x,y)}^i$ and $N_{(x,y)}^j$. Next, the image pixel sampling times are converted to the image sampling rate. The number of sampling times of pixel at $(x,y)$ of image i is $N_{(x,y)}^i$, so the sampling rate can be denoted as

$$\begin{array}{c}\hfill L_{(x,y)}^i=\frac{1}{N_{(x,y)}^i}=\frac{1}{T\left\{R\right(X,Y,Z\left)\right\}}\end{array}$$

(2)

where $N_{(x,y)}^i\ne 0$; $(X,Y,Z)$ are the latitude, longitude, and height of each object-space grid; $R(X,Y,Z)$ represents rational function model [25] that relates image coordinates $(x,y)$ to object coordinates $(X,Y,Z)$; $T\left\{R\right(X,Y,Z\left)\right\}$ is a function that counts the number of grids corresponding to $(x,y)$.

In Figure 6a, the three object-space grids marked as yellow are transformed into the image-space coordinates of different images. In Figure 6b1, the three grids correspond to the same image coordinates, resulting in $N_{(x,y)}^i=3$ and $L_{(x,y)}^i=1/3$. However, in Figure 6b2, the three grids correspond to different image coordinates. This implies that image i exhibits layover phenomenon, while image j does not.

Figure 6b1,b2 represent sampling rate images corresponding the object-space grids in the image-space. As for the sampling rate image in object-space, it can be obtained by converting the sampling rate image from image-space to object-space. Based on the sampling rate image, the extent of the layover area and the degree of layover can be quickly acquired. In a sampling rate image, the area where the pixel value is greater than 0 and less than 1 represents the layover area, with lower pixel values indicating more severe layover phenomena. It is important to note that the proposed method does not distinguish between active layover and passive layover in the sampling rate image; both are uniformly labeled as layover.

Due to the limited precision of DEM data, errors may arise in the calculation of the sampling rate. The layover mask extracted from the sampling rate image might exhibit “islands”, “holes”, and “zigzags” phenomena. To mitigate these issues and prevent any adverse impact on the visual quality of the compensation results, morphological operations like erosion, dilation, closing, and opening can be applied to these masks, as expressed in Equation (3) [37]. The erosion operation minimizes the extent of the target area and can be used to eliminate “islands” phenomena; the dilation operation increase the extent of the target area and can be used to fill “holes” in the target area, and the opening and closing operations can smooth the contour of the target area to achieve the elimination of “zigzags”.

$$\begin{array}{cc}\hfill A\ominus B& =\left\{z\right|{\left(B\right)}_z\subseteq A\}\hfill \\ \hfill A\oplus B& =\left\{z\right|{\left(B\right)}_z\cap A\ne ⌀\}\hfill \\ \hfill A\circ B& =\left(A\ominus B\right)\oplus B\hfill \\ \hfill A•B& =(A\oplus B)\ominus B\hfill \end{array}$$

(3)

where A represents the original image, B is the structure element, and z represents the output pixel value. $A\ominus B$, $A\oplus B$, $A\circ B$, and $A•B$ are erosion, dilation, closing, and opening operation, respectively.

In this paper, opening and closing operations are successively applied to accomplish the morphological processing, as depicted in Equation (4) and illustrated in Figure 7.

$$M^{\prime }=(M\circ B)•B$$

(4)

where structure element B is a square of size 5; the definitions of M and $M^{\prime }$ are shown in Figure 7.

2.4.3. Layover Information Compensation Strategy

After ortho-rectification, a common mosaic strategy involves searching for an optimal seamline between the overlapping areas of the images, and then complete the mosaic processing according to the seamline, as shown in Figure 8a. In the overlapping area, it is crucial to determine which image pixels will be utilized as the final mosaic image pixels based on the seamline. For the non-overlapping area, the pixels of the area are used directly as the pixels of the mosaic image. However, such a mosaic image suffers from the loss of useful information due to layover phenomenon. Therefore, MDLIC processing is conducted between the ortho-rectification processing and mosaic processing to enhance the quality of the mosaic image.

In fact, in the overlapping area of sampling rate image, the boundary of the layover area can then be considered as a seamline, forming a closed line. As shown in Figure 8b, for the case of two images, red closed lines represent seamlines. Additionally, the seamline in this case can even be nested, especially as the terrain becomes more complex; the contour of the seamline becomes more complex. When multiple overlapping areas are present, the seamlines can become extremely complex, as determined from the sampling rate images.

Therefore, layover information compensation is essentially a form of mosaicking, akin to traditional mosaicking, aimed at enhancing the information content of SAR images. In the context of SAR images, both information compensation and traditional mosaicking together constitute a comprehensive mosaicking process.

The layover information compensation and traditional mosaicking are conducted sequentially. For layover information compensation, the processing of non-overlapping areas is consistent with the traditional mosaic method, while in the multiple overlapping areas, the final pixels are determined according to each sampling rate image obtained above, and the pixels with large sampling rate should be selected. As shown in Figure 6, the three pixels marked yellow of image j should be used as the final mosaic image pixels, instead of the pixels marked yellow in image i which are repeatedly assigned three times.

Assuming that there are two SAR ortho-rectified image i and image j in the ROI, the determination of the image source for each pixel in the compensated image is based on the sampling rate when performing layover information compensation in the overlapping area of i and j, as expressed in Equation (5).

$$\begin{array}{c}{I}_{(x^{\prime },y^{\prime })}=f\left(I_{(x^{\prime },y^{\prime })}^i,I_{(x^{\prime },y^{\prime })}^j\right)\hfill \\ =\left\{\begin{array}{c}{I}_{(x^{\prime },y^{\prime })}^i,(L_{(x^{\prime },y^{\prime })}^i>L_{(x^{\prime },y^{\prime })}^j)\\ I_{(x^{\prime },y^{\prime })}^j,(L_{(x^{\prime },y^{\prime })}^j>L_{(x^{\prime },y^{\prime })}^i)\\ g\left(I_{(x^{\prime },y^{\prime })}^i,I_{(x^{\prime },y^{\prime })}^j\right),(L_{(x^{\prime },y^{\prime })}^i=L_{(x^{\prime },y^{\prime })}^j)\end{array}\right.\hfill \\ g\left(I_{(x^{\prime },y^{\prime })}^i,I_{(x^{\prime },y^{\prime })}^j\right)=\left\{\begin{array}{c}{I}_{(x^{\prime },y^{\prime })}^i,\left(\mathrm{When}\mathrm{compensating}\mathrm{for}\mathrm{image}i\right)\hfill \\ I_{(x^{\prime },y^{\prime })}^j,\left(\mathrm{When}\mathrm{compensating}\mathrm{for}\mathrm{image}j\right)\hfill \end{array}\right.\hfill \end{array}$$

(5)

where $I_{(x^{\prime },y^{\prime })}^i$ and $I_{(x^{\prime },y^{\prime })}^j$ are the pixel values of two SAR images at the same spatial coordinate $(x^{\prime },y^{\prime })$. $I_{(x^{\prime },y^{\prime })}$ represents the pixel value at coordinate $(x^{\prime },y^{\prime })$ in overlapping area, which is also the pixel value with the optimal sampling rate at coordinate $(x^{\prime },y^{\prime })$. In addition, $L_{(x^{\prime },y^{\prime })}^i$, and $L_{(x^{\prime },y^{\prime })}^j$ are the pixel sampling rates of two images at the same spatial coordinate $(x^{\prime },y^{\prime })$. Moreover, the two sampling rate images corresponding to image i and j can also finish the mutual sampling rate update using Equation (5), as shown in Figure 9. The sampling rates less than 1 are visibly reduced in Figure 9c,d, indicating the compensation of layover information in the SAR images. This confirms the effectiveness of Equation (5).

For two SAR images, Equation (5) can be applied with the purpose of selecting the pixel with the maximum sampling rate and layover information compensation. Consequently, for all multi-view data in ROI, layover information compensations of all SAR images can be completed using this method, as depicted in Equation (6).

$$I_{(x^{\prime },y^{\prime })}=f\left(I_{(x^{\prime },y^{\prime })}^1,I_{(x^{\prime },y^{\prime })}^2,\dots ,I_{(x^{\prime },y^{\prime })}^n\right)$$

(6)

where $I_{(x^{\prime },y^{\prime })}$ represents the pixel value in the overlapping area; $I_{(x^{\prime },y^{\prime })}^n$ represents the pixel value of the image n at the same spatial coordinate $(x^{\prime },y^{\prime })$; and n is the number of images.

In this study, the compensated mosaic images primarily serve applications such as visual interpretation, image fusion, and change detection, rather than quantitative inversion of remote sensing. Although layover information compensation may alter the orientation dependence of certain non-homogeneous scatterers, it is essential for the visual interpretation of mosaic images. The presence of layover can significantly impact interpretation results, making compensation crucial for accurate visual assessments.

3. Experimental Results and Analysis

3.1. Experimental Dataset

To validate the effectiveness of the proposed method, three sets of experiments were conducted using several SAR images from Chinese Gaofen-3 01 satellite. The Gaofen-3 01 satellite is the first civil microwave remote sensing imaging satellite listed in the “National High-resolution Earth Observation System Major Project” and is the first C-band multi-polarization SAR satellite with a nominal resolution of 1 m to 500 m. The three datasets are as follows:

(1) $Dataset 1$: The ROI is located between 29.41° and 30.28°N latitude and 106.88° and 107.94°E longitude. There are 8 ascending orbit images and 6 descending orbit images in the area, totaling 14 images, which were collected from November 2019 to July 2022.

(2) $Dataset 2$: The ROI is located between $33.{86}^{\circ }$ and 34.78°N latitude and 108.59° and 109.60°E longitude. There are 6 ascending orbit images and 6 descending orbit images in the area, totaling 12 images, which were collected from November 2019 to June 2022.

(3) $Dataset 3$: The ROI is located between 30.85° and 30.51°N latitude and 111.96° and 112.61°E longitude. There are three ascending orbit images and three descending orbit images in the area, totaling six images, which were collected from October 2021 to August 2022.

All the SAR images are SLC products from the Gaofen-3 01 satellite, and the imaging mode of the image is ultra-fine stripe (UFS). Detailed information is presented in

Table 1. Detailed information of three datasets.

Dataset	Orbit (Direction)	OrbitID	Imaging Mode	Nominal Resolution	Incidence Angle	Imaging Date
1	ASC	019813	UFS	3 m	32.57°	15 May 2020
	ASC	019813	UFS	3 m	32.57°	15 May 2020
	ASC	029845	UFS	3 m	31.57°	11 April 2022
	ASC	029845	UFS	3 m	31.57°	11 April 2022
	ASC	024237	UFS	3 m	24.96°	18 March 2021
	ASC	024237	UFS	3 m	24.96°	18 March 2021
	ASC	031170	UFS	3 m	21.27°	12 July 2022
	ASC	031170	UFS	3 m	21.27°	12 July 2022
	DEC	017037	UFS	3 m	30.54	4 November 2019
	DEC	017037	UFS	3 m	30.54°	4 November 2019
	DEC	022399	UFS	3 m	23.76°	10 November 2020
	DEC	022399	UFS	3 m	23.76°	10 November 2020
	DEC	023725	UFS	3 m	31.57°	10 February 2021
	DEC	023725	UFS	3 m	31.57°	10 February 2021
2	ASC	022637	UFS	3 m	31.57°	27 November 2020
	ASC	022637	UFS	3 m	31.57°	27 November 2020
	ASC	026990	UFS	3 m	34.49°	25 September 2021
	ASC	026990	UFS	3 m	34.49°	25 September 2021
	ASC	030997	UFS	3 m	32.57°	30 June 2022
	ASC	030997	UFS	3 m	32.57°	30 June 2022
	DEC	017456	UFS	3 m	28.40°	3 December 2019
	DEC	017456	UFS	3 m	28.40°	3 December 2019
	DEC	017210	UFS	3 m	24.96°	16 November 2019
	DEC	017210	UFS	3 m	24.96°	16 November 2019
	DEC	023970	UFS	3 m	31.57°	27 February 2021
	DEC	023970	UFS	3 m	31.57°	27 February 2021
3	ASC	030479	UFS	3 m	30.54°	25 May 2022
	ASC	031661	UFS	3 m	38.81°	15 August 2022
	ASC	027808	UFS	3 m	31.57°	28 March 2022
	DEC	027808	UFS	3 m	19.99°	20 November 2021
	DEC	027808	UFS	3 m	19.99°	20 November 2021
	DEC	027285	UFS	3 m	34.49°	15 October 2021

[36,38,39], where DEC means descending, and ASC is the abbreviation of ascending. And the geometric distribution can be found in Figure 10. It is obvious that the dataset contains various viewing conditions, including different incidence angles and different-side views, which can be helpful for a comprehensive evaluation. The DEM data used in the experiments is the public-available ALOS World 3D-30 m (AW3D30) with 30 m resolution over the world, which is released by the Japan Aerospace Exploration Agency, Tokyo, Japan, with a resolution of about 30 m and an accuracy of about 5 m [40].

3.2. Experimental Results and Visual Assessments

The above three datasets undergo identical preprocessing steps, followed by the implementation of the MDLIC method. The results obtained from the proposed method are then compared with those from Refs. [8,24]. In Ref. [24], the threshold for layover area detection is set as several times the mean value of the image, with this paper choosing three times the mean value as the threshold. Ref. [8] involves designating the master image data and slave image data, with the descending orbit image designated as the master image data for mosaicking.

Moreover, the proposed method does not necessitate specifying the master image data and slave image data, and the descending orbit data is also mosaicked together for comparison purposes. However, in practical applications, preference should be given to scenes with less layover in the image. The experimental results are illustrated in Figure 11, Figure 12 and Figure 13.

In these three datasets, the layover phenomenon of mosaic images is obvious, as shown in Figure 11b1,c1, Figure 12b1,c1 and Figure 13b1,c1. It is apparent that different information compensation methods can effectively eliminate the layover phenomenon to varying degrees.

To evaluate the performance of the proposed method, visual interpretation is performed based on uncompensated SAR mosaic images of three datasets, in which layover areas are labeled, and the final results are taken as the true value. Figure 14, Figure 15 and Figure 16 show the layover areas produced by MDLIC method, along with a detailed comparison of the ground truth with the results of different methods. Non-layover areas are marked in green, while the layover areas are marked in red. For comparison purposes, the different degrees of layover areas produced by MDLIC method are also marked in red.

As can be seen that three methods have large differences in the results of layover detection. The Ref. [24] method produces a large number of missed judgments, compared to the true value of layover, and detected layover areas are highly discretized, as shown in Figure 14c2, Figure 15b2, and Figure 16c2. Meanwhile, in Figure 15b2,c2, the building areas are identified as the layover areas. The reason for the above phenomenon is that the method is based on the threshold, which recognizes high brightness areas larger than the threshold as layover areas. However, high brightness areas can also be caused by double-bounce when the radar signal encounters smooth surfaces oriented at ${90}^{\circ }$ angles to each other. More significantly, inherent random scattering noise in SAR images makes it challenging to accurately detect layover through threshold segmentation or digital image processing. Therefore, the Ref. [24] method has limited effectiveness in layover information compensation.

In Figure 14b3 and Figure 15c3, layover areas detected by Ref. [8] method are relatively small compared to the actual layover areas. This method tends to produce fewer false positives but can result in more misses, especially in areas with significant terrain undulation, as shown in Figure 14c3 and Figure 16b3.

Regarding MDLIC method, the detected layover areas are slightly larger compared to the actual layover areas, as shown in Figure 16b4,c4. Moreover, in moderately undulating terrain areas, the detection results may be misjudged, as illustrated in the left half of Figure 14b4.

The focus of the research in this paper is to produce SAR mosaic images with rich information content and not just to detect layover areas. The proposed method aims to identify additional layover areas while maintaining fundamental consistency with actual layover areas, leading to a more effective compensation effect in the resulting images after compensation, as depicted in Figure 14c4 and Figure 15b4. From a visual perspective, it is crucial to identify more additional layover areas during layover compensation. Moreover, even if some areas are misjudged, the MDLIC method utilizes data with diverse incidence angles and viewing angles, leveraging the comparison of sampling rates to determine the best information. This approach maximizes the utilization of information from each image, mitigating the impact of misjudged areas on the compensated mosaic image.

The compensation effect of the MDLIC method surpasses that of the Ref. [24] method in both overall and local details across all three datasets. Compared to the Ref. [8] method, the MDLIC method can detect more layover areas, significantly enhancing the compensation effect. Additionally, MDLIC does not require specifying master and slave images or considering the overlapping rate between images, making it more versatile for multi-view data.

The proposed method is analyzed using DEM data with different accuracies and the layover results are presented in Figure 17. In Figure 17b, it can be seen that when the DEM data accuracy decreases, the main layover areas are unchanged, but significantly fewer layover areas are detected as well as a “zigzags” phenomenon in areas with little terrain undulation. Meanwhile, the edges of the layover areas become less smooth with 90 m DEM data, as shown in Figure 17d. Therefore, the primary error of the MDLIC method originates from the DEM, and the MDLIC method performs better with highly accurate DEM data.

In summary, the proposed MDLIC method demonstrates effective detection of layover areas and efficient compensation of missing information, resulting in the generation of mosaic images with enriched information content.

3.3. Statistical Analysis

To further evaluate the performance of the proposed method, the indicators including mean value, standard deviation, non-layover precision (NLP), layover precision (LP), critical success index (CSI), and overall accuracy (OA) are selected to analyze the results quantitatively. NLP, LP, CSI, and OA are derived in Equation (7) [41].

$$\begin{array}{cc}\hfill \mathrm{NLP}& =\frac{TN}{TN+FN}\hfill \\ \hfill \mathrm{LP}& =\frac{TP}{TP+FP}\hfill \\ \hfill \mathrm{CSI}& =\frac{TP}{TP+FP+FN}\hfill \\ \hfill \mathrm{OA}& =\frac{TP+TN}{TP+TN+FP+FN}\hfill \end{array}$$

(7)

where $TN$, $TP$, $FN$, and $FP$ are the number of correctly classified non-layover, the number of correctly classified layover, the number of incorrectly classified non-layover, and the number of incorrectly classified layover, respectively, as shown in

Table 2. Confusion matrix for layover detection.

	Non-Layover (Result)	Layover (Result)
Non-layover (True)	True Negative (TN)	False Positive (FP)
Layover (True)	False Negative (FN)	True Positive (TP)

. CSI, also called the threat score, is a verification measure of categorical forecast performance equal to the total number of correct event forecasts divided by the total number of storm forecasts plus the number of misses, and it varies from 0 (bad) to 1 (good). OA is the ratio of the number of correctly classified pixels to the number of all pixels, and is a commonly used measure of classification results.

The mean and standard deviation of the uncompensated mosaic image of $Dataset 1$ are 104.48 and 140.87, respectively. The uncompensated mosaic image of $Dataset 2$ has a mean of 45.95 and a standard deviation of 73.66. As for the $Dataset 3$, the mean of uncompensated mosaic image is 97.10, with a standard deviation of 137.12.

Table 3. Statistical results of different methods.

Dataset	Indicator	Ref. [24] Method	Ref. [8] Method	Proposed Method
1	Mean	95.35	103.58	102.30
	Std	113.74	139.72	137.59
	Layover pixels	32,142,854	37,103,771	119,057,743
	TN	496,015,280	512,703,490	491,201,894
	TP	9,110,946	30,760,073	91,212,449
	FN	84,272,170	62,623,043	2,170,667
	FP	23,031,908	6,343,698	27,845,294
	NLP	85.48%	89.12%	99.56%
	LP	28.35%	82.90%	76.61%
	CSI	7.83%	30.85%	75.24%
	OA	82.48%	88.74%	95.09%
2	Mean	42.43	45.26	44.68
	Std	57.59	73.09	72.93
	Layover pixels	20,310,410	12,301,361	26,539,520
	TN	503,340,284	518,276,633	517,273,328
	TP	4,410,273	11,337,573	24,572,427
	FN	20,228,625	13,301,325	66,471
	FP	15,900,137	963,788	1,967,093
	NLP	96.14%	97.50%	99.98%
	LP	21.71%	92.17%	92.59%
	CSI	10.88%	44.29%	92.35%
	OA	93.36%	97.38%	99.62%
3	Mean	93.82	95.99	94.80
	Std	120.60	135.36	135.63
	Layover pixels	6,738,977	6,830,917	46,875,098
	TN	269,271,163	271,988,665	263,931,258
	TP	2,262,882	5,072,324	37,059,098
	FN	1,967,093	32,567,194	580,420
	FP	4,476,095	1,758,593	9,816,000
	NLP	88.39%	89.31%	99.78%
	LP	33.58%	74.26%	79.05%
	CSI	5.38%	12.88%	78.09%
	OA	87.20%	88.98%	96.66%

shows the statistical results of mean and standard deviation using different methods.

For mean and standard deviation, the lowest values are observed in all three datasets after applying the Ref. [24] method, suggesting compensation for a relatively large number of high-brightness pixels. However, high-brightness areas may not exclusively represent layover but could include double-bounce areas. In comparison to the Ref. [8] method, the MDLIC method shows slightly smaller mean and standard deviation, indicating compensation for slightly more high-brightness pixels than the Ref. [8] method. It is important to note that the lowest mean and standard deviation do not necessarily imply optimal processing; these metrics reflect high-brightness pixels before and after compensation without directly reflecting the compensation effectiveness. The assessment based on these metrics aligns with the visual assessment results.

To provide a more intuitive comparison of the effectiveness of different methods, NLP, LP, CSI, and OA are calculated for different methods, as shown in Table 3.

In $Dataset 1$, the Ref. [8] method detects more layover pixels than the Ref. [24] method, while the MDLIC method outperforms both, detecting about three times as many layover pixels as the Ref. [8] method. $Dataset 1$ presents challenging terrain undulation areas, impacting layover detection. The LP value for the Ref. [24] method is the lowest, indicating the worst performance. Although the LP value of the Ref. [8] method is higher than that of the MDLIC method, the NLP value is lower, implying that the Ref. [8] method detects fewer layover pixels than the MDLIC method. In terms of CSI and OA values, the MDLIC method performs better than the other two methods.

For $Dataset 2$, the Ref. [24] method detects more layover pixels than the Ref. [8] method due to urban areas, but the MDLIC method still outperforms both by detecting the most layover pixels. With fewer undulating terrains in $Dataset 2$, NLP values are high for different methods. The performance of the Ref. [8] method and MDLIC method is closer, but the MDLIC method is slightly better. Additionally, the Ref. [24] method shows the lowest LP due to the presence of urban areas.

$Dataset 3$ exhibits undulating terrain and flat areas, favoring the performance of the Ref. [24] method slightly better than the other two methods. However, the MDLIC method achieves the highest LP and NLP in this dataset, indicating optimal compensation effectiveness.

Table 3 reveals that the NLP of the MDLIC method is the highest in the three datasets. Different terrains influence the LP value of the method, but OA is the highest, demonstrating the effectiveness of the MDLIC method. Furthermore, in the three datasets, the proposed method has the highest CSI, confirming its superior performance.

In summary, the experimental results indicate that the MDLIC method effectively compensates for missing information due to layover in SAR images, enhancing their visual quality and quantitative performance.

4. Conclusions

In this study, a MDLIC method is proposed for SAR image mosaicking to enhance the information content in layover areas. The method leverages multi-view data to effectively compensate for missing information caused by the layover phenomenon. By establishing a multi-view information compensation model that considers geometrical distortions in SAR images, the sampling rate image of each SAR image is generated based on object-space projection relationships. This sampling rate image reveals the extent and degree of layover in each image. Together with a layover information compensation strategy, the MDLIC method significantly enhances texture information in SAR images captured from multiple ascending and descending orbits. The proposed method integrates layover information compensation with traditional mosaicking, creating a comprehensive mosaicking approach for SAR images. Experimental results demonstrate that the MDLIC method effectively and consistently mitigates the impact of layover phenomenon, yielding improved visual quality and quantitative performance in SAR mosaic images. However, it is noted that the accuracy of DEM data in urban areas, influenced by urban noise and human-made objects, may limit the performance of the MDLIC method in such regions. Moreover, the proposed method provides robust support for SAR mosaic image interpretation. When fused with optical imagery, it can offer valuable information for subsequent tasks such as target identification, feature classification, and other interpretation endeavors.