Reconstructing Snow-Free Sentinel-2 Satellite Imagery: A Generative Adversarial Network (GAN) Approach

Abstract

Sentinel-2 satellites are one of the major instruments in remote sensing (RS) technology that has revolutionized Earth observation research, as its main goal is to offer high-resolution satellite data for dynamic monitoring of Earth’s surface and climate change detection amongst others. However, visual observation of Sentinel-2 satellite data has revealed that most images obtained during the winter season contain snow noise, posing a major challenge and impediment to satellite RS analysis of land surface. This singular effect hampers satellite signals from capturing important surface features within the geographical area of interest. Consequently, it leads to information loss, image processing problems due to contamination, and masking effects, all of which can reduce the accuracy of image analysis. In this study, we developed a snow-cover removal (SCR) model based on the Cycle-Consistent Adversarial Networks (CycleGANs) architecture. Data augmentation procedures were carried out to salvage the effect of the limited availability of Sentinel-2 image data. Sentinel-2 satellite images were used for model training and the development of a novel SCR model. The SCR model captures snow and other prominent features in the Sentinel-2 satellite image and then generates a new snow-free synthetic optical image that shares the same characteristics as the source satellite image. The snow-free synthetic images generated are evaluated to quantify their visual and semantic similarity with original snow-free Sentinel-2 satellite images by using different image qualitative metrics (IQMs) such as Structural Similarity Index Measure (SSIM), Universal image quality index (Q), and peak signal-to-noise ratio (PSNR). The estimated metric data shows that Q delivers more metric values, nearly 95%, than SSIM and PRSN. The methodology presented in this study could be beneficial for RS research in DL model development for environmental mapping and time series modeling. The results also confirm the DL technique’s applicability in RS studies.

1. Introduction

Remote Sensing (RS) technology plays an important role in Earth Observation (EO) and research, and the Sentinel-2 Satellite has been one of the major instruments dedicated to EO within the last decade. Sentinel-2 can acquire optical imagery at high spatial resolution (10–60 m) observations over land and coastal waters. One of its primary goals is to complement other existing satellite missions while improving data availability for end users. It achieves this by providing high-resolution satellite data for real-time monitoring of Earth’s atmosphere and surface, facilitating the detection of climate changes and natural disasters over any given geographical area during satellite passage, among other functions [1]. In a nutshell, leveraging RS information acquired through Sentinel-2 has made EO and timely intervention in natural disaster monitoring more effective [1,2,3]. Interestingly, researchers worldwide have been employing RS techniques to assess the effectiveness and applications of Sentinel-2. Their findings reveal that optical imagery from Sentinel-2 frequently encounters clarity or obstruction challenges due to either partial or absolute cloud cover [4]. However, there is another significant challenge and impediment in satellite RS that has received less attention. In recent years, with an improved study of long-term time series for climate change and hydrology using Sentinel-2, certain features observed as snow have been reported [5,6] to cover specific geographical areas of the land surface. As a result, the quality and accuracy of Sentinel-2 satellite measurements have been reduced. Differentiating between snow and clouds can be challenging when mapping snow coverage using satellite imagery, as they often have remarkably similar appearances and color dis-attributions. The Sentinel Hub Earth Observation browser: “https://www.sentinel-hub.com/explore/eobrowser/ (accessed on 24 January 2024)” now incorporates an algorithm that can systematically select Satellite passage sessions with cloud-free Sentinel-2 imagery. Hence, this leaves us solely with satellite imagery with snow-cover (SC) features. Furthermore, several independent RS researchers have utilized machine learning (ML) as a computational tool to eliminate cloud-cover features from satellite imagery [7,8], resulting in satellite imagery exclusively exhibiting snow-cover (SC) features.

Snow cover is predominantly observed during the winter and early spring seasons of the year. This phenomenon has been one of the most influential factors affecting the quality of satellite imagery, as it obscures crucial surface features. Satellite imagery with SC presents various limitations and challenges. First, it hinders the acquisition of necessary features due to satellite signal interference and contamination, resulting in information loss [6]. Secondly, it complicates image data processing by causing masking effects that can diminish the accuracy of computer vision systems [9]. Lastly, this effect can systematically reduce or limit the availability of satellite images required for data training in deep learning (DL) model development and climate change time series studies [10]. Moreover, addressing the SC and removal issue in optical satellite images has been imperative in RS studies. It is worth noting that optical RS satellites operating in wavelengths like Sentinel-2 are limited in their ability to penetrate the snow and ice [11], rendering them incapable of revealing the underlying features. Therefore, it is essential to explore alternative methods for reconstructing synthetic RS image data, enabling the prediction of scenes beneath the areas of SC in geographical interest. Previous reports have outlined three different methods for reconstructing RS image data [7,8,12]. These methods include the spatial-based approach, which involves restoring corrupted pixel sections on the image through [13]; the spectral-based method, which utilizes different sensors to obtain additional satellite images for reconstructing missing information in the defective image data; and the temporal-based method, where the corrupted or defective pixels in the image are removed and replaced with non-defective pixel obtained from a relatively close image of the same area. These techniques have been widely utilized in RS applications, and it would be interesting to explore their applicability to image data affected by SC. Unfortunately, we cannot make use of them because the outcomes of these methods in various use cases have shown failures in recovering corrupted image pixels, inability to handle large areas, lack of color information, poor image resolution conditions, generic image distortion, etc. [12,14,15]. Therefore, we employed Generative Adversarial Networks (GANs).

GANs are generative models based on deep learning (DL). They have been utilized across various application areas since their development by Ian Goodfellow [16], particularly in unsupervised learning tasks such as image-to-image translation. They consist of two sub-models: the generator and discriminator [4,17]. In simple terms, the generator (G) networks take sample data and generate another sample which we regard as synthetic data, while the discriminator (D) assesses and determines whether the data is generated or originates from the real sample pool. Detecting clouds and snow in remote sensing images is a significant preprocessing task for EO and remote sensing imagery analysis. It is therefore important to note that the experimental results from authors who have reported the application of GANs at different use cases in EO studies have largely focused on cloud noise remover from satellite images. It is worth noting that several works on SC removal from images using GANs of different architectures have been carried out by different independent researchers. For instance, Aiwen et al. [18] reported the detection and negative effect of snowfall features on synthetic images obtained from digital cameras. They deployed the GAN technique integrated with the attention mechanism in the generator component. Zhang et al. [9] also proposed a GAN algorithm for effective snow particles and snow streak removal from images.

In addition, other researchers have reported the removal of snow noise from satellite images using different GAN architectures for pix2pix image translation. For instance, Praveer and Nikos [4] created a CloudGAN model that is capable of extracting thin clouds from Sentinel-2 images. This model does require a high wavelength input, like SAR images, for a region that is heavily obscured, which lowers the model’s effectiveness. Enomoto et al. [19] created a Multispectral Conditional Generative Adversarial Networks (McGANs) model using the conditional Generative Adversarial Networks (cGANs) framework. Their model can remove clouds from visible light RGB satellite images; however, the resulting cloud-free images often contain artifacts. Meanwhile, Toizumi et al. [20] used a GAN with thick cloud masks to eliminate thin clouds from cloudy images while maintaining thick cloud areas. Their model output produces artifact-free non-cloudy images. In addition, Sarukkai et al. [21] proposed a trainable spatial–temporal generator network (STGAN) model, also based on the cGAN architecture. Their model generates realistic cloud-free images while removing the various atmospheric factors that could affect satellite images. Moreover, another major challenge that has gotten little or no attention, contributing to the limited success rate in RS imagery processing and environmental modeling activities, is snow-covered scenes on “Satellite images”. Moreover, there have been some deployments of different algorithms for model development capable of removing SC from images.

To mention a few, the DesnowNet model, which was a CNN-based method proposed to remove snow from images [22]; the Multi-scale Stacked Densely Connected Convolutional Network (MS-SDN) model was another efficient neural network based on the stacked dense network for snow removal [23]; and the Joint size and transparency aware snow removal (JSTASR) model based on modified partial convolution algorithm [24]. In 2021, Kaihao Zhang et al. [25] proposed a Deep Dense Multi-Scale Network (DDMSNet) using semantic and geometric priors for snow removal from images, and recently, Shan Lie et al. [26] proposed an efficient snow removal transformer with a global windowing network (SGNet) for snow removal from images. Interestingly, majority of the studies on SC removal such as the ones listed above made use of digital camera photos or some sort of pictures taken by drones and not RS satellite images, which is the main interest in this study. However, some studies have reported different model development using various neural network algorithms for the mapping and detection of SC features on RS satellite images [5,6,27], but not SC removal. Hence, we implemented the Cycle-Consistent Adversarial Networks (CycleGANs) framework [28], which is a type of GAN architecture for mapping between two different domains of images without the need for paired training of the image data. CycleGANs have been reported to be an efficient and optimized GAN architecture over other GAN architectures [16,29] as far as pix2pix image translation is concerned. Its major advantage over other GAN architectures is that CycleGANs do not require alignment of training image set, as it can perform image translation on unpaired image data. CycleGANs techniques have been demonstrated by Jun-Yan et al. [28] with many examples using different aerial photos, nature photos, and picture–paintings, where they translated an image from a source domain (X) to another target domain (Y) in the absence of paired samples. Their results reveal that the translated images captured some distinctive qualities from image domain X and translated them into another image domain Y. In addition, Praveer and Nikos [4] have demonstrated the RS usage of CycleGANs for the prediction of cloud-free images of the same scene as reflected in the cloudy images. Their results further show that paired (cloud/cloud-free) training dataset procedures are not necessary.

In this study, we developed a snow-cover removal (SCR) model based on the CycleGANs framework. The aim of our SCR model is to automatically capture snow and other notable features in Sentinel-2 satellite images and then generate new snow-free synthetic optical images that share the same properties as the original snow-free satellite images. The generated snow-free synthetic images are evaluated to determine the level of visual and semantic similarity with the original snow-free Sentinel-2 satellite images using various image qualitative metrics (IQMs). This article is structured as follows. In Section 2, the proposed method is discussed, while the materials and implementation are stated in Section 3. Results and the metric quality measurement of the generated images are presented in Section 4. The discussions of the results are presented in Section 5.

2. Methods

Detecting snow cover features on RS optical images and translating them to snow-free synthetic optical images can improve the quality of output images. In this study, the SCR model is built on the CycleGAN framework. The schematic representation of the CycleGAN framework is shown in Figure 1. It involves training two neural networks: the generator network (G) composed of [G_X2Y, F_Y2X] and the discriminator network (D) comprising [D_X, D_Y], working concurrently between two domains. G_X2Y represents the mapping function G: X → Y, where G generates images ($\tilde{x}$) from domain X to domain Y, while F_Y2X represents the reverse mapping function F: Y → X, where F generates images ($\tilde{y}$) from domain Y to domain X. Adopting a mathematical geostatistical interpolation technique such as kriging is a good strategy we could deploy for snow cover remover by reconstructing image pixels [13], but this method has been shown to fail and be incapable of dealing with image spatial information [12] such as RS satellite images. Interestingly, a few of the GAN architectures, such as cGAN, can potentially work as well, but a major edge our model framework has over others is that CycleGAN is committed to solving the problem of lack of paired training data in actual model training, unlike cGAN [19,21] where the baseline is trained with only conditional paired data. Our technique is similar to Praveer and Nikos [4] who used the CloudGAN model to extract thin clouds from Sentinel-2 images, but differ in hyperparameter settings and use case. For instance, our model framework’s use case is snow cover removal, and we trained the model from scratch with a learning rate of 0.0002 for 1000 epochs, whereas Praveer and Nikos [4] used 0.0002 for 200 epochs, although the hyperparameter settings may vary depending on the task. In our setup, snowy satellite images (SSI) belong to domain X, while snow-free satellite images (SFSI) belong to domain Y, with no direct correspondence between the images. During model training, image data flows in two directions of the domains: the forward direction indicated by the blue-colored arrows, and the backward direction indicated by red-colored arrows in Figure 1. Both G and D compete with each other, resulting in improved performance during model training. The objective of D_X is to distinguish between real images X and the translated images ($\tilde{x}$), while D_Y aims to differentiate between the real images Y and the translated images ($\tilde{y}$). The two major components behind the efficient operations of CycleGANs are the introduction of adversarial loss (AL) and cycle consistency loss (CCL) [30].

Adversarial loss (AL) ensures that the translated image closely resembles the real samples in both X and Y domains by extracting significant features and statistical properties of the target domain. The fundamental concept of CCL is that if an image (x) is translated from domain X to Y and then back to domain X, the final output must closely resemble the initial input.

This procedure is known as forward cycle consistency and compares the similarity between the real image (x) and the output $\widehat{x}$(i.e., $x_i\stackrel{G}{\to }\widetilde{y_i}\stackrel{F}{\to }\widehat{x_i}\approx x_i$: forward direction). Similarly, the same procedure is performed on image (y) in the Y domain but in a reverse order, known as backward cycle consistency (i.e., $y_i\stackrel{F}{\to }\widetilde{x_i}\stackrel{G}{\to }\widehat{y_i}\approx y_i$: backward direction). The generator network (G) aims to minimize this objective against the adversary, while the discriminator network (D) seeks to maximize it (i.e., min_G max_DY ℓ_GAN (F, D_Y, Y, X)). For a mapping function G: X → Y and its corresponding discriminator D_Y, the AL mathematical expression in CycleGAN is given as [28,31]:

ℓ_GAN (G, D_Y, X, Y) = 𝔼_{y ∼ pdata(y)} [log D_Y (y)] + 𝔼_{x ∼ pdata(x)} [log (1 − D_Y (G(x))]

(1)

where x and y denote SSI and SFSI, respectively, while x ∼ pdata(x) and y ∼ pdata(y) are the distributions of SS1 and SFSI. The discriminator’s evaluation scores for every real image Y are denoted by E_y. Hence, a similar procedure for mapping function F: Y → X, and corresponding discriminator D_X is carried out as well (i.e., minF maxDX ℓGAN (F, D_X, Y, X)). Following [4,28], we adopted the use of least-square loss to avoid the negative logarithm possibility and optimize the image generation output in terms of image quality during model training. Therefore, for GAN loss ℓ_GAN (G, D_Y, X, Y), we trained G to minimize 𝔼_{x ∼ pdata(x)} [(D(G(x)) − 1)²] and train the D to minimize 𝔼_{y ∼ pdata(y)} [(D(y)) − 1)²] + 𝔼_{x ∼ pdata(x)} [D(G(x))²]. To prevent the discriminator from failing to provide insightful input that could enhance the generator output, the CCL ensures that crucial information details (i.e., essential visual data) from the original image are not lost during image information translation between domains [31]. The mathematical expression of CCL is given below:

ℓ_cyc (G, F) = 𝔼_{x ∼ pdata(x)} [‖F(G(x)] − x‖₁] + 𝔼_{y ∼ pdata(y)} [‖G(F(x)) − y‖₁]

(2)

We compared the generated SSI with the real SSI and then calculated the sum of the absolute pixel value of the differences between the original image and the image after being converted by both generators. By integrating these loss components (i.e., CCL and AL), we can construct the overall objective for the model by combining Equations (1) and (2):

ℓ(G, F, D_X, D_Y) = ℓ_GAN (G, D_Y, X, Y) + ℓ_GAN (F, D_X, Y, X) + λℓ_cyc (G, F)

(3)

where λ is the regularizing factor that controls the relative importance of the two objective functions. Further details about AL and CCL are discussed in [28,31,32].

Model Output Performance Metrics

Generated synthetic images are prone to loss of visual quality during image acquisition, processing, and reproduction, among others [33]. Hence, further qualitative procedures are required to measure the quality of generated synthetic images. Zhou Wang et al. [34] reported the Structural Similarity Index Measure (SSIM) as an effective tool for assessing image quality, among other metrics. On the contrary, Jim Nilsson et al. [35] demonstrated in their review study that SSIM can produce incorrect or invalid results when used. Therefore, we added two more techniques of image quality measurements since we cannot ascertain the absolute correctness of the reported measuring metric.

(a) 

SSIM examines similarities between luminance, contrast, and structure [33]. Given two N-dimensional images; x = (x1, …, x_N), and y = (y1, …, y_N), where x and y are original and generated synthetic images, respectively. The summarized and simplified version of SSIM is given as [33,34]:

$$S(x,y)=S_1(x,y)S_2(x,y)=\left[\frac{2\overline{x}\overline{y}+{ϵ}_1}{{\overline{x}}^2+{\overline{y}}^2+{ϵ}_1}\right]\left[\frac{2S_{xy}+{ϵ}_2}{S_x^2+S_y^2+{ϵ}_2}\right]$$

Ɛ_i is the numerical stability coefficient for preventing zero denominators, $\overline{x} \mathrm{a}\mathrm{n}\mathrm{d} \overline{y}$ are the local means, S_x and S_y are the standard deviations, and S_xy is the cross-covariance for images x and y sequentially. The larger the computed value, the higher the similarity between the real image and the synthetic generated image.

(b) 

Universal image quality index denoted as ‘Q’ is often used to determine the average visual correlation between the original and the generated synthetic image [36]. Q is a product of 3 components (Loss of correlation, Luminance distortion, and Contrast distortion). When the generated image is closer to the original image, then the Q value is closer to one. The value of 1 is achieved if and only if y_i = x_i for all i = 1, 2, …, N.

Let x = {xi: I = 1, 2, 3, ……., N} and y = {yi: I = 1, 2, 3, ……., N} be the original and generated synthetic images, respectively. Q can be mathematically represented as:

$$Q=\frac{{\sigma }_{xy}}{{\sigma }_x{\sigma }_y}*\frac{2\overline{x}\overline{y}}{(\overline{x}{)}^2+(\overline{y}{)}^2}*\frac{2{\sigma }_x{\sigma }_y}{{\sigma }_x^2+{\sigma }_y^2}$$

$$\overline{x}=\frac{1}{N}\stackrel{N}{\underset{i=1}{\Sigma }}{x}_{i },\overline{y}=\frac{1}{N}\stackrel{N}{\underset{i=1}{\Sigma }}{y}_i , {\sigma }_x^2=\frac{1}{N-1}\stackrel{N}{\underset{i=1}{\Sigma }}(x_i-\overline{x}{)}^2$$

$${\sigma }_y^2=\frac{1}{N-1}\stackrel{N}{\underset{i=1}{\Sigma }}(y_i-\overline{y}{)}^2, { \sigma }_{xy}=\frac{1}{N-1}\stackrel{N}{\underset{i=1}{\Sigma }}(x_i-\overline{x}\left)\right(y_i-\overline{y})$$

where $\overline{x} \mathrm{a}\mathrm{n}\mathrm{d}{ \sigma }_x$ are the means of original and generated synthetic images, respectively, ${\sigma }_x \mathrm{a}\mathrm{n}\mathrm{d} {\sigma }_y$ denote the same image name but for variance, while ${\sigma }_{xy}$ is the cross variance between x and y.

(c) 

Peak signal-to-noise ratio (PSNR) is a commonly used mean-squared-error-based metric for measuring the quality of a generated image by quantifying the pixel-to-pixel difference between two (original and generated) images. The higher the PSNR, the better the quality of the reconstructed image, and it is expressed in terms of the logarithmic decibel scale [34,37].

$$PSNR=10.lo{g}_{10}\left(\frac{MA{X}_L^2}{MSE}\right), MSE=\frac{1}{mn}\stackrel{m-1}{\underset{i=0}{\Sigma }}\stackrel{n-1}{\underset{j=0}{\Sigma }}\left(f\right(i,j)-g(i,j){)}^2$$

where MSE is the Mean Squared Error between two images X (i, j) and Y (i, j)), m denotes the number of rows of pixels of the images and i represents the index of that row, while n denotes the number of columns of pixels of the image and j represents the index of that column. f represents the matrix data of the original image, while g represents the matrix data of the generated synthetic image in this study.

3. Materials and Method

3.1. Dataset

Our dataset collection consists of high-resolution level 2 Sentinel-2 cloud-free optical imagery with false color composition spanning the years 2016–2022. For our investigation, we specifically selected satellite imagery with a spatial resolution of 10 m. Datasets were downloaded from “https://apps.sentinel-hub.com/eo-browser/ (accessed on 24 January 2024)”. Most of our cloud-free satellite images exhibit cloud coverage within the range of 0–2%, enabling clear visibility of images with and without snow. All the downloaded images were acquired over the Cariboo region along Williams Lake in British Columbia, Canada, as it is a geographical area of interest for our study. We opted for 95 samples of SSI and 67 samples of SFSI for training the GANs. Interestingly, training SSI to SFSI requires a large number of images; therefore, we performed data augmentation [38] on both SSI and SFSI to generate a sufficient quantity of images suitable for GAN training. Following data augmentation and data cleaning, we obtained 1000 images each for SSI and SFSI.

3.2. Network Architectures and Training

In constructing our SCR model, we adopted the architecture and naming pattern for the CycleGAN network from Praveer and Nikos [4] and Jun-Yan et al. [28], respectively. Figure 2 shows the overview of the CycleGAN architecture process workflow for the SCR model. Computational cost is one of the primary aspects that was taken into account during the model training process when building our SCR model.

The high-resolution image from the Sentinel-2 satellite, which had 512 × 512 pixels, was resized to 256 × 256 pixels. This process was followed based on our needs and to prevent significantly higher computational costs in terms of computation time and significant memory usage by the computer system. The internal component configuration of the computer system we used to develop the model includes an Intel(R) Xeon(R) W-2245 processor running at 3.90 GHz, a workstation running Windows 10 Pro, a processor with an optimized high-performance NVIDIA Quadro card, which is more accurate and dependable than a typical GPU, and 16.0 GB of RAM. In the generator (G) architecture, we employed 6 residual blocks for training images of size 128 × 128, and 9 residual blocks for images of size 256 × 256. As for the discriminator (D) architecture, we used the 70 × 70 PatchGAN approach introduced by Phillip Isola et al. [29]. PatchGAN divides the input image into arrays of size 70 × 70 patches, which are then used to classify the images as real or generated based on an average aggregation technique. To enhance the accuracy of the model output, we added more data, as described in Section 3.1. Furthermore, we changed the learning rate from 0.002 to 0.0002 and used 1000 epochs.

4. Results

SCR Model Prediction Results

Based on the SCR model, we generated a variety of synthetic satellite images (i.e., SSI and SFSI). Figure 3 shows the flowchart of SCR model deployment and possible outputs. Our SCR model successfully removes the dispersed snow from the satellite image and replaces it with the underlying ground textures while preserving other notable features.

Figure 4a,b shows the two-column layout, with the first stand-alone row representing the original SFSI. The second, third, and fourth rows contain the original SSIs and their corresponding synthetic SFSIs, respectively. The date tag appears on the left edge of each image. Since obtaining the original SFSI and original SSI for the same selected day or season of interest is not possible, we chose to use a snow-free satellite image of another day (11 August 2021) as a benchmark or reference image. The chosen benchmark satellite image was relatively clear, cloud-free, fuzzy-free, and cloud-shadow-free.

As observed in Figure 4a,b, the SCR model output can retain significant feature details such as fields and rivers (the black-colored, thick, curve-like line across the image). In most cases, its output is aesthetically pleasing and seems natural, almost identical to the original SFSI. The model was notably able to remove highly thick snow cover (e.g., SAT-Img1 and SAT-Img6). Furthermore, it is important to note that, in comparison to other generated images, the generated image with the extremely thick snow coverage scenario in Figure 4b does not produce an output of good quality (SAT-Img6).

Three bar charts are displayed in Figure 5, with three bars representing each image in each of the qualitative metrics (IQMs). We consider the original SFSI to be the benchmark image or reference image. The bar chart shows various metric measures between original SFSI vs. generated SFI and original SFSI versus original SSI. The color code description is provided in the legend of the satellite image status bar chart.

For convenience, we expressed the IQM values in percentages. It is assumed that the generated SFI metric values are in the range of 75–95.1%, 88.1–95.7%, and 18.6–29.1% for SSIM, Q, and PSNR, respectively, while the original SSI metric values are in the range of 75.1–83.6%, 73.1–83.4%, and 13.4–18.4%, respectively. The quantitative findings of the assessment of the metric scores of image-to-image similarities between the generated and original images are presented in

Table 1. Results of metric evaluations for original SFSI versus original SSI, and original SFSI versus generated SFI.

	SSIM		Q		PSNR
Sentinel-2 Satellite Images	Ori. SFSI versus Ori. SSI (%)	Ori. SFSI versus Gen. SFI (%)	Ori. SFSI versus Ori. SSI (%)	Ori. SFSI versus Gen. SFI (%)	Ori. SFSI versus Ori. SSI (%)	Ori. SFSI versus Gen. SFI (%)
SAT-Img1	75.10	84.10	76.10	92.00	16.40	23.20
SAT-Img2	80.10	85.20	81.20	91.10	17.00	25.06
SAT-Img3	81.40	86.20	82.20	91.10	20.80	23.40
SAT-Img4	81.30	85.00	83.40	88.70	13.40	19.30
SAT-Img5	83.60	95.10	84.10	94.70	18.40	28.50
SAT-Img6	75.10	77.30	73.10	88.10	12.10	18.60

.

Table 1 demonstrates how the generated SFI metric values are consistently greater than the original SSI values for every index. This suggests that our approach performs admirably across the board for all simulated situations.

5. Discussion

We present an analysis of the results achieved using our SCR model, which efficiently detects and eliminates snow cover scenes from satellite images, replacing them with the underlying ground details, as seen in Figure 4a and Figure 4b, respectively.

As observed, in Figure 4a,b, the SCR model can retain significant feature details, such as fields and rivers (the thick curve-like line across the image). In most cases, the model output is aesthetically pleasing and seems natural, almost identical to the original SFSI. Our model’s output findings were comparable to those published by Praveer and Nikos [4], who employed Cloud-GAN with the CycleGAN framework to generate cloud-free synthetic images. However, they reported a few failed scenarios in which the model output images were blank or overly smoothed. These unsuccessful cases could be the consequence of insufficient data augmentation or a hyperparameter tweaking issue. Our model was notably able to remove highly thick snow cover (e.g., SAT-Img1 and SAT-Img6). Moreover, it is noteworthy to state that the generated image with a highly thick snow coverage scene in Figure 4b does not give a high-quality output compared to other generated images, as in the case of SAT-Img6. This situation could be a result of the limited availability of satellite image samples with high snow coverage scenes during SCR model training, which consequently affected the model’s performance. In addition, we observed that the images in Figure 4a and Figure 4b are slightly fuzzy, respectively. Unfortunately, we could not get it better than this. Since the satellite passage over the region of interest was during the winter season, therefore we suspected that the image resolution might have been affected by the atmospheric particulate matter suspended in the air which could potentially make the satellite image fuzzy.

In Figure 5, we estimated that the metric value for original SFSI versus original SFSI is one, hence the green bar is always 100%. Each metric (SSIM, Q, and PSNR) displays the metric values for the original SFSI, generated SFI, and original SFSI of different dates. The bar chart displays the results of quantitative and comparative evaluations of the model output (i.e., generated images) by estimating how similar the output is to the original SFSI. As observed, all IQM values of the generated SFI performed similarly in that their metric values were higher than those of the original SSI. Additionally, the relatively high SSIM values of the model output signify a higher resemblance to the original SFSI in terms of luminance, contrast, and structure compared with the model input image (i.e., original SSI). In the example of SAT-Img6 in Figure 4b, the generated image has a different contrast compared with the corresponding target image, which can be misleading about the created image’s quality. This effect could be the reason why the generated SFI has a relatively low SSIM value (~2% higher than SAT-Img6) compared to other SSIM values. Interestingly, other generated SFIs exhibited higher percentage values than SAT-Img6 in other metrics.

In addition, the higher PSNR percentage values are indications of similarity in the pixel-to-pixel difference between the original SFSI and the generated image, as we can observe the significant increase in signal information after the snow cover scenes have been removed—by as much as ~7%, ~8%, ~3%, ~6%, ~10%, and ~7%, for the satellite images SAT-Img1, SAT-Img2, SAT-Img3, SAT-Img4, SAT-Img5, and SAT-Img6, respectively. In this case, the target image (i.e., original SFSI) was taken as a reference, and the PSNR values of the original SSI and the generated SFI were compared to record the information gained. High values of PSNR indicate a high similarity between the generated SFI and the original SFSI. The complexity and diversity of satellite images under various settings make it difficult to compare our model metric value results to previously reported model metric results [4,7,8].

Q in our case has the highest range of metric values (91–~95%) compared to other indices (SSIM and PSNR). This is comparable to [7], where Q was likewise shown to have the highest metric value. Q is a universal image quality index, and it reveals the degree of visual and semantic similarity between the original SSFI and the generated SFI. Furthermore, Figure 5 shows high metric values of over 77% for all generated images and also reveals the variability in the IQM value index depending on the amount of distortion and contrast in the generated image. Consequently, it is difficult to acquire the same quality rating even when the same distortion and contrast are present in different generated images. Despite this, we found that Q produces a result closer to 95% than SSIM and PRSN.

6. Conclusions

In this article, we proposed a snow cover remover (SCR) model that employs a CycleGAN framework. GAN deployment has been successful in many image analysis and visual tasks, as stated in Section 1, but the use of GANs for RS satellite snow cover removal is still relatively small. Hence, we used the CycleGAN framework and Sentinel satellite images for model training and the development of a novel SCR model. The SCR model recognizes and eliminates snow-cover scenes from satellite images, replacing them with underlying ground information without using any corresponding snow-free pair for a snowy image. Data augmentation procedures were carried out to salvage the effect of the limited availability of the Sentinel-2 image dataset exhibiting snow cover features. In order to achieve more computing efficiency, we resized the 512 × 512 pixels to 256 × 256 pixels, compromising the greater pixel sizes.

The model was tested on real snowy satellite images that effectively removed snow cover scenes and then reconstructed underlying ground textures while preserving important features. However, we believe we can get a more refined model output if there is a larger number of original SSIs for model training. The model performance was evaluated using selected Image Quality Metrics (SSIM, Q, PSNR). The bar charts of the IQM display the model’s efficiency. This consequently exhibits the quality of the generated snow-free images expressed in percentage, thus validating the effectiveness of the model’s output.

Finally, we demonstrated that the generated snow-free images are useful for RS research based on the exhibited IQM values. Our metric data shows that Q delivers more metric values, nearly 95%, than SSIM and PRSN. We think that our approach has demonstrated the advancement of the DL technique and its applicability in RS studies. Hence, this approach has contributed to resolving some of the problems associated with processing RS imagery and can increase the amount of satellite image data available for use in future studies and applications, particularly for data training in DL model development. Another potential application for this method is the area of cloud shadow removal. Future research directions may involve the development of an optimized snow cover removal model using a multi-source SSI dataset capable of providing high-resolution images for environmental mapping and time series modeling.