Using Haze Level Estimation in Data Cleaning for Supervised Deep Image Dehazing Models

Abstract

Recently, supervised deep learning methods have been widely used for image haze removal. These methods rely on training data that are assumed to be appropriate. However, this assumption may not always be true. We observe that some data may contain hazy ground truth (GT) images. This can lead to supervised deep image dehazing (SDID) models learning inappropriate mapping between hazy images and GT images, which negatively affects the dehazing performance. To address this problem, two difficulties must be solved. One is to estimate the haze level in an image, and the other is to develop a haze level indicator to discriminate clear and hazy images. To this end, we proposed a haze level estimation (HLE) scheme based on dark channel prior and a haze level indicator accordingly for training data cleaning, i.e., to exclude image pairs with hazy GT images in the data set. With the data cleaning by the HLE, we introduced an SDID framework to avoid inappropriate learning and thus improve the dehazing performance. To verify the framework, using the RESIDE data set, experiments were conducted with three types of SDID models, i.e., GCAN, REFN and cGAN. The results show that our method can significantly improve the dehazing performance of the three SDID models. Subjectively, the proposed method generally provides better visual quality. Objectively, our method, using fewer training image pairs, was capable of improving PSNR in the GCAN, REFN, and cGAN models by 3.10 dB, 5.74 dB, and 6.44 dB, respectively. Furthermore, our method was evaluated using a real-world data set, KeDeMa. The results indicate that the better visual quality of the dehazed images is generally for models with the proposed data cleaning scheme. The results demonstrate that the proposed method effectively and efficiently enhances the dehazing performance in the given examples. The practical significance of this research is to provide an easy but effective way, that is, the proposed data cleaning scheme, to improve the performance of SDID models.

1. Introduction

In the field of image restoration, image haze removal has been a topic of increasing interest. The haze is mainly due to light scattering from particles in the air. Hazy images generally have a reduced contrast and visibility, which degrades the performance of image-based applications, such as automatic driving and surveillance. Consequently, haze removal methods are sought to improve image quality. In general, there are two types of dehazing methods. The first type uses physical models where model parameters are estimated based on assumptions or statistical priors, e.g., [1,2,3,4,5]. In this type, a popular method is based on the dark channel prior reported in ref. [1] where the hazy image captured by a camera is modeled as

$$I\left(x\right)=J\left(x\right)t\left(x\right)+A\left[1-t\left(x\right)\right]$$

(1)

where $I\left(x\right)$ is the captured image; $J\left(x\right)$ is the clear radiance; $A$ is the global atmospheric light or simply atmospheric light; and $t\left(x\right)=e^{-\beta d\left(x\right)}$ is the transmittance that represents the portion of the non-scattered light to the camera; $\beta$ is the scattering coefficient of the atmosphere, and $d\left(x\right)$ is the scene depth at position $x$. The main challenge for model-based dehazing approaches is to estimate atmospheric light $A$ and transmittance $t\left(x\right)$ from a single hazy image.

The second type of image dehazing employs deep learning models. Deep learning has demonstrated its advantages in image processing. Recently, it has been introduced in the field of image haze removal. The existing deep learning methods in image dehazing can be divided into two categories: model-based methods and end-to-end methods. Model-based methods use deep learning to estimate the model parameters $A$ or $t\left(x\right)$ or both, based on Equation (1), such as [6,7,8,9,10,11,12]. End-to-end methods, which are currently more popular in the image dehazing community, directly learn the mapping between hazy and ground truth (GT) images from a training data set. This study concentrates on end-to-end methods, that is, supervised deep image dehazing (SDID) models.

To date, many deep learning methods have been proposed for image dehazing. For example, based on the encoder–decoder framework, the densely connected pyramid dehazing network [13] and the enhanced context aggregation network [14] were proposed. Based on the retinex decomposition, the deep retinex dehazing network [15] was presented. Using residual learning or generative adversarial networks, the gated context aggregation network (GCAN) [16] and the conditional generative adversarial network [17,18] were developed. By introducing novel modules or domains, many deep models have been reported. Some examples are the self-supporting dehazing network [19] with a self-filtering block and a self-support module; the wavelet hybrid network [20] with a convolution neural network in the wavelet domain; and the dynamic multi-attention dehazing network [21] with dynamic feature attention and adaptive feature fusion modules. In [22], a deep model was presented in which guide information and progressive feature fusion were used. Basically, the main differences in SDID models are in network structures, learning rules, and cost functions. For a complete survey, see [23,24].

The data set is an important aspect in SDID models. At present, SDID models use the data set as is, assuming that all GT images are clear. But this assumption is not always true. Note that hazy GT images may degrade dehazing performance since an SDID model learns the mapping between the hazy images and their corresponding GT images. Unfortunately, little attention has been paid to the suitability of the data set used in SDID models so far. To distinguish clear GT images requires a scheme to estimate the haze level in an image. To date, there exists no such haze level estimation scheme. Thus, haze level information is rarely used in SDID models. Several researchers had attempted to use haze level information in SDID models, such as the level aware progressive network [25], the knowledge distillation dehazing network [26], and the adaptive network for haze concentration [27]. However, these methods have at least two problems. First, they implicitly use haze level information in the networks, and consequently, it is hard to apply them to other models. Second, they do nothing about the image pairs with hazy GT images that negatively affect dehazing performance. To this end, in this study we will devise a haze level estimation scheme that can explicitly use haze level information to filter out image pairs with hazy GT images in the data set. This will be called data cleaning in this study. After the data cleaning, the cleaned data are then used to train SDID models. Therefore, we expect that the dehazing performance of SDID models can be improved.

The challenge of this study is to devise a scheme that can adequately estimate the haze level in an image. To this end, we develop a haze level estimation scheme and a haze level indicator based on the dark channel in ref. [1]. In ref. [1], the dark channel was related to the haze level in an image and was used to estimate atmospheric light and transmittance in Equation (1). Although it relates the haze level to the dark channel, no further research has been performed on a haze level indicator. Therefore, it is not trivial to develop a haze level estimation scheme. To do so, there are at least two problems to solve. First, as described in ref. [1], sky regions and white objects are not met with the assumption of dark channel prior. A scheme should be developed to eliminate this case. Second, an appropriate haze level indicator should be introduced to distinguish clear and hazy images. The problems will be solved in this study. When compared to the way of using the dark channel in ref. [1], our proposed haze level estimation (HLE) has at least two fundamental differences. First, the dark channel was used to estimate the model parameters in ref. [1]. But we used it to develop a haze level indicator and applied it to training data cleaning. Second, the dark channel was used in a model-based dehazing. However, we used it in an end-to-end SDID framework in this study.

Based on the proposed HLE, a data cleaning scheme is introduced, which will be used to exclude image pairs with hazy GT images in a data set to improve the dehazing performance of SDID models. To verify our method, using the RESIDE data set [28], experiments were conducted with three different types of SDID models, that is, the GCAN [16], the RefineDNet (REFN) [29] and the cGAN [30]. The results show that the proposed data cleaning improved PSNR by 3.10 dB, 5.74 dB, and 6.44 dB for the GCAN, REFN, and cGAN models, respectively. Furthermore, the proposed method was further evaluated using a real-world data set, KeDeMa. The results indicate that objectively and subjectively, the models with our data cleaning generally outperform those without it. In summary, the dehazing performance of the GCAN, REFN, and cGAN models can be improved by the proposed data cleaning scheme. The results imply that it may also benefit other SDID models. The three main contributions of this study are listed below.

• We introduced a haze level estimation (HLE) scheme in which a haze level indicator was devised based on the dark channel in ref. [1]. In ref. [1], it was observed that the haze level in an image is related to its dark channel. However, there is no further work on haze level estimation. To date, no such research has been conducted on using haze level information in a data set for SDID models. Thus, our HLE is a pioneering work in the SDID field.
• We presented a data cleaning scheme based on the proposed HLE for SDID models. The haze level indicator is used to distinguish clear and hazy GT images in a data set. When a GT image is hazy, its corresponding hazy images are excluded together from the data set. The process to discard image pairs with hazy GT images is called data cleaning in this study. The experiment showed that the proposed data cleaning scheme can significantly improve the dehazing performance of SDID models. So far, no such work has been reported on SDID methods. Therefore, this paper is a contribution in this field.
• We proposed an SDID framework that uses a data cleaning scheme to exclude image pairs with hazy GT images. This prevents an SDID model from learning an inappropriate mapping that degrades the dehazing performance. The proposed framework requires fewer training image pairs, yet achieves better dehazing performance. The framework can be easily applied to any SDID model. This is another contribution to the research community.

This paper is organized as follows. Section 2 introduces the proposed HLE and then explains how it applies to data cleaning. Section 3 presents a deep dehazing framework that uses our data cleaning method as a preprocessing scheme before training an SDID model. Section 4 provides three SDID models to justify the proposed SDID framework. Section 5 concludes this study and describes further research.

2. Haze Level Estimation and Data Cleaning

In this section, a haze level estimation (HLE) scheme is introduced in Section 2.1. Then, the application of the proposed HLE to data cleaning is described in Section 2.2.

2.1. Haze Level Estimation

This section proposes an HLE scheme in which a haze level indicator is introduced based on the dark channel in ref. [1]. In ref. [1], the haze level in an image was related to its dark channel. However, no further work has been conducted on the development of hazy level estimation with a haze level indicator. Thus, although the proposed HLE is based on the dark channel, there are at least two main differences. First, the dark channel was applied to estimate the model parameters in ref. [1], that is, atmospheric light and transmittance in Equation (1), while the dark channel was developed as a haze level indicator in this study. Second, the dark channel was used in a model-based dehazing method, while we used it for data cleaning and developed an end-to-end dehazing framework in this study. Consequently, our use of the proposed HLE is essentially different from the way the dark channel was used in ref. [1].

To date, haze level information has rarely been used in SDID models. Although some researchers, e.g., [24,25,26], implicitly incorporated it into SDID models, they did not explicitly apply it to the data set itself. The reason could be that there is no way to extract an accurate estimate of the haze level from an image. Therefore, data cleaning that uses haze-level information has rarely been the subject of SDID research. As a result, current SDID researchers use the data set as is. However, not every image pair is good for an SDID model, that is, an image pair with a hazy GT image. When it happens, it will negatively affect the dehazing performance of SDID models since an inappropriate mapping is learned from the image pairs. To eliminate the problem, in this study, an HLE scheme is developed and applied to data cleaning that excludes image pairs with hazy GT images for SDID models. The details are described in the following.

Based on the dark channel prior [1], the HLE is developed. The dark channel prior states that in a block of a haze-free image, excluding sky regions and white objects, there are some pixels with very low or zero values in at least one of the RGB components. The dark channel can be obtained by a block minimum filter. For a haze-free image, the dark channel is dark, while for a hazy image, it is not. The brightness of the dark channel is positively proportional to the haze level of the image. Figure 1a shows a clear image “Cage” as an example to show the property of the dark channel prior. Figure 1b is the dark channel of Figure 1a obtained by the 15 × 15 minimum filter. As described, the dark channel in Figure 1b is dark except for the sky region. Figure 1c is a hazy image generated from Figure 1a, and its dark channel is shown in Figure 1d. A brighter dark channel is obtained compared to that in Figure 1b. The results in Figure 1 demonstrate the properties of the dark channel prior.

Although Figure 1 shows the haze level is related to its dark channel, it provides no further information about the haze level. Thus, it is required to have an indicator for haze level estimation. Fortunately, Figure 1 gives us a hint. If sky regions and white objects can be ignored, then the average of other pixel values in the dark channel can be a good indicator of haze level. In other words, the average of the selected pixels in the dark channel can be used as an indicator of the haze level, as long as sky regions and white objects are excluded from the calculation. To achieve this, in this study, a threshold $\mathsf{\tau }$ is introduced to exclude the values of pixels that are greater than τ in the dark channel. From the viewpoint of the histogram, this is a truncated mean in which pixel values greater than $\mathsf{\tau }$ are not included in the calculation. The truncated mean is used as a haze level indicator and denoted as ${\tilde{\mu }}_{dc}\left(\tau \right)$. When $\mathsf{\tau }$ is properly set, ${\tilde{\mu }}_{dc}\left(\tau \right)$ works well in most of cases. However, we observe that a clear image may be considered hazy in some cases. Thus, a further check is required to avoid this problem. Fortunately, we find that for a clear image, ${\tilde{\mu }}_{dc}\left(\tau \right)$ has a small change, as $\tau$ varies while a hazy image does not. Thus, we use the difference $\Delta {\tilde{\mu }}_{dc}={\tilde{\mu }}_{dc}\left({\tau }_1\right)-{\tilde{\mu }}_{dc}\left(\tau \right)$ for the second check, where ${\tau }_1>\tau$. With the above ideas, for a given image $\mathit{I}$, the proposed HLE scheme is implemented as follows.

Step 1.

Obtain the dark channel using the $15\times 15$ minimum filter as below.

$$I_{15}^{dark}\left(x\right)=\underset{y\in \mathsf{\Omega }\left(x\right)}{\min }\underset{c}{\min }\left[I_c\left(y\right)\right]$$

(2)

where $c\in \left\{R, G,B\right\}$ and $\mathsf{\Omega }\left(x\right)$ is a $15\times 15$ window centered at $x$.

Step 2.

Calculate the haze level indicator ${\tilde{\mu }}_{dc}\left(\tau \right)$, a truncated mean of $I_{15}^{dark}\left(x\right)$, as below.

$${\tilde{\mu }}_{dc}\left(\tau \right)=\mathrm{mean} \left[I_{15}^{dark}\left(x\right)\le \mathsf{\tau }\right]$$

(3)

where $0<\mathsf{\tau }<1$ is a user-defined threshold to exclude the pixel values from the calculation.

Step 3.

Check if the inequality ${\tilde{\mu }}_{dc}\left(\tau \right)<\eta$ holds, where $0<\eta <1$ is a user-defined threshold. If ${\tilde{\mu }}_{dc}\left(\tau \right)<\eta$ is true, then image $\mathit{I}$ is considered as clear. Otherwise, go to Step 4 for the second check.

Step 4.

Calculate the difference of ${\tilde{\mu }}_{dc}\left(\tau \right)$, $\Delta {\tilde{\mu }}_{dc}={\tilde{\mu }}_{dc}\left({\tau }_1\right)-{\tilde{\mu }}_{dc}\left(\tau \right)$, and check if the inequality $\Delta {\tilde{\mu }}_{dc}<ϵ$ holds, where ${\tau }_1>\tau$ and $0<ϵ<1$. Notations ${\tau }_1$ and $ϵ$ are user-defined thresholds. If $\Delta {\tilde{\mu }}_{dc}<ϵ$ is true, then image $\mathit{I}$ is considered as clear. Otherwise, it is hazy.

Figure 2 summarizes the steps for the proposed HLE. Figure 2 shows that the HLE consists of two checks. The first check is performed in Step 3. If image $\mathit{I}$ is considered hazy, a second check is performed in Step 4 to alleviate the false negative problem.

2.2. Application of the HLE to Data Cleaning

This section applies the HLE scheme developed in Section 2.1 to data cleaning to exclude training image pairs with hazy GT images. In the training process, an SDID model learns the mapping from training image pairs. However, an inappropriate mapping may be learned when image pairs with hazy GT images are involved in the training process. It affects the dehazing performance of the SDID model. Conversely, it also implies that better dehazing performance can be expected when image pairs with hazy GT images are excluded from the training process. Here, we explain how the proposed HLE scheme is applied to data cleaning for SDID models. In the following, the RESIDE data set in [28] is used as an example for data cleaning.

The RESIDE data set is widely used in the image dehazing field. It includes 8970 GT images, through which 313,950 hazy images were generated. That is, one GT image created 35 hazy images. To date, the RESIDE data set has not been investigated for its image pairs. By inspection, we can find hazy GT images in the RESIDE data set. To see a whole picture, we used the proposed haze level indicator ${\tilde{\mu }}_{dc}\left(0.9\right)$ to examine the haze level distribution of the GT images. The distribution is shown in Figure 3 and indicates that many GT images are of a large value ${\tilde{\mu }}_{dc}\left(0.9\right)$. A larger ${\tilde{\mu }}_{dc}\left(0.9\right)$ means more haze in a GT image. By observations, we find that GT images of ${\tilde{\mu }}_{dc}\left(0.9\right)>0.4$ are hazy. Five sample images of ${\tilde{\mu }}_{dc}\left(0.9\right)>0.4$ are given in Figure 4. The results show that many GT images exist in the RESIDE data set and that ${\tilde{\mu }}_{dc}\left(\tau \right)$ is a good measure for haze level in an image.

Although GT images of ${\tilde{\mu }}_{dc}\left(0.9\right)>0.4$ are hazy, that does not imply that GT images of ${\tilde{\mu }}_{dc}\left(0.9\right)\le 0.4$ are clear. Therefore, further research was conducted. By experiments, we find that GT images of ${\tilde{\mu }}_{dc}\left(0.9\right)\le 0.4$ may be clear or hazy. Three examples selected from the RESIDE data set, whose ${\tilde{\mu }}_{dc}\left(0.9\right)\le 0.4$, are given in

Table 1. Three RESIDE GT images and the related ${\tilde{\mu }}_{dc}\left(0.4\right)$, ${\tilde{\mu }}_{dc}\left(0.9\right)$, and $\Delta {\tilde{\mu }}_{dc}$.

	${\mathit{I}}_1$	${\mathit{I}}_2$	${\mathit{I}}_3$
Haze level indicator
${\tilde{\mu }}_{dc}\left(0.4\right)$	0.0252	0.2461	0.2312
${\tilde{\mu }}_{dc}\left(0.9\right)$	0.2528	0.2742	0.4661
$\Delta {\tilde{\mu }}_{dc}$	0.2276	0.0281	0.2349
Discrimination result	clear	clear	hazy

, where ${\mathit{I}}_1$ and ${\mathit{I}}_2$ are clear, and ${\mathit{I}}_3$ is hazy. To distinguish a clear GT image, a further check is required where ${\tilde{\mu }}_{dc}\left(0.4\right)$ is used as the haze level indicator based on the above observation. Through experiments, it is observed that a GT image is clear if ${\tilde{\mu }}_{dc}\left(0.4\right)<0.05$ in the first check. In addition, a GT image is clear when $\Delta {\tilde{\mu }}_{dc}={\tilde{\mu }}_{dc}\left(0.9\right)-{\tilde{\mu }}_{dc}\left(0.4\right)<0.1$ in the second check. In summary, we empirically set $\tau =0.4$, $\eta =0.05$, ${\tau }_1=0.9$, and $ϵ=0.1$ in the HLE described in Section 2.1.

To verify the feasibility of the data cleaning by the HLE, the three GT images in Table 1 are given as examples. Table 1 shows the three GT images and the corresponding ${\tilde{\mu }}_{dc}\left(\tau \right)$, $\Delta {\tilde{\mu }}_{dc}$, and the discrimination result. In Table 1, the subscript in ${\mathit{I}}_{\mathit{I}}$ is the order of examples, that is, 1 as the first example, and so on. In Table 1, ${\mathit{I}}_1$ is considered as clear because ${\tilde{\mu }}_{dc}\left(0.4\right)<0.05$. For image ${\mathit{I}}_2$, ${\tilde{\mu }}_{dc}\left(0.4\right)>0.05$, so $\Delta {\tilde{\mu }}_{dc}<0.1$ was further checked. It is clear because $\Delta {\tilde{\mu }}_{dc}<0.1$ holds. For image ${\mathit{I}}_3$, ${\tilde{\mu }}_{dc}\left(0.4\right)>0.05$ and $\Delta {\tilde{\mu }}_{dc}\ge 0.1$ hold, thus it is considered hazy. The HLE discrimination result is consistent with the visual result. It suggests that the proposed HLE is feasible. In this study, we will discard image pairs with hazy GT images. This process is called data cleaning in this paper.

3. The Proposed SDID Framework

In this section, we present an SDID framework, shown in Figure 5, that includes the proposed data cleaning scheme. Denote an original data set as ${\mathit{S}}_o$, which includes a hazy image set ${\mathit{S}}_h$ and a GT image set ${\mathit{S}}_g$. The set ${\mathit{S}}_o$ consists of a collection of image pairs. Each image pair consists of a hazy image from ${\mathit{S}}_h$ and its corresponding GT image from ${\mathit{S}}_g$. Before training an SDID model, ${\mathit{S}}_g$ is processed with the proposed data cleaning scheme. Then, the image pairs with the hazy GT images are discarded. The resulting set is denoted as ${\mathit{S}}_o^c$. That is, the proposed data cleaning preprocesses ${\mathit{S}}_o$ to ${\mathit{S}}_o^c$. Then, ${\mathit{S}}_o^c$ is divided into a training data set ${\mathit{S}}_{tr}^c$ and a testing data set ${\mathit{S}}_{tt}^c$. Then, the SDID is trained with ${\mathit{S}}_{tr}^c$ as in Figure 5a. By this, better dehazing performance is expected since a better mapping is learned from ${\mathit{S}}_{tr}^c$ than that from the original set ${\mathit{S}}_o$. The trained SDID model is tested by ${\mathit{S}}_{tt}^c$ as shown in Figure 5b, where ${\mathit{I}}_h$ is a hazy image in ${\mathit{S}}_{tt}^c$ and ${\widehat{\mathit{I}}}_g$ is the predicted GT image, and the performance is evaluated. Note that the performance evaluation may be biased by hazy GT images when a full reference metric is used. For example, a less dehazed ${\widehat{\mathit{I}}}_g$ is considered as a better result when a hazy GT image is used as a reference. Thus, in later experiments, we use the cleaned data set ${\mathit{S}}_{tt}^c$ in the testing stage to avoid bias in performance evaluation.

It should be mentioned that the only difference of the proposed SDID framework from a common SDID framework is the preprocessing scheme, i.e., data cleaning, on ${\mathit{S}}_o$. Thus, the proposed framework can be applied to any SDID model, and we expect that any SDID model can benefit from the proposed framework for better dehazing performance. This is verified in Section 4.

4. Results and Discussion

In this section, the proposed framework in Figure 3 is evaluated using three SDID models. In the experiment, the GCAN model [16], the REFN model [27], and the cGAN model [30] were used as examples. Section 4.1 explains how the parameter $\eta$ in the HLE was empirically determined. The comparison results for the three models are shown in Section 4.2 for the GCAN; Section 4.3 for the REFN, and Section 4.4 for the cGAN, respectively. The purpose of this study was not to compare the dehazing performance of the three models, but to demonstrate the effectiveness of the proposed framework by comparing the performance with and without training data cleaning.

4.1. Determination of $\eta$ in the HLE

In this section, the parameter $\eta$ in the HLE is empirically determined. As given in Section 2, there are four thresholds, $\tau$, $\eta$, ${\tau }_1$, and $ϵ$, in the HLE. For the RESIDE data set, we set the three thresholds as $\tau =0.4$, ${\tau }_1=0.9$, and $ϵ=0.1$ by experiment. In the experiment, we find that the threshold $\eta$ significantly affects the cleaning result, that is, the number of image pairs excluded from the data set. Thus, only $\eta$ is considered here. In the experiment, the RESIDE data set [28] was used as the original set ${\mathit{S}}_o$, which contains 313,950 hazy images generated from 8970 GT images. In the set ${\mathit{S}}_o$, one GT image generated 35 hazy images with different model parameters, that is, different haze levels. As shown in Figure 4, some GT images in the RESIDE data set are hazy and degrade the dehazing performance of an SDID model since the SDID model learns an inappropriate mapping. Therefore, image pairs with hazy GT images should be excluded before training an SDID model for better performance. To this end, in the experiment, the HLE was used to discriminate clear and hazy GT images, and the data cleaning was then used to discard image pairs with hazy GT images in the RESIDE data set.

Table 2. The number of hazy images and GT images after the data cleaning by the HLE.

$\eta$	0.025	0.05	0.075	0.1	Original
$N_h^c$	104,440	113,295	136,570	166,425	313,950
$N_g^c$	2984	3237	3902	4755	8970
$R \%$	33.27%	36.09%	43.50%	53.01%	100%

summarizes the number of hazy images $N_h^c$ and the number of GT images $N_g^c$ in the RESIDE data set with various $\eta$ values and also gives the ratio ($R \%$) of the number of cleaned images to the original number.

Table 2 shows that only 53.01% of GT images were clear when $\eta \le 0.1$ was used in the HLE. This suggests that using a cleaned training data set can save a lot of training time for an SDID model. Table 2 indicates a small difference in $N_h^c$ between $\eta =0.05$ and $\eta =0.025$. Therefore, in the following experiment, the cleaned data set with $\eta =0.05$, denoted as ${\mathit{S}}_o^c\left(0.05\right)$, was used and was split into training set ${\mathit{S}}_{tr}^c\left(0.05\right)$ and testing set ${\mathit{S}}_{tt}^c\left(0.05\right)$.

4.2. The GCAN Results for RESIDE Data Set

This section uses the cleaned data set ${\mathit{S}}_{tr}^c\left(0.05\right)$ to train the fully supervised deep image dehazing model called GCAN with a training set identical to that reported in [16]. The trained GCAN was then tested by ${\mathit{S}}_{tt}^c\left(0.05\right)$. The experiment was conducted as follows.

4.2.1. Objective Comparison

In the experiment, we trained the GCAN with 10,000 (10 K), 20,000 (20 K), and 30,000 (30 K) image pairs randomly selected from ${\mathit{S}}_{tr}^c\left(0.05\right)$. Then, we evaluated the three trained GCANs with three testing data subsets of ${\mathit{S}}_{tt}^c\left(0.05\right)$, with 10 K, 30 K, and 50,000 (50 K) image pairs.

Table 3. The PSNR comparison of $GCA{N}_o$, $GCA{N}_{10K}$, $GCA{N}_{20K}$, and $GCA{N}_{30K}$ models (RESIDE).

Testing Set	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{\mathit{o}}$	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{10\mathit{K}}$	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{20\mathit{K}}$	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{30\mathit{K}}$
10 K	24.89	27.53	27.93	27.99
30 K	24.96	27.56	28.01	28.07
50 K	24.97	27.57	28.02	28.08
Average	24.94	27.55	27.99	28.04

shows the PSNR result, where the subscripts in GCAN, o, 10 K, 20 K, and 30 K refer to the original GCAN and the GCANs trained with image pairs of 10 K, 20 K, and 30 K, respectively. The result in Table 3 reveals three points. First, the $GCA{N}_{10K}$, $GCA{N}_{20K}$, $GCA{N}_{30K}$, on average, have higher PSNR than the $GCA{N}_o$ by 2.61 dB, 3.05 dB, 3.10 dB, respectively. This suggests that there is better performance for the models with the proposed data cleaning scheme. Second, the PSNR increases as the number of training image pairs increases for the GCANs with cleaned training data. Moreover, it indicates a small difference of 0.44 dB between the $GCA{N}_{10K}$ and the $GCA{N}_{20K}$; 0.05 dB between the $GCA{N}_{20K}$ and the $GCA{N}_{30K}$. This suggests that a good trade-off can be made between training time and PSNR loss. Third, it has a similar performance for each comparison model when different numbers of test image pairs were used. It implies that the performance is steady and is not related to the number of test image pairs. In summary, Table 3 indicates that the proposed SDID framework can improve dehazing performance in the GCAN model, both in efficiency and effectiveness. In the following experiment, the $GCA{N}_{30K}$ was used because it has the highest PSNR.

To better understand the performance of our method, six more objective image quality assessments were used to compare the performance of $GCA{N}_o$ and $GCA{N}_{30K}$. They include three full reference evaluations (SSIM [31], TMQI [32], FSITM [33]) and three non-full reference evaluations (BRISQUE [34], ILNIQE [35] and DHQI [36]). Note that each objective assessment has its own preference. Therefore, the average rank was used as the overall performance index for a fair comparison. In the experiment, only 10 K test image pairs were used since similar PSNRs were obtained with the cases of 30 K and 50 K image pairs.

Table 4. Objective comparison of the $GCA{N}_o$ and $GCA{N}_{30K}$ models (RESIDE).

	SSIM↑	TMQI↑	FSITM↑	BRISQUE↑	ILNIQE↓	DHQI↑	$\overline{\mathit{R}}$↓
$GCA{N}_o$	0.91(2)	0.92(2)	0.73(2)	20.46(1)	21.31(2)	54.96(2)	1.83
$GCA{N}_{30K}$	0.93(1)	0.93(1)	0.74(1)	19.84(2)	20.88(1)	55.36(1)	1.17

shows the comparison result with the rank in parentheses and the average rank $\overline{R}$ where the arrow indicates the direction of better performance. The result shows that the $GCA{N}_{30K}$ has a better overall performance than the $GCA{N}_o$, as expected. It suggests that our method works well for the GCAN model.

4.2.2. Subjective Comparison

Here, a subjective comparison was made for the $GCA{N}_o$ and $GCA{N}_{30K}$ models. We selected six images with different scenes and haze levels for this evaluation. The six images are shown in

Table 5. Subjective comparison of the $GCA{N}_o$ and $GCA{N}_{30K}$ models (RESIDE).

	${\mathit{I}}_{\mathit{g}}$	${\mathit{I}}_{\mathit{h}}$	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{\mathit{o}}$	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{30\mathit{K}}$
${\mathit{I}}_1$		${\tilde{\mu }}_{dc}\left(0.9\right)$ = 0.11	PSNR = 28.09	PSNR = 28.69
${\mathit{I}}_2$		21.03/0.26	26.23	28.70
${\mathit{I}}_3$		14.22/0.36	25.61	28.68
${\mathit{I}}_4$		13.97/0.45	25.67	25.48
${\mathit{I}}_5$		0.54	28.07	29.48
${\mathit{I}}_6$		0.63	19.72	25.44

where the corresponding GT image, PSNR and ${\tilde{\mu }}_{dc}\left(0.9\right)$ are given for reference. In Table 5, the subscripts $g$ and $h$ in $\mathit{I}$ are for the GT image and the hazy image. In addition, the number in the subscript of ${\mathit{I}}_{\mathit{I}}$ is related to the haze level in ${\mathit{I}}_h$, for example, ${\mathit{I}}_1$ has the lowest haze level in the given examples. Table 5 shows that the $GCA{N}_o$ has a satisfactory dehazed result, except in ${\mathit{I}}_6$. On the other hand, the $GCA{N}_{30K}$ has a better result than the $GCA{N}_o$ except ${\mathit{I}}_4$ has a lower PSNR. Even so, ${\mathit{I}}_4$ still has an excellent visual quality. In addition, the $GCA{N}_{30K}$ is able to consistently provide high quality dehazed images at different haze level, while the $GCA{N}_o$ has difficulty in the high haze level case, i.e., ${\mathit{I}}_6$. Besides, all GT images (${\mathit{I}}_g$) in Table 5 by the proposed data cleaning are clear. This justifies that the proposed data cleaning is effective in the selection of GT images. In summary, the $GCA{N}_{30K}$ generally has better visual quality of dehazed images than the $GCA{N}_o$, even using fewer training image pairs. It implies that our data cleaning can improve the GCAN model.

4.3. The REFN Results for RESIDE Data Set

This section uses RefineDNet (REFN) [29], a weakly supervised deep image dehazing model, as the second example to verify our method. In the experiment, the REFN was trained with three subsets of 10 K, 20 K and 30 K image pairs from the training data set ${\mathit{S}}_{tr}^c\left(0.05\right)$. As before, trained REFNs were then tested by three randomly selected subsets of 10 K, 30 K, and 50 K image pairs from ${\mathit{S}}_{tt}^c\left(0.05\right)$. Objective and subjective comparisons are presented in Section 4.3.1 and Section 4.3.2, respectively.

4.3.1. Objective Comparison

In this section, we tested the REFNs trained by 10 K, 20 K and 30 K image pairs, using three subsets of 10 K, 30 K and 50 K image pairs randomly selected from ${\mathit{S}}_{tt}^c\left(0.05\right)$.

Table 6. Comparison of PSNR for $REF{N}_o$, $REF{N}_{10K}$, $REF{N}_{20K}$, and $REF{N}_{30K}$ models (RESIDE).

Testing Set	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{\mathit{o}}$	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{10\mathit{K}}$	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{20\mathit{K}}$	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{30\mathit{K}}$
10 K	23.49	28.56	29.17	28.86
30 K	23.43	28.57	29.19	28.87
50 K	23.45	28.58	29.20	28.87
Average	23.45	28.57	29.19	28.87

shows the PSNRs of the REFNs, where the subscripts o, 10 K, 20 K, and 30 K represent the original REFN and the REFNs trained with the 10 K, 20 K, and 30 K image pairs, respectively. The results in Table 6 have three implications. First, the $REF{N}_{10K}$, $REF{N}_{20K}$, and $REF{N}_{30K}$ improved the average PSNR of $REF{N}_o$ by 5.12 dB, 5.74 dB, and 5.42 dB, respectively. The results show that training an SDID model with hazy GT images degrades the dehazing performance due to the inappropriate mapping learned. Second, a larger number of training image pairs may not improve dehazing performance. That is, the $REF{N}_{30K}$ has no better result than the $REF{N}_{20K}$ that uses fewer training image pairs. Thus, it is not necessary to use all image pairs in ${\mathit{S}}_{tr}^c\left(0.05\right)$ to train the REFN. Third, a little PSNR loss happens when the $REF{N}_{10K}$ is used to trade a lower training cost. In summary, the proposed SDID framework can improve the performance of the REFN model effectively and efficiently. Since the $REF{N}_{20K}$ has the best performance, it was used in the following experiment.

As before, the $REF{N}_o$ and $REF{N}_{20K}$ were further evaluated with the six objective metrics described in Section 4.2.1.

Table 7. Objective comparison of the $REF{N}_o$ and $REF{N}_{20K}$ models (RESIDE).

	SSIM↑	TMQI↑	FSITM↑	BRISQUE↑	ILNIQE↓	DHQI↑	$\overline{\mathit{R}}$↓
$REF{N}_o$	0.93(2)	0.93(2)	0.77(2)	16.28(1)	19.78(1)	52.67(2)	1.67
$REF{N}_{20K}$	0.97(1)	0.94(1)	0.80(1)	17.66(2)	20.24(2)	56.26(1)	1.33

shows the results of SSIM, TMQI, FSITM, BRISQUE, ILNIQE and DHQI, along with the average rank $\overline{R}$. The results indicate that the $REF{N}_{20K}$ has a better $\overline{R}$ than the $REF{N}_o$, suggesting that the proposed data cleaning scheme is able to improve the dehazing performance of the REFN model, as expected.

4.3.2. Subjective Comparison

This section compares the subjective visual quality of $REF{N}_o$ and $REF{N}_{20K}$. Six images with different scenes and haze levels were selected as examples. The notations used here are the same as those in Section 4.2.2. The six images are shown in

Table 8. Subjective comparison of the $REF{N}_o$ and $REF{N}_{20K}$ models (RESIDE).

	${\mathit{I}}_{\mathit{g}}$	${\mathit{I}}_{\mathit{h}}$	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{\mathit{o}}$	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{20\mathit{K}}$
${\mathit{I}}_1$		${\tilde{\mu }}_{dc}\left(0.9\right)$ = 0.15	PSNR = 23.05	PSNR = 31.50
${\mathit{I}}_2$		0.25	27.19	33.69
${\mathit{I}}_3$		0.36	19.49	24.48
${\mathit{I}}_4$		0.49	25.31	25.40
${\mathit{I}}_5$		0.53	22.30	27.89
${\mathit{I}}_6$		0.64	20.42	22.66

with their dehazed images by the $REF{N}_o$ and $REF{N}_{20K}$. For reference, the corresponding PSNR and ${\tilde{\mu }}_{dc}\left(0.9\right)$ are also given in Table 8, except for ${\mathit{I}}_g$. Table 8 indicates that the $REF{N}_o$ has an excellent visual quality, while the $REF{N}_{20K}$ generally gives an even better result. As expected, the selected GT images by the proposed data cleaning are clear in the given examples. The results in Table 8 indicate that our method is capable of improving the performance of the REFN model.

4.4. The cGAN Results RESIDE Data Set

In this section, cGAN [30] is used as the final example to justify our method. The cGAN is a type of SDID model that uses a generative adversary scheme. Section 4.4.1 shows the objective comparison results, while Section 4.4.2 gives the subjective comparison results.

4.4.1. Objective Comparison

In the experiment, the conditional adversarial generative network (cGAN) in [30] was trained using 10 K, 20 K, and 30 K image pairs randomly selected from ${\mathit{S}}_{tr}^c\left(0.05\right)$. The trained cGAN models were then tested, respectively, by 10 K, 20 K and 30 K image pairs randomly selected from ${\mathit{S}}_{tt}^c\left(0.05\right)$.

Table 9. Comparison of PSNR for $cGA{N}_e$, $cGA{N}_{10K}$, $cGA{N}_{20K}$, and $cGA{N}_{30K}$ models (RESIDE).

Testing Set	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{\mathit{e}}$	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{10\mathit{K}}$	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{20\mathit{K}}$	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{30\mathit{K}}$
10 K	22.34	28.11	28.41	28.75
30 K	22.33	28.14	28.45	28.78
50 K	22.31	28.15	28.46	28.79
Average	22.33	28.13	28.44	28.77

shows the PSNR for the compared models, where the subscripts e, 10 K, 20 K, and 30 K represent the enhanced cGAN in [18] and the cGAN trained with the 10 K, 20 K, and 30 K image pairs, respectively. Since no SDID model using cGAN [30] is available, the enhanced cGAN in [18] was used here for comparison. There are three points that should be mentioned for the results in Table 9. First, the PSNR of $cGA{N}_{10K}$, $cGA{N}_{20K}$, and $cGA{N}_{30K}$ is better than that of $cGA{N}_e$ by 5.8 dB, 6.11 dB, and 6.44 dB, respectively. It proves that the proposed data cleaning is effective. Second, the PSNR increases as the number of training image pairs increases, although the difference is not significant. It implies that more training image pairs benefit the dehazing performance of cGAN. Third, the PSNR differences for $cGA{N}_{10K}$, $cGA{N}_{20K}$, and $cGA{N}_{30K}$ are less than 0.64 dB. Thus, it gives a good trade-off between PSNR and the number of training image pairs. In summary, our method improves effectiveness and efficiency in the cGAN model.

The $cGA{N}_e$ and $cGA{N}_{30K}$ were further evaluated with SSIM, TMQI, FSITM, BRISQUE, ILNIQU, and DHQI, as before.

Table 10. Objective comparison of the $cGA{N}_e$ and $cGA{N}_{30K}$ models (RESIDE).

	SSIM↑	TMQI↑	FSITM↑	BRISQUE↑	ILNIQE↓	DHQI↑	$\overline{\mathit{R}}$↓
$cGA{N}_e$	0.91(2)	0.89(2)	0.74(2)	10.99(1)	20.39(1)	57.90(1)	1.50
$cGA{N}_{30K}$	0.94 (1)	0.94(1)	0.76(1)	11.88(2)	33.77(2)	57.75(2)	1.50

shows the results and the average rank $\overline{R}$. Table 10 indicates that the $cGA{N}_{30K}$ has $\overline{R}$ equal to that of the $cGA{N}_e$. Even so, the $cGA{N}_{30K}$ has a better SSIM, TMQI, and FSITM, which are full reference metrics. That is, they are more reliable and suggest that better visual quality is obtained for the $cGA{N}_{30K}$. This is verified in the following.

4.4.2. Subjective Comparison

For a subjective comparison, six images were selected from the dehazed images by the $cGA{N}_e$ and $cGA{N}_{30K}$. As in the previous section, these images were selected with different scenes and hazy levels. They are shown in

Table 11. Subjective comparison of the $cGA{N}_e$ and $cGA{N}_{30K}$ models (RESIDE).

	${\mathit{I}}_{\mathit{g}}$	${\mathit{I}}_{\mathit{h}}$	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{\mathit{e}}$	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{30\mathit{K}}$
${\mathit{I}}_1$		${\tilde{\mu }}_{dc}\left(0.9\right)$ = 0.12	PSNR = 23.44	PSNR = 30.78
${\mathit{I}}_2$		0.27	27.41	27.70
${\mathit{I}}_3$		0.35	15.49	24.14
${\mathit{I}}_4$		0.43	26.30	24.92
${\mathit{I}}_5$		0.55	16.45	27.63
${\mathit{I}}_6$		0.60	23.57	26.58

with the corresponding GT images, the dehazed images by the $cGA{N}_e$ and $cGA{N}_{30K}$, and their haze levels measured by ${\tilde{\mu }}_{dc}\left(0.9\right)$. For notation, the number in the subscript of ${\mathit{I}}_{\mathit{I}}$ relates to its haze level. In the given examples, the $cGA{N}_e$ works well, except for ${\mathit{I}}_5$, while the $cGA{N}_{30K}$, which used fewer training image pairs, has better visual quality in all cases. Though the $cGA{N}_{30K}$ has the same $\overline{R}$ as the $cGA{N}_e$, it has a better subjective result, as described above. In summary, the results show that our method can improve the cGAN effectively and efficiently.

4.5. The KeDeMa Results

In this section, the $GCA{N}_o$, $GCA{N}_{30K}$, $REF{N}_o$, $REF{N}_{20K}$, $cGA{N}_e$, and $cGA{N}_{30K}$ were further evaluated by a real-world image data set, KeDeMa in [37]. The KeDeMa data set consists of 25 natural images with different scenarios that were collected from several pioneering papers. The purpose of this experiment was to understand the robustness of our method. In other words, the dehazing performance of the proposed method was investigated using a data set outside ${\mathit{S}}_o^c\left(0.05\right)$. Objective and subjective comparisons for the GCAN, REFN and cGAN models are given in Section 4.5.1, Section 4.5.2, and Section 4.5.3, respectively.

4.5.1. The GCAN Results for KeDeMa Data Set

Here, the $GCA{N}_o$ and $GCA{N}_{30K}$ were evaluated. Since PSNR, SSIM, TMQI, and FSITM require GT images in the calculation, they are not used in the objective comparison. Only BRISQUE, ILNIQE, and DHQI were used in the experiment. The objective results are shown in

Table 12. Objective comparison of the $GCA{N}_o$ and $GCA{N}_{30K}$ models (KeDeMa).

	BRISQUE↑	ILNIQE↓	DHQI↑	$\overline{\mathit{R}}$↓
$GCA{N}_o$	19.27(2)	26.30(2)	50.23(2)	2
$GCA{N}_{30K}$	17.28(1)	25.93(1)	50.90(1)	1

. They indicate that the $GCA{N}_{30K}$ is objectively superior to the $GCA{N}_o$ by the average rank $\overline{R}$.

Next, a subjective comparison was made for the $GCA{N}_o$ and $GCA{N}_{30K}$. Six images were selected from the dehazed images, as shown in

Table 13. Subjective comparison of the $GCA{N}_o$ and $GCA{N}_{30K}$ models (KeDeMa).

	${\mathit{I}}_{\mathit{h}}$	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{\mathit{o}}$	$\mathit{G}\mathit{C}\mathit{A}{\mathit{N}}_{30\mathit{K}}$
${\mathit{I}}_1$
${\mathit{I}}_2$
${\mathit{I}}_3$
${\mathit{I}}_4$
${\mathit{I}}_5$
${\mathit{I}}_6$

, where the number in the subscript of ${\mathit{I}}_{\mathit{I}}$ is the order of presentation and is not related to the haze level. For visual quality, the $GCA{N}_o$ has the problems of halo, artifacts and color distortion. The color distortion in an image refers to hue distortion, saturation distortion, or both. Thus, all six dehazed images by the $GCA{N}_o$ have severe color distortion in different degrees. In general, in a dehazed image, halos occur in the large depth discontinuities. In the given examples, ${\mathit{I}}_3$ and ${\mathit{I}}_5$ have the halo problem for the $GCA{N}_o$. It occurs in the front tree areas in ${\mathit{I}}_3$, while occurring in the branches of the trees in ${\mathit{I}}_5$. An artifact refers to any feature introduced in the dehazing process that is not in the original image. According to this definition, the $GCA{N}_o$ has the artifact problem in ${\mathit{I}}_2$ (sky region), ${\mathit{I}}_3$ (dark areas on the road), ${\mathit{I}}_4$ (sky region), and ${\mathit{I}}_6$ (right lower corner). On the contrary, the $GCA{N}_{30K}$ has none of the above problem. However, the $GCA{N}_{30K}$ has difficulty in removing haze in ${\mathit{I}}_3$. For other dehazed images, it gives satisfactory visual quality. Although our method shows an improvement in the GCAN model, dehazing effects of our method should be improved in future research.

4.5.2. The REFN Results for KeDeMa Data Set

In this section, the objective and subjective comparisons of $REF{N}_o$ and $REF{N}_{20K}$ were made. The objective results of BRISQUE, ILNIQE, and DHQI are recorded in

Table 14. Objective comparison of the $REF{N}_o$ and $REF{N}_{20K}$ models (KeDeMa).

	BRISQUE↑	ILNIQE↓	DHQI↑	$\overline{\mathit{R}}$↓
$REF{N}_o$	14.33(1)	23.43(1)	48.25(2)	1.33
$REF{N}_{20K}$	16.97(2)	26.11(2)	51.06(1)	1.67

with $\overline{R}$. Table 14 shows that the $REF{N}_o$ has a better $\overline{R}$ than the $REF{N}_{20K}$. Even so, the visual quality is better for the $REF{N}_{20K}$ because of a better DHQI, which was specifically developed to evaluate dehazed images. That is, a better DHQI generally means a better dehazed image. This is justified in the following.

Next, we compared the visual quality of the dehazed images obtained from the $REF{N}_o$ and $REF{N}_{20K}$. As before, we selected six images, which are shown in

Table 15. Subjective comparison of the $REF{N}_o$ and $REF{N}_{20K}$ models (KeDeMa).

	${\mathit{I}}_{\mathit{h}}$	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{\mathit{o}}$	$\mathit{R}\mathit{E}\mathit{F}{\mathit{N}}_{20\mathit{K}}$
${\mathit{I}}_1$
${\mathit{I}}_2$
${\mathit{I}}_3$
${\mathit{I}}_4$
${\mathit{I}}_5$
${\mathit{I}}_6$

. Table 15 indicates that the $REF{N}_o$ has a stronger dehazing effect than the $REF{N}_{20K}$. However, it introduces artifacts in ${\mathit{I}}_1$, i.e., contours in the sky region and color distortion in ${\mathit{I}}_3$, ${\mathit{I}}_4$, ${\mathit{I}}_5$, and ${\mathit{I}}_6$. On the other hand, the $REF{N}_{20K}$ retains the mood of hazy image ${\mathit{I}}_h$ and has a less dehazed result in ${\mathit{I}}_2$ and ${\mathit{I}}_5$. For other images, the $REF{N}_{20K}$ yields good visual quality. In summary, the $REF{N}_{20K}$ is generally able to provide a satisfactory result in the given examples. The results imply that our method can improve the REFN model. Furthermore, better dehazing performance should be sought for our method to avoid less dehazed cases.

4.5.3. The cGAN Results KeDeMa Data Set

In this section, the $cGA{N}_e$ and $cGA{N}_{30K}$ were objectively and subjectively compared. The objective comparison results are given in

Table 16. Objective comparison of the $cGA{N}_e$ and $cGA{N}_{30K}$ models (KeDeMa).

	BRISQUE↑	ILNIQE↓	DHQI↑	$\overline{\mathit{R}}$↓
$cGA{N}_e$	10.50(1)	24.84(1)	63.46(2)	1.33
$cGA{N}_{30K}$	29.97(2)	37.81(2)	55.31(1)	1.67

. The $cGA{N}_{30K}$ has a worse $\overline{R}$ than the $cGA{N}_e$. However, the $cGA{N}_{30K}$ has a much better DHQI, which implies better visual quality of dehazed images. This is verified in the following.

Next, a subjective comparison was made with six selected dehazed images for the $cGA{N}_e$ and $cGA{N}_{30K}$. The six images and the corresponding dehazed images are given in

Table 17. Subjective comparison of the $cGA{N}_e$ and $cGA{N}_{30K}$ models (KeDeMa).

	${\mathit{I}}_{\mathit{h}}$	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{\mathit{e}}$	$\mathit{c}\mathit{G}\mathit{A}{\mathit{N}}_{30\mathit{K}}$
${\mathit{I}}_1$
${\mathit{I}}_2$
${\mathit{I}}_3$
${\mathit{I}}_4$
${\mathit{I}}_5$
${\mathit{I}}_6$

. The results show that the $cGA{N}_e$ has a much stronger dehazing effect than the $cGA{N}_{30K}$. However, it has problems in the dehazed images. Severe color distortion is observed in all six images. In addition, halos are found in ${\mathit{I}}_3$ (around the policeman), ${\mathit{I}}_4$ (around the woman), ${\mathit{I}}_5$ (around the front hill) and ${\mathit{I}}_6$ (the ridge of the front mountain). As for the $cGA{N}_{30K}$, none of the above problems are observed. In general, the dehazed image retains a tone similar to that of the corresponding input image. However, the $cGA{N}_{30K}$ has a less dehazed result in ${\mathit{I}}_3$ and ${\mathit{I}}_5$. It suggests that the approach proposed in this study can improve the cGAN model. However, further research should be performed to improve it.

The experimental results show that the proposed method generally performs better in the given GCAN, REFN, and cGAN models. However, they also indicate that a few images are not dehazed well in the KeDeMa data set. The reason might be that we only used the RESIDE data set to train the models in this study. It would be helpful to include more data sets in training, that is, to increase modalities for better generalization. This will be our work in further research.

5. Conclusions and Further Research

This paper has presented a data cleaning scheme for supervised deep image dehazing (SDID) models that uses haze level estimation (HLE). In light of the dark channel, HLE was developed and applied to detect hazy GT images in a data set, e.g., the RESIDE data set [28]. The image pairs with hazy GT images were excluded when training an SDID model. The preprocessing scheme was called data cleaning in this study. Furthermore, an SDID framework was given in which data cleaning was integrated to improve the dehazing performance. The proposed framework was verified by three SDID models, the GCAN, REFN, and cGAN, using the RESIDE data set. The results indicate that the dehazing performance was improved by 3.10 dB, 5.74 dB, and 6.44 dB in PSNR for the GCAN, REFN, and cGAN, respectively. Furthermore, our method was further verified by the real-world data set KeDeMa in [37]. The results show that the models with our data cleaning scheme generally have better visual quality than those without it. This suggests that our method is able to improve the dehazing performance in the given example models. It could also benefit other SDID models. Although our method generally works well, four points need to be mentioned. First, this method is suitable for a large-scale data set, e.g., the RESIDE data set, since it may significantly reduce the data size after data cleaning and affect dehazing performance. Second, the parameters in the proposed HLE should be adjusted when using a data set other than the RESIDE data set. Third, the HLE works well in most cases to discriminate clear and hazy images. However, it fails in some images, and thus, an improvement should be made in the HLE. Fourth, the models that use our method have a less dehazed result in the KeDeMa data set. More data sets in training an SDID model may be needed for better performance. These points will guide our further research.