PM_2.5 Concentration Measurement Based on Image Perception

Abstract

PM_2.5 in the atmosphere causes severe air pollution and dramatically affects the normal production and lives of residents. The real-time monitoring of PM_2.5 concentrations has important practical significance for the construction of ecological civilization. The mainstream PM_2.5 concentration prediction algorithms based on electrochemical sensors have some disadvantages, such as high economic cost, high labor cost, time delay, and more. To this end, we propose a simple and effective PM_2.5 concentration prediction algorithm based on image perception. Specifically, the proposed method develops a natural scene statistical prior to estimating the saturation loss caused by the ’haze’ formed by PM_2.5. After extracting the prior features, this paper uses the feedforward neural network to achieve the mapping function from the proposed prior features to the PM_2.5 concentration values. Experiments constructed on the public Air Quality Image Dataset (AQID) show the superiority of our proposed PM_2.5 concentration measurement method compared to state-of-the-art related PM_2.5 concentration monitoring methods.

1. Introduction

With the rapid development of society and the further acceleration of energy consumption, the atmospheric environmental quality is also severely affected. Particulate matter is one of the major pollutants causing air pollution, especially PM_2.5, which has become the primary pollutant in most cities. PM_2.5 is seriously harmful to both people’s health and the ecological environment [1,2,3,4]. The complex chemical composition of PM_2.5 can cause a wide variety of health hazards [5]. PM_2.5 will cause serious irritation to the respiratory tract when it enters the bronchial tubes of the human body and will have a serious impact on the gas exchange in the lungs, causing coughing, breathing difficulties, asthma, bronchitis, and other problems. In serious cases, it can cause heart arrhythmia, and heart disease [6,7,8]. People with low resistance, such as the elderly and children, are particularly vulnerable to the effects of PM_2.5 [9,10]. Therefore, effective monitoring and control of PM_2.5 concentration has a positive role in improving human health [11,12,13]. Moreover, the picture-based PM_2.5 concentration estimation methods can also be applied to other visual tasks, such as PM_2.5 concentration, which can be used to assess the credibility of person re-identification results in hazy weather [14,15,16]. In the rain and fog image enhancement task, the suitable image defogging and rain removal algorithms can be selected adaptively for different rain and fog concentration images based on PM_2.5 concentration [17,18].

To monitor PM_2.5 concentration effectively, many high-precision electronic instruments have been designed and applied to daily life. Figure 1 shows part of the PM_2.5 electronic monitoring equipment. The professional electronic equipment can usually obtain accurate PM_2.5 monitoring results. However, high economic and labor costs are required to purchase and maintain electronic equipment, which limits the application scenarios of this equipment. Thus, it is of great significance to design a more convenient PM_2.5 concentration prediction solution. To this end, in recent years, some researchers have proposed some photo-based PM_2.5 concentration prediction algorithms. With the popularization of intelligent devices with the photo-taking function, such as driving recorders and smartphones, people can access high-quality photos anytime and anywhere. Therefore, the picture-based PM_2.5 monitoring measurement is more convenient for people to use in their daily lives. Liu et al. extracted six image features of transmission—sky smoothness, color, global and local contrast, and entropy—to measure PM_2.5 concentration [19]. In the literature [20], Gu et al. found that the entropy features of high and low PM_2.5 concentration images are statistically different in the spatial and transform domains and mapped the deviation degree to PM_2.5 concentration values by a nonlinear function. Zhang et al. introduced visual saliency to build a PM_2.5 concentration detector [21]. The method first obtains the non-salient regions of the photo through the saliency detection technique. Then, the entropy and intensity values of the saturation map of the image non-salient regions are extracted to detect the PM_2.5 concentration. All the abovementioned PM_2.5 monitors based on the image entropy and energy features are usually influenced by the diversity of image content, thus limiting the performance of these PM_2.5 concentration monitors. To alleviate the problem, Yue et al. calculate the gradient and Weibull distribution of image saturation map to quantify the loss of PM_2.5 to the image and use it to estimate the PM_2.5 concentration [22]. However, this also ignores the image content loss caused by particulate matter. Sun et al. designed a discrepancy learning-based PM_2.5 concentration estimator [23]. The method first uses discrepancy learning to obtain the ability to capture the error map due to particulate matter from a large number of fog images with their corresponding clean images, and then estimates PM_2.5 concentration via the structure, color, and energy features of the error map. The method inevitably learns information about the image itself besides particulate matter, which is also the main factor in the method producing PM_2.5 concentration measurement error. To further solve the shortcomings of the above works, this paper designs a prior to measure the color loss introduced by particulate matter during imaging from the perspective of natural scene statistics. The key to the effectiveness of the developed algorithm is that the color loss features based on natural scene statistics are able to effectively attenuate the adverse impact of image content diversity on the performance of the photo-based PM_2.5 concentration measurement model.

To measure the PM_2.5 concentration more effectively based on image perception, this paper develops a natural scene statistical prior to estimating the image color saturation loss due to the ‘haze’ formed by PM_2.5. The natural scene statistical prior for the color saturation loss (NSSP-CSL) consists of two steps: (a) obtaining the image saturation map by performing hue-saturation-value (HSV) transformation on the PM_2.5 image; and (2) using the asymmetric generalized Gaussian distribution (AGGD) function to fit the mean subtracted contrast normalized coefficients (MSCN) of the saturation map. Then, to implement the final PM_2.5 concentration monitor, we employ a feedforward neural network (FNN) model to learn the mapping of the obtained prior features to PM_2.5 concentration values. Extensive experiments on the public Air Quality Image Database (AQID) [20] show that the proposed method has great advantages over mainstream photo-based PM_2.5 monitoring benchmark methods.

The remaining sections of this paper are outlined as follows. Section 2 describes the natural scene statistical prior designed for image color loss and the FNN-based PM_2.5 monitoring model. Section 3 validates the effectiveness of our PM_2.5 concentration measurement method. Section 4 summarizes the whole paper.

2. Proposed Method

This section elaborates the proposed photo-based PM_2.5 concentration monitor in terms of both the natural scene statistical prior extraction and the FNN-based PM_2.5 concentration monitoring model. As shown in Figure 2a, the increase in PM_2.5 concentration is most intuitively represented in an image as reduced color richness. For this purpose, we propose a natural scene statistical prior, namely NSSP-CSL, for the color loss of PM_2.5 images.

2.1. Natural Scene Statistical Prior for the Color Saturation Loss

As shown in Figure 2b, we can easily find that the saturation pixel values of the high concentration PM_2.5 images are closer to zero compared to the low concentration PM_2.5 images. The saturation channel of the high concentration PM_2.5 image retains less image information compared to the grayscale map. Existing studies have shown that the AGGD of the mean subtracted contrast normalized (MSCN) coefficients of grayscale images can effectively quantify the pollution degree of different artifacts (such as JPEG, white noise, and blur, etc.) to the image quality [24,25,26,27,28,29]. In addition, PM_2.5 can be seen as an artifact that causes degradation of image quality. Thus, we suppose whether the MSCN distributions of the saturation channels corresponding to different PM_2.5 concentration images are also statistically different. To this end, we perform a conjecture validation on 500 images with different PM_2.5 concentrations derived from the AQID [20]. Figure 3 presents an example of the MSCN distributions of the saturation and grayscale maps for three images with different PM_2.5 concentrations. From Figure 3b, we can see that the MSCN distributions of image saturation corresponding to different PM_2.5 concentrations have apparent differences. Specifically, with the decrease of PM_2.5 concentration, the corresponding peak value of MSCN distribution decreases along with it. In addition, the distribution shape of MSCN at low PM_2.5 concentration is also smoother than that at high PM_2.5 concentration. Figure 3c shows that the MSCN distribution of images with different PM_2.5 concentrations in the grayscale channel is not significantly different. Section 3.4 compares the performance of the MSCN distribution of image saturation and grayscale channels on the public AQID.

The above findings indicate that the MSCN distribution of image saturation channel information can effectively evaluate the color loss caused by PM_2.5 to photos during imaging, thus enabling accurate photo-based monitoring of PM_2.5 concentration. Next, we will describe how to extract the MSCN distribution of the image saturation channel. First, we transform the image from RGB color space to HSV color space to achieve the image saturation channel, which is calculated as follows:

$$S(x,y)=\left\{\begin{array}{cc}\frac{M(x,y)-N(x,y)}{M(x,y)},\hfill & \mathrm{if}M(x,y)\ne 0\hfill \\ 0,\hfill & \mathrm{if}M(x,y)=0\hfill \end{array}\right.$$

(1)

where x and y are the pixel indices in the horizontal and vertical directions of the image, respectively. $M(x,y)$ and $N(x,y)$ denote the maximum and mimimum pixel values at pixel location $(x,y)$ in channels R, G, and B of an image, respectively, i.e., $M(x,y)$ = max[R$(x,y)$ G$(x,y)$ B$(x,y)$], $N(x,y)$ = min[R$(x,y)$ G$(x,y)$ B$(x,y)$]. We then calculate the MSCN coefficients of the image saturation map by using the following formula:

$$S_{MSCN}(x,y)=\frac{S(x,y)-\mu (x,y)}{\sigma (x,y)+C},$$

(2)

where C is a positive constant to avoid the instability brought about by a zero denominator. $\mu (x,y)$ and $\sigma (x,y)$ represent the mean and variance of the local area, respectively. The specific definitions of $\mu (x,y)$ and $\sigma (x,y)$ are as follows:

$$\mu (x,y)=\sum _{k=-K}^K\sum _{l=-L}^L{\omega }_{k,l}{S}_{k,l}(x,y),$$

(3)

$$\sigma (x,y)=\sqrt{\sum _{k=-K}^K\sum _{l=-L}^L{\omega }_{k,l}{\left(S_{k,l}(x,y)-\mu (x,y)\right)}^2},$$

(4)

where $\omega =\left\{{\omega }_{k,l}\right|k=-K,...,K,l=-L,...,L\}$ is a 2D Gaussian weighting function. We empirically set K and L to 3 according to related studies [24,25]. Figure 3b shows that the MSCN distribution of the image saturation channel generally follows an asymmetric generalized Gaussian distribution. Thus, we use the AGGD function to fit the MSCN distribution of the image saturation channel to realize the extraction of the MSCN distribution characteristics, as the first group of natural scene statistical priors for color loss in the proposed PM_2.5 monitoring algorithm. The AGGD function is defined as follows:

$$\begin{array}{ccc}\hfill f(x;v,& \left.{\sigma }_l^2,{\sigma }_r^2\right)\hfill & \hfill =\left\{\begin{array}{cc}\frac{v}{\left({\beta }_l+{\beta }_r\right)\mathrm{\Gamma }\left(\frac{1}{v}\right)}exp\left(-{\left(\frac{-x}{{\beta }_r}\right)}^v\right),\hfill & x\ge 0,\hfill \\ \frac{v}{\left({\beta }_l+{\beta }_r\right)\mathrm{\Gamma }\left(\frac{1}{v}\right)}exp\left(-{\left(\frac{-x}{{\beta }_l}\right)}^v\right),\hfill & x<0,\hfill \end{array}\right.\end{array}$$

(5)

where

$$\begin{array}{c}\mathrm{\Gamma }\left(a\right)={\int }_0^{\infty }{t}^{(a-1)}{e}^{-t}dt,a>0,\\ {\beta }_l={\sigma }_l\sqrt{\frac{\mathrm{\Gamma }\left(\frac{1}{v}\right)}{\mathrm{\Gamma }\left(\frac{3}{v}\right)}},{\beta }_r={\sigma }_r\sqrt{\frac{\mathrm{\Gamma }\left(\frac{1}{v}\right)}{\mathrm{\Gamma }\left(\frac{3}{v}\right)}}.\end{array}$$

(6)

The parameter v represents the shape of the MSCN distribution. The parameters ${\sigma }_l^2$ and ${\sigma }_r^2$ denote the spread on the left and right sides of the MSCN distribution, respectively. The three parameters v, ${\sigma }_l^2$ and ${\sigma }_r^2$ are estimated by using the moment-matching method in the literature [30]. The mean of the left and right side of the distribution is also another valid parameter commonly used to characterize the distribution, which is defined as

$$\eta =\left({\beta }_r-{\beta }_l\right)\frac{\mathrm{\Gamma }\left(\frac{2}{v}\right)}{\mathrm{\Gamma }\left(\frac{1}{v}\right)}.$$

(7)

The above four parameters ($v,{\sigma }_l^2,{\sigma }_r^2,\eta$) of the AGGD fit serve as the first set of natural scene statistical priors for color saturation loss due to PM_2.5 during imaging.

2.2. Proposed FNN-Based PM_2.5 Concentration Measurement Model

Among existing studies on picture-based PM_2.5 concentration monitoring, the support vector machine (SVM) and random forest (RF) are often adopted to fuse the extracted image-aware priors to measure PM_2.5 concentration [31,32,33]. However, due to the shallow structure of SVM and RF, it is difficult for such machine-learning models to optimally map perceptual features to a PM_2.5 concentration value. To this end, the FNN is adopted to learn our PM_2.5 concentration measurement model, which contains multiple hidden layers that can effectively overcome the shortcomings of the SVM and RF algorithms. Section 3.3 performs the ablation studies of the SVM, RF, and FNN algorithms. Figure 4 shows the overview of our PM_2.5 concentration monitoring method. The adopted FNN contains a feature input layer, 28 hidden layers, and an output layer; the details about the setting of the number of hidden layers are described in Section 3.3. In addition, this paper adopts the MSE loss and the gradient descent method to optimize the parameters of FNN.

3. Results

3.1. Testing Dataset

The AQID [20,22] consists of 750 pictures with PM_2.5 concentrations ranging from 1 to 423 mu g/m³, where the resolution range of the photos is 500 × 261 to 978 × 550. The testing dataset contains various scenes such as roads, cars, houses, squares, etc. The PM_2.5 concentration values corresponding to the photo samples in the dataset were collected by the MetOne BAM-1020 instrument. The MetOne BAM-1020 instrument detects the PM_2.5 concentration value based on the β-ray absorption method. To ensure the quality of the collected samples, the following two rules were followed when collecting samples. First, the photo sample collection area is within a radius of 1 km centered on the air-quality monitoring equipment to ensure the PM_2.5 concentration value is close to the real PM_2.5 concentration value reported by the monitoring point. Second, because the sky area of the photo is more sensitive to PM_2.5 concentration changes, the sky area in the sample should occupy about 1/3–1/2 of the entire photo. At the same time, these photos were collected without facing the sun to avoid the influence of the light intensity on PM_2.5 monitoring accuracy. Figure 5 presents some samples from the testing dataset.

3.2. Evaluation Criteria

Drawing on Gu et al.’s [20], Yue et al.’s [22], and Sun et al.’s [23] studies, this paper adopts the Pearson correlation coefficient (PCC), the Spearman correlation coefficient (SCC), and the Kendall correlation coefficient (KCC) to evaluate the performance of the PM_2.5 concentration monitoring algorithms. The specific definition of the Pearson correlation coefficient is given as follows:

$${\hslash }_{PCC}=\frac{{\sum }_i(b_i-\overline{b})(l_i-\overline{l})}{\sqrt{{\sum }_i{(b_i-\overline{b})}^2{\sum }_i{l}_i-\overline{l}{)}^2}},$$

(8)

where $b_i$ is the PM_2.5 concentration estimate corresponding to the i-th PM_2.5 photo. $\overline{b}$ is the mean value of all $b_i$. The spearman correlation coefficient is defined as

$${\hslash }_{SCC}=1-\frac{6{\sum }_{q=1}^Q{d}_q^2}{Q(Q^2-1)},$$

(9)

where the Q is the pair number of the estimated and actual values of the PM_2.5 concentration. $d_q$ is the ranked difference between the estimated and actual values of PM_2.5 concentration in each group. The Kendall correlation coefficient is computed as

$${\hslash }_{KCC}=\frac{Q_c-Q_d}{0.5(Q^2-Q)},$$

(10)

where $Q_c$ and $Q_d$ represent the number of prediction consistent and inconsistent pairs, respectively. The PCC measures the estimation accuracy. In addition, the outputs of the PM_2.5 concentration measurement models need to be non-linearly mapped by a logistic function before calculating the PCC value [20,33,34]. The SCC and KCC measure the estimation monotonicity. The larger values of the PCC, SCC, and KCC represent the better performance of the PM_2.5 monitoring algorithm.

3.3. Ablation Study on the Regression Models

This part tests the impact of different types of machine-learning algorithms, including SVM, RF, and FNN, on the accuracy of our proposed PM_2.5 concentration measurement method. The experimental results are shown in

Table 1. The performance of our proposed PM_2.5 concentration estimation algorithm based on SVR, RF, and FNN on the AQID.

Model	${\mathit{\hslash }}_{\mathbf{PCC}}$	${\mathit{\hslash }}_{\mathbf{SCC}}$	${\mathit{\hslash }}_{\mathbf{KCC}}$
SVR	0.8411	0.8039	0.5883
RF	0.8385	0.7995	0.5717
FNN	0.8228	0.8241	0.6100

. From Table 1, it is clear that different types of machine-learning algorithms have a significant impact on the prediction accuracy of the proposed PM_2.5 monitoring algorithm. The SVR algorithm outperforms the RF and FNN algorithms in terms of estimation accuracy. The excellent performance of SVR on PCC may lie in the nonlinear mapping of the logistic function to the model outputs. However, the SCC and KCC can better reflect the sensitivity of the PM_2.5 concentration prediction models to the dynamic transformation of PM_2.5 concentration. The FNN algorithm is much better than the RF and SVR algorithms in terms of estimation monotonicity. Therefore, combining accuracy and monotonicity, the FNN algorithm is finally used to fuse image perception features to achieve our PM_2.5 concentration monitoring model. Note that the training, validation, and test data account for 80%, 10%, and 10% of the whole dataset, respectively.

Moreover,

Table 2. The performance of the FNN-based PM_2.5 concentration monitoring model with the different number of hidden layers on the AQID. The reason for the numbers 0 and NAN in Table 2 is that the shallow hidden layer leads to the difficult convergence of model training.

Num.	${\mathit{\hslash }}_{\mathbf{PCC}}$	${\mathit{\hslash }}_{\mathbf{SCC}}$	${\mathit{\hslash }}_{\mathbf{KCC}}$	Num.	${\mathit{\hslash }}_{\mathbf{PCC}}$	${\mathit{\hslash }}_{\mathbf{SCC}}$	${\mathit{\hslash }}_{\mathbf{KCC}}$
1	0	NAN	NAN	16	0.7932	0.7806	0.5594
2	0	NAN	NAN	17	0.1015	0.6048	0.4316
3	NAN	NAN	NAN	18	0.7953	0.7954	0.5869
4	0.1227	0.1799	0.1478	19	0.1293	0.6598	0.4590
5	0.1145	0.4200	0.3280	20	−0.1206	−0.3824	−0.2900
6	0.8009	0.7962	0.5840	21	0.8070	0.8093	0.6006
7	0.1778	0.3257	0.2484	22	0.8012	0.7991	0.5897
8	0.0710	0.3443	0.2446	23	0.7883	0.8030	0.5984
9	0.0672	0.4975	0.3718	24	0.8035	0.8001	0.5890
10	−0.1265	−0.3282	−0.2587	25	0.7821	0.7770	0.5587
11	0.7981	0.7926	0.5796	26	0.8290	0.8202	0.6027
12	−0.1573	−0.6926	−0.5092	27	0.8112	0.7931	0.5688
13	0.3125	0.3008	0.2037	28	0.8228	0.8241	0.6100
14	0.7909	0.7867	0.5760	29	0.7999	0.7958	0.5897
15	0.7764	0.7817	0.5746	30	0.8130	0.7950	0.5767

presents the ablation experiment results for the effect of the number of hidden layers included in the FNN on the prediction accuracy of our proposed PM_2.5 concentration monitoring method. The experimental results in Table 2 demonstrate that when the number of hidden layers reaches 21, the prediction robustness of our proposed PM_2.5 concentration monitoring algorithm tends to be stable. Considering the three performance indicators, this paper finally sets the number of hidden layers to 28.

3.4. Performance Comparison with the Benchmark Algorithms

To prove the performance superiority of our proposed PM_2.5 monitoring algorithm, three groups of algorithms, namely the image sharpness measurement, the image contrast measurement, and the photo-based PM_2.5 monitoring, are selected as competing algorithms. The image contrast measurement methods include NIQMC [35] and BIQME [36]. The image sharpness assessment methods consist of FISH [37], ARISM [38], and BIBLE [39]. The picture-based PM_2.5 monitoring algorithms include PPPC [20], Yue et al.’s method [22], Zhang et al.’s method [21], and Sun et al.’s method [33]. The reason for collecting the image sharpness and contrast estimation algorithms as the competing algorithms is that PM_2.5 usually affects the contrast and sharpness of photos during imaging.

Table 3. Performance comparison of the picture-based PM_2.5 monitoring algorithms, the image contrast and sharpness measurement algorithms on the AQID. * denotes that the corresponding results are collected in the original papers.

Method	Type	${\mathit{\hslash }}_{\mathbf{PCC}}$	${\mathit{\hslash }}_{\mathbf{SCC}}$	${\mathit{\hslash }}_{\mathbf{KCC}}$
NIQMC	Contrast	0.4229	0.4427	0.2966
BIQME	Contrast	0.5441	0.5375	0.3719
FISH	Sharpness	0.4687	0.4106	0.2784
ARISM	Sharpness	0.2990	0.2192	0.1472
BIBLE	Sharpness	0.1250	0.0802	0.0537
PPPC	PM2.5	0.8115	0.8189	0.6078
Ref. [22]	PM2.5	—	0.7823 *	0.5809 *
Ref. [21]	PM2.5	0.8011 *	—	0.6102 *
Ref. [33]	PM2.5	0.8082	0.8177	0.6115
MSCN-Gray	PM2.5	0.3537	0.3840	0.2597
Pro. (MSCN-Sat)	PM2.5	0.8228	0.8241	0.6100

presents the performance of our proposed PM_2.5 concentration estimation algorithm, and the three types of competing algorithms on AQID. The best performing method among the image contrast estimation algorithms, BIQME, achieves PCC, SCC, and KCC values on AQID of 0.5441, 0.5375, and 0.3719, respectively. The best performing method among the image sharpness estimation algorithms, FISH, obtains PCC, SCC, and KCC values on AQID of 0.4687, 0.4106, and 0.2784, respectively. This indicates that it is difficult to accurately measure PM_2.5 concentration, only focusing on contrast or sharpness. The PCC, SCC, and KCC values obtained by the proposed algorithm on AQID are 0.8228, 0.8241, and 0.6100, respectively. Encouragingly, the proposed method outperforms all competing picture-based PM_2.5 monitoring algorithms on PCC and SCC metrics. On the indicator KCC, the proposed algorithm is also second only to the best-performing algorithm. Moreover, we compare the performance of the MSCN distribution features of the grayscale and saturation channels on the AQID, and the experimental results are shown in Table 3. The PM_2.5 concentration estimation model based on the MSCN distribution features of the grayscale channel (i.e., MSCN-Gray) obtains the 0.3537, 0.3840, and 0.2597 of PCC, SCC, and KCC values, respectively. The experimental results show that the MSCN distribution of the grayscale channel makes it difficult to effectively distinguish the different levels of PM_2.5 concentrations compared to the MSCN distribution of the saturation channel. We train the PM_2.5 concentration estimation models based on the MSCN distributions of the grayscale and saturation channels in the same way. Note that all the learning-based models are retrained on AQID, and the ratio of training, validation, and testing data is 8:1:1.

To observe the algorithm performance more intuitively, we further test the scatter plots of some representative algorithms, including BIQME, Sun et al.’s method, PPPC, and the proposed method, which are shown in Figure 6. The scatter points are closer to the black baseline, and the corresponding PM_2.5 concentration estimates are more accurate. The experimental results in Figure 6 present that our proposed PM_2.5 concentration measurement algorithm is significantly superior to the competing SOTA methods.

4. Conclusions

This paper presents an image perception-based PM_2.5 concentration measurement method. The main contributions of the paper include two points. (a) We design a natural scene statistical prior to evaluate the color richness loss caused by PM_2.5 during imaging. (b) In this paper, a learning method better suitable for PM_2.5 concentration monitoring is used to complete the mapping from the image perception features to PM_2.5 concentration values. Extensive experiments have shown that the proposed image-aware PM_2.5-based monitoring algorithm can effectively estimate PM_2.5 concentrations. We consider designing the image priors from both structural and depth domains in future work to estimate PM_2.5 concentrations more effectively.