A New Method for the Evaluation and Visualization of Air Pollutant Level Predictions

Abstract

Accurately predicting air pollutant levels is very important for mitigating their effects. Prediction models usually fail to predict sudden large increases or decreases in pollutant levels. Conventional measures for the assessment of the performance of air pollutant prediction models provide an overall assessment of model behavior, but do not explicitly address model behavior when large changes are observed. In our work, we propose a method to automatically label the observed large changes. We also propose two visualization methods and two measures that can help assess model performance when sudden large changes in pollutant levels occur. The developed measures enable the assessment of model performance only for large changes (MAE of large changes), or weigh the model residuals by the rate of change (WErr), making the evaluation measures “cost-sensitive”. To show the value of the novel evaluation and visualization methods, we employ them in the evaluation of three empirical examples—different statistical models used in real-life settings and a popular atmospheric dispersion model. The proposed visualizations and measures can be a valuable complement to conventional model assessment measures when the prediction of large changes is as important as (even if they are rare) or more important than predictions of other levels.

1. Introduction

Inhalation of polluted air is a major environmental health risk [1,2,3,4]. Its short- and long-term exposure can lead to severe adverse health effects such as reduced lung function, respiratory infections, asthma, heart disease, and diabetes. This is why most countries have put legislation in place to dictate how air pollutants should be measured, evaluated, and forecasted with regard to specific predefined limit values. In Europe, air pollutant level measurements and their limit values are defined in Directive 2008/50/EC [5]. The directive also dictates that member states have to forecast the exceedances of the limit values. Accurate forecasting of elevated levels of air pollutants can alleviate their impact on human health. EU member states have to predict the exceedances of the daily limit value for PM${}_{10}$, the exceedances of the information threshold for ozone (O${}_3$), and the exceedances of the alert threshold for sulfur dioxide (SO${}_2$) and nitrogen dioxide (NO${}_2$).

Different types of models are used to forecast air pollutant levels, most commonly photochemical atmospheric dispersion models [6,7,8,9,10] and different statistical models. Some statistical models used for the prediction of pollutant levels are described in Taheri S. and Sodoudi [11], Lu and Wang [12], Dutot et al. [13], Faganeli Pucer et al. [14], Sharma et al. [15], Kocijan et al. [16] and de Gennaro et al. [17]. Photochemical atmospheric dispersion models usually produce spatial three-dimensional rasters of pollutant levels as outputs while statistical models mostly produce point predictions. Models are usually validated by comparing their outputs to pollutant measurements from a particular location (point measurements) [18], so both types of models (chemical dispersion and statistical models) are validated as point predictions. There have been several criteria used for the assessment of the performance of air quality models, such as the measures that evaluate the distance between point prediction and measured values e.g., root mean square error (RMS) [13,19,20,21,22,23,24,25,26], mean absolute error (MAE) [13,14,25,26,27], and mean absolute percentage error [28]. Those measures are useful when models predict pollutant levels as continuous values (e.g., regression models). Different indexes that assess model performance as normalized measures have also been used [15] for example the correlation coefficient, Willmott Index [25,29], Nash–Sutcliffe Efficiency [30], and Legates and McCabe Index [31]. Multiple evaluation measures used in air pollutant level forecasting are described in Sharma et al. [15], Kocijan et al. [16], and Carslaw and Ropkins [32]. When the model predicts the exceedance of a predefined threshold (e.g., PM${}_{10}$ daily limit value of 50 $\mathsf{\mu }$$\mathrm{g}$/${\mathrm{m}}^3$) or predicts the range (class) of the pollutant level, the evaluation is performed in terms of accuracy, true positive rate, false positive rate [17,26,33,34], or the success index [13,17]. If the model predicts the probability of exceedance of a predefined threshold, the performance can be evaluated by using the logarithmic score [14].

Measures that assess the mean or squared mean (MSE, MAE) of the residuals estimated as the difference between the observed and modeled values provide an overall assessment of model performance, which is intuitive (probably MAE is the most intuitive measure). When using MSE instead of MAE for the assessment of model performance large residuals provide a greater contribution to the overall error. All the coefficients and indices listed above give a comparison of modeled and measured levels. If we model daily air pollutant levels with the simplest model possible, the “persistence model” (today’s level is the same as yesterday’s), and assess its performance using the previously mentioned measures, the persistence model does not seem that inefficient. For example, if we assess the performance of the persistence model in terms of MAE, in most instances the error would not be very high as sudden large changes in pollutant levels are not very common in Europe [35]. The residuals (of the persistence model) would be small when the change in pollutant level is small from one day (one modelled value) to the next. Still, persistence is not a good model as it is incapable of predicting changes in pollutant levels. All measures listed above cannot assess the performance of the models when we are particularly interested in their ability to predict large pollutant level changes. The conventional measures are not “cost-sensitive”.

For model evaluation, different visualizations are also used. Still, the most popular visualization is the plot depicting the observed values against the modeled values [18,24,26,36]. Different scatter plots, residual plots, and quantile-quantile plots are also frequently used [18,36]. For the visual evaluation of air quality models, the Taylor diagram [37] and the Target plot [38], which enable the visualization of more than one metric in the same plot, or the polar coordinate diagram of the relative prediction error [26], which shows the distribution of the errors for different times of the day, are also used. Recently, some visualizations for feature importance have been proposed [21]. All these visualizations show interesting aspects of model performance but tell nothing about model performance when predicting large changes.

When we were developing models to predict PM${}_{10}$ and O${}_3$ levels with the Slovenian Environment Agency (ARSO) [14], air quality experts emphasized that the critical task when predicting pollutant levels is to correctly predict the onset and offset of an episode of high PM${}_{10}$ or ozone levels. According to ARSO, correctly predicting large increases and decreases is more important than the average model performance. They also wanted measures that would enable them to evaluate model performance during these rare events. This was the main motivation for our work.

High PM${}_{10}$ levels in Slovenia occur in winter when the meteorological situation is stable with an unfavorable dispersion situation (temperature inversion and low wind conditions) [39]. This is usually known as air stagnation, which is characterized by stable weather, low wind in the lower atmosphere, and no precipitation [27,40,41]. Even emissions are affected by the meteorological situation, e.g., a sudden decrease in temperature affects the amount of indoor heating and discourages people from biking or walking to work, which increases the amount of traffic. Such a decrease in temperature accompanied by stable, low wind, low-temperature weather in Slovenia marks the beginning of an episode of high PM${}_{10}$ levels. When such an episode is forecasted, precautions can be taken to limit PM emissions (e.g., limit traffic or discourage people from burning wood). Similarly, high ozone levels are not only induced by the share of ozone precursors and solar radiation, but also by air stagnation [40]. High ozone levels are common in summer when solar radiation is high and hours of daylight are long [42]. If high ozone levels are forecasted for Slovenia, people are advised not to exercise outdoors and to avoid going to the mountains. As such measures affect people’s lives it is also important to carefully predict the end of such episodes, so the implemented measures can cease. A similar situation occurs in all of central Europe. This is the main reason that air quality experts want their models to be able to predict large increases and decreases.

In our work, we present two new measures for the evaluation of prediction model performance adapted for the evaluation of large pollutant level changes. We also propose two new visualizations for the visual evaluation of model performance in these extreme situations.

The remainder of the paper is organized as follows. In Section 2 we define the pollutant level changes and discuss what constitutes large changes, we propose new visualizations and measures that evaluate large changes. In Section 3 we present three examples of the applicability of the presented visualizations and measures. We conclude the paper with Section 4.

2. The Visualization and Evaluation of Large Changes

In this section, we define changes in pollutant levels (see Section 2.1), large pollutant level changes (see Section 2.2) and propose two new visualizations (see Section 2.3) and two new measures (see Section 2.4) for the evaluation of air quality models. Figure 1 represents the steps taken to get to the final model evaluation.

2.1. Defining Pollutant Level Changes

We denote the change in pollutant level as $\Delta$C(x,$\Delta$ T), where $C(x,T)$ denotes the observed value of an investigated pollutant at a certain location (x) at a certain time and $C(x,T+t)$ the same pollutant t time in advance at the same location (x). $Cpred(x,T)$ denotes the level predicted by a certain model for the same location as $C(x,T)$. As all observations and predictions are made for the same location x, we will discard x from the equations and write the changes in pollutant level as $\Delta$C($\Delta$t), which is defined in Equation (1). As an example, we can evaluate a statistical model that predicts today’s PM${}_{10}$ levels in Ljubljana and we want to assess if the model correctly predicted how much the daily average PM${}_{10}$ level will increase today from the level that was observed yesterday, so we define:

$$\Delta C(\Delta t)=C(T+t)-C\left(T\right).$$

(1)

$C\left(T\right)$ is the last available observation before we run the model, e.g., if we are predicting the PM${}_{10}$ level for tomorrow and the last available PM level for today is at midnight. $\Delta$C($\Delta$t) denotes the difference in observed pollutant level (measured values) at time instant T and time instant $T+t$.

We also define the predicted change in pollutant level $\Delta$Cpred($\Delta$t):

$$\Delta Cpred(\Delta t)=Cpred(T+t)-C\left(T\right).$$

(2)

Cpred$(T+t)$ is the value predicted by the model that is located at t time instants in the future. $\Delta$C($\Delta$t) describes how much the pollutant level changed from one observation to the next (the true increase) while $\Delta$Cpred($\Delta$t) describes the change assessed by the model.

2.1.1. Defining the $\Delta$Cpred($\Delta$t) for Photochemical Atmospheric Dispersion Models

When assessing the performance of statistical models, we define $\Delta$Cpred($\Delta$t) (described in Section 2.1) as the difference between the predicted value and the last observed value. If we are dealing with photochemical atmospheric dispersion models (e.g., model EMEP MSC-W [6] model, see Section 3.3), the observed value is not available or at least is not taken into account by the model. Photochemical dispersion models are usually run without past measurements as inputs.

If we want to evaluate such models using the new measures, we must define $\Delta$Cpred($\Delta$t) differently than with statistical models. We propose defining $\Delta$Cpred($\Delta$t) as the difference between the predicted level at time T and the predicted level at time $T+t$. This shows how the model is capable of predicting large changes in pollutant levels disregarding the actual predicted levels. We speculate that sometimes photochemical dispersion models are capable of predicting large changes correctly, but the produced value (C(T)) is not correct due to some inherent biases.

2.2. Defining Large Pollutant Level Changes

Intuitively, a large change is a change in pollutant level between time T and $T+t$ that is much larger than the average change in pollutant levels in this time interval. Defining what a large change is can be very ambiguous. It could be defined by a field expert or could be defined statistically. For example, an expert on air pollution can quickly identify sudden large increases or decreases in pollutant levels at a certain location, while people who do not deal with these issues cannot spot these changes. This is why we propose a method based on the statistical properties of pollutant level changes in a certain time interval.

We propose modeling the differences $\Delta$C($\Delta$t) with the t-distribution [43], which is very similar to the normal distribution, but it can accommodate heavier tails. The t-distribution is a symmetric and bell-shaped distribution. It is a class of distributions, not one distribution. When fitting a t-distribution, we have to specify the degrees of freedom. The larger the number of degrees of freedom, the more similar to the normal distribution it becomes (when the t-distribution has infinite degrees of freedom it becomes the normal distribution). At lower degrees of freedom, it can model heavier tails than the normal distribution [44]. The t-distribution is usually assumed to be centered at 0 and is not scaled. In our case, we use a t-distribution with a location and a scale parameter (metRology R package). The location parameter enables the distribution not to be centered at 0 and the scale parameter enables the adaptation of the width of the distribution. To fit the distributions, we use the fitdistr function of the ‘fitdistrplus’ [45] R [46] package.

Figure 2 shows the fit of a normal and t-distribution to $\Delta$C($\Delta$t) values from Ljubljana, Slovenia. The shape of the fitted t-distribution gives a better fit than the normal distribution. Using the fitted distribution we can evaluate the limits of the largest $\Delta$C($\Delta$t) values (e.g., the largest 5%, 10% level changes).

2.3. Visualizations of $\Delta$C($\Delta$t) for the Assessment of Model Performance

To assess regression model performance, we usually examine different plots. The preferred plots are those showing the predicted values against the actual values or different residual plots. To better understand how the models perform with large changes, we propose two additional visualizations. The visualizations do not focus on the predicted values but on the predicted change.

Figure 3 shows the residuals of the model against the observed values ($\Delta$C($\Delta$t)). The figure shows the predictions against the observed change in PM${}_{10}$ levels in Ljubljana in the period from 2014 to 2016. In this example, a random forest model [47] was trained as described in [14] (this is a re-analysis of those models) for predicting the daily mean PM${}_{10}$ levels in the morning of the current day. The time series of observed and predicted values of different models for Ljubljana are shown in the Appendix B. The vertical lines in the plot show the limits of the 95% and 90% intervals (the intervals containing those percentages of data as described in Section 2.2). The large changes are represented by points lying outside the selected interval (on the right of the upper limit and the left of the lower limit of the interval). The width of the interval is arbitrary, defined by the individual performing the analysis (whether 95% or 90% or even 85%, 80% of the largest changes are observed). In these examples, the large change was defined statistically. For smaller changes, the residuals are distributed evenly around the x-axis, but for large changes, they are not. The plot shows that the observed model is prone to underestimate large increases (on the right values lie above the x-axis) and large decreases (on the left values lie under the x-axis), especially where the largest changes are observed (95% interval). We cannot spot such model behavior by examining an ordinary residual plot.

Figure 4 shows the actual change $\Delta$C($\Delta$t) against the predicted change $\Delta$Cpred($\Delta$t). The figure shows the same predictions for Ljubljana as the first visualization. The dashed vertical lines represent the limits of the 95% interval. The diagonal (y = x) shows the location of the “perfect” predictions. The closer the points lie to the line, the more accurate the predicted $\Delta$C($\Delta$t) are. The greatest mistake a model can make is to wrongly predict the direction of the change when large changes occurred in reality; that is, predicting an increase instead of a large decrease or a decrease instead of a large increase. So all points on the left of the y-axis (most importantly on the left of the left dashed line) should be lying below the x-axis and the points on the right of the y-axis (most importantly on the right of the right dashed line) should be lying below the x-axis. The further the points lie from the y-axis the more important it is that they lie on the correct side of the x-axis. In this regard, the example model from Figure 4 performs well; there is only one point lying on the left side of the left dashed line and above the x-axis. From Figure 4 we can also see that the model mostly underestimates large changes.

The visualization shown in Figure 3 helps evaluate how the residuals relate to observed changes in pollutant levels if there is some kind of expected behavior associated with the amount of the observed change, e.g., in our case the model underestimates almost all large changes. The visualization from Figure 4 helps evaluate the relationship between the observed changes and the predicted changes. The greatest mistake air quality models can make is to predict a decrease instead of a large increase or vice versa and, by looking at Figure 4, it is easy to spot such instances.

2.4. Performance Measures

As discussed in the Introduction section, numeric measures for the evaluation of model performance usually evaluate the overall (unweighted) performance of the model or the classification performance (accuracy, sensitivity, specificity, etc.) when a threshold is predefined. To evaluate the performance of our models when the prediction of large changes is crucial, we propose two new performance measures.

The first proposed evaluation measure based on the proposed visualization shown in Figure 3 is to calculate the MAE or the MSE only for the data lying outside of the predefined intervals. Figure 5 shows MAE with two standard errors for different shares of largest changes taken into account (from the upper 5% of the largest to the 20% of the largest $\Delta$C($\Delta$t)) we can compare it to the same measure calculated for all values (see Figure 5). In our example (see Figure 5), MAE increases when we narrow the width of the observed intervals. MAE is the largest when only the most extreme changes are observed.

To assess the behavior of all predictions at once, not splitting them as we did with the first proposed measure, we propose to weight the large changes $\Delta$C($\Delta$t) as more important than smaller changes. We propose to weight each absolute error (residual) with the associated absolute value of $\Delta$C($\Delta$t) and divide it by the average absolute value of $\Delta$C($\Delta$t). We will denote the weighted error as WErr and calculate it as:

$$WErr=\frac{|\Delta C(\Delta t\left)\right||Cpred-C|}{|\overline{\Delta C(\Delta t)|}}.$$

(3)

This way, the residuals of the models predicting large changes are weighed more than residuals of smaller changes.

Both presented measures focus on large changes. The first one only evaluates the residuals for the chosen percentage of largest changes and ignores all others. The second measure tries to evaluate all model residuals at the same time but weighs the residuals for large changes more, and the residuals for smaller changes less.

Figure 6 shows a comparison of different models used for PM${}_{10}$ prediction in Ljubljana in the period from 2014 to 2016 based on $WErr$. The random forest model shows a superior performance over other models in terms of $WErr$. Still, the confidence intervals of the RF and GP models show a great overlap.

3. How Can the New Visualizations and Measures Improve the Evaluation of Our Models?

In this section, we show three examples of model evaluation using the methodology described above. We do not train and test new models but render a new analysis of existing models. The first two examples comprise the comparison of different statistical models for PM${}_{10}$ and ozone forecasting for the current and following day (a re-analysis of the results presented in [14]); the third example shows a case of a comparison of the performance of a statistical model with a photochemical dispersion model.

3.1. Evaluation of Different Models for PM${}_{10}$ Prediction in Nova Gorica, Slovenia

Nova Gorica is a town in western Slovenia with around thirty thousand inhabitants. It is located in the Mediterranean part of Slovenia close to the Italian border. PM${}_{10}$ levels are higher in winter than in summer, but they are usually quite low. In winter, PM${}_{10}$ levels are affected by increased emissions from indoor heating and an unfavorable dispersion situation. Another contributing factor is the long-range pollution from heavily industrialized and densely populated northern Italy, which has a notable effect on this part of Slovenia [39]. Meteorological conditions (especially the changes in the dispersion situation) can induce sudden increases or decreases in pollutant levels. The following example shows the comparison in performance for different models predicting PM${}_{10}$ levels (see [14]) for the current day (morning predictions for the current day).

Figure 7 shows the comparison of the performances of the lasso (Lasso) [48], Bayesian lasso (BLasso) [49], random forest (RF) [47] and Gaussian process (GP) [50] models in Nova Gorica tested on data from 2014 to 2016. The models are briefly described in Appendix A. Figure 7 top left shows that in terms of MAE all four models perform equivalently, while, in terms of WErr, the GP model outperforms all other models. When observing MAE for different percentages of highest $\Delta$ C(1 day) values (as described in Section 2.4) the GP model outperforms other models in the prediction of the largest changes (see Figure 7 top right). If we only judge the performance of the models with conventional measures we can conclude that all models perform equivalently. When we explore model performance for large changes, we can see the clear advantage of using the GP model. If we want our model to predict large changes well, the GP model is the best model and it clearly outperforms the RF model.

3.2. Evaluation of Different Models for Ozone Prediction in Koper, Slovenia

Koper is a town on the Slovenian coast. It is home to about thirty thousand inhabitants and lies across the bay from Trieste (Italy), which is a much larger city. The climate in Koper is Mediterranean with hot, dry summers and mild winters. Long sunny, warm days accompanied by local emissions and emissions from across the bay of Trieste and neighboring Italian regions contribute to high ozone levels [39,51]; one of the highest in Slovenia (apart from high altitude measurement sites).

Figure 8 shows the comparison of the Lasso, BLasso, RF, and GP models for the prediction of the daily maximal values of ozone in Koper (tested on 2014–2016 data). On average, according to MAE, the best performing model is GP (see Figure 8 up left), but when observing WErr the GP model does not stand out much (see Figure 8 upper figure on the right). According to WErr, all observed models perform similarly although the GP model exhibits the best performance. Overall RF and GP models show a similar performance when predicting large changes; they both outperform the two linear models, which is not evident when only observing the MAE for all instances. By using our evaluation measures, we can conclude that when it is crucial to carefully predict large changes the non-linear models (RF or GP models) are the best models for predicting ozone in Koper.

3.3. Comparison of the EMEP MSC-W Model with Statistical Models for the Prediction of PM${}_{10}$ Levels

The same evaluation methodology can also be applied to photochemical dispersion models (they are briefly described in Appendix A). As an example, we compare the results of the best performing statistical model for the prediction of PM${}_{10}$ levels for the next day ($\Delta$t = 2 days) in Ljubljana (GP model) to the predictions of the EMEP MSC-W model [6]. We compare both predictions as point predictions for the same location (modeling point) where the measurements were performed. We selected the EMEP MSC-W model because the results are available online and it is a popular photochemical dispersion model used for the assessment of long-range transport pollution in Europe. The presented example is not intended as a strict comparison of statistical and photochemical dispersion models, nor as a critique of a model type. This is simply a case study on how we can compare the performance of two different types of models as point predictions using the presented methodology. The $\Delta$C(2 days) are calculated as described in Section 2.1.1.

Figure 9 shows similar visualizations to the ones we observed when comparing different statistical models. If we examine the top plots (MAE and WErr) we can conclude that the GP model is a much better performing model than the EMEP model. When analyzing only large changes, we see that as we restrict the observed percentage of the largest changes, the difference between the GP and EMEP models becomes smaller. For only extreme changes, the EMEP model outperforms the GP model. Still, this outperformance of the EMEP model for large changes is not evident in the WErr plot (Figure 9) in the top left. This leads us to question, how is this possible? To further analyze this inconsistency, we visualize our results as proposed in Section 2.3. The top of Figure 10 shows the plots of the residuals against $\Delta$C(2 days) for the two observed models. The changes predicted by the EMEP model sometimes exhibit very high deviation from the observed changes (even when the changes in pollutant levels are small), but in some instances the difference between the actual and predicted changes for very large changes is small. The EMEP model does not underestimate large decreases as consistently as the GP model, but similar to the GP model, generally underestimates large increases. Figure 10 shows $\Delta$C(2 days) against $\Delta$Cpred(2 days) for the observed models. As in the previous figure, the predictions of the GP model show smaller divergence (in this figure this means less scatter around the y = x line). The EMEP MSC-W model contrary to the GP model often predicts an increase instead of a large decrease (points lying on the left of the left dashed line and above the x-axis). It also predicts more increases instead of large decreases as compared to the GP model. By looking at those plots, we can better compare the performance of both models and understand why the MAE for extreme changes (the largest 5%) is smaller for the EMEP model, but when observing WErr the GP model clearly outperforms the EMEP model. Some EMEP model residuals are large when small changes in pollutant levels occur and even if they are weighed with much smaller weights than large changes, they increase WErr substantially. By observing the plots in Figure 9 the performance of the GP model looks much more coherent than that of the EMEP model, especially when observing the bottom plots where the points lie much closer to the $y=x$ line for the GP model.

Usual Assessment of Model Performance

We usually assess the performance of the models with the help of plots showing the measured PM${}_{10}$ levels (results of different models) against the levels predicted by our models (see Figure 11). These plots can help us with the usual model assessment. In Figure 11 on the right we can observe the performance of the EMEP model. The predicted levels are further from the true levels as PM${}_{10}$ levels get higher. The spread around the $y=x$ line is quite large. In Figure 11 left, we can observe the performance of the GP model for the same location (Ljubljana). Here the spread around the $y=x$ line is smaller as with the EMEP model, but it still increases with increasing PM${}_{10}$ levels. The GP model generally underestimates large PM${}_{10}$ levels. From these two plots, it is impossible to say anything about model predictions of different level changes.

4. Conclusions

In our article, we have presented new and “cost-sensitive” measures for the performance evaluation of prediction models used for the prediction of air pollutant levels (point predictions). We have also presented two new visualizations. Our first method is based on the estimation of large pollutant level changes that occur in the time interval $\Delta$t. We use the identified large changes to evaluate the performance of prediction models only in the event large changes were observed (MAE for different percentages of largest changes). The second evaluation metric weights the model residuals with the absolute value of the observed change (WErr). We have also proposed two new visualizations, where the predicted values or model residuals are plotted against the change in pollutant level, which allows for a graphic evaluation of model performance related to the rate of the observed change in $\Delta$t. As shown in Section 3, the proposed visualizations and measures can better provide an understanding of model performance. The reanalysis of existing models shows that models usually underestimate large changes.

When examining model performance with our measures and visualizations, we showed that, when only large changes are important, the evaluation should be performed with the first measure—MAE for the selected percentage of largest changes. When large changes are as important as any other prediction, conventional measures should be used, such as MAE or MSE. When all results are important but predictions of large changes are proportionally more important, the WErr measure demonstrates a good trade-off between our first measure and conventional measures. The same measures could be used for the evaluation of meteorological models or other environmental models.

The presented measures and visualizations are useful only when dealing with models that predict some temporal variable (that changes with time) and we have a strictly defined time step; e.g., one day predictions are made for the next day. The measures are useful when large changes are more important than usual changes and are not frequent.

As properly predicting large changes is very important for relevant institutions to fulfill the requirements dictated by the European Directive 2008/50/EC [5] and to implement adequate measures, the proposed evaluation measures and visualizations of model predictions should provide further insight into model performance. In future work, we will employ the new measures and visualizations alongside the ones generally used and use them to better evaluate our air quality models.