Nonlinear Bias Correction of the FY-4A AGRI Infrared Radiance Data Based on the Random Forest

Abstract

Bias correction is a key prerequisite for radiance data assimilation. Directly assimilating the radiance observations generally involves large systematic biases affecting the numerical prediction accuracy. In this study, a nonlinear bias correction scheme with Random Forest (RF) technology is firstly proposed based on the Fengyun-4A (FY-4A) Advanced Geosynchronous Radiation Imager (AGRI) channels 9–10 observations in the Weather Research and Forecasting Data Assimilation (WRFDA) system. Two different settings of the predictors are additionally designed and evaluated based on the performance of the RF model. It seems that an apparent scene temperature-dependent bias could be effectively resolved by the RF scheme when applying the RF method with newly added predictors. Results suggest that the proposed nonlinear scheme of RF performs better than the linear scheme does in terms of reducing the systematic biases. A more idealized error distribution of observation minus background (OMB) is found in the RF-based experiments that measure the nonlinear relationship between the OMB biases and the predictors when using the Gaussian distribution as the reference. Furthermore, the RF scheme shows a consistent improvement in bias correction with the potential to ameliorate the atmospheric variables of analyses.

1. Introduction

Directly assimilating the geostationary imager observations has been demonstrated as a practical approach to enhancing the numerical prediction accuracy [1,2]. However, the systematic biases between the observation (O) and the background (B) exist in the assimilation of the uncorrected radiance data, which are always included in the observation minus background (OMB) residuals [3]. The sources of systematic errors with large uncertainties originate from errors in the numerical models, biases in the observations, and the bias caused by the observation operator. Most data assimilation methods are able to account for the random error by a combination of spectral, temporal, and spatial filtering rather than for the systematic bias. Thus, the bias correction (BC) procedure has become an indispensable part of most assimilation systems in the numerical weather prediction (NWP) [4]. As a fundamental requirement, BC schemes are generally designed to remove systematic biases between the observation and the model-simulated background [5,6].

The differences between observations and their NWP background equivalents were often quantitated for the ‘offline’ BC method to remove systematic biases [7]. Considering the sources of systematic biases are statistically changing with the atmosphere- and satellite-related parameters, researchers generally applied a linear combination of several related BC predictors to correct the scan and air-mass dependent biases, for example, for the TIROS Operational Vertical Sounder (TOVS) observations and for the Advanced TOVS (ATOVS) of NOAA observations, respectively [8,9]. Meanwhile, the air-mass dependent biases may be attributed to the design of the observation operators. For this reason, Watts and McNally (2004) corrected the absorption coefficient error of each channel in radiative transfer modeling (RTM) for the Atmospheric Infrared Sounder (AIRS) [10]. However, those offline schemes are not available to continuously track the systematic bias for radiance observations that change in time and space. Thus, an adaptive BC method called variational bias correction (VarBC) is widely adopted in most data assimilation systems [11]. The VarBC method is designed to correct the systematic bias adaptively by including BC coefficients of predictors as control variables in the cost function for iterative minimizations [12,13,14]. Auligné et al. (2007) proved that the VarBC scheme can be considered as an optimization result of the static and the offline adaptive schemes [15]. On the other hand, following the assumption that BC should be independent of the data assimilation system, the online BC with an ensemble Kalman filter was applied for the state-space-dependent biases correction for the 15 channels of AIRS in the primitive equation dynamics (SPEEDY) model with simplified parameterizations [16]. Furthermore, an innovative method called “Constrained Variational Bias Correction” is proposed by Han et al. (2016) to avoid unrealistic drifts from the full global 4D-Var system models [17]. Though previous studies have shown that the VarBC-based schemes are able to effectively remove systematic errors during the clear-sky data assimilation experiments, it was proved that the VarBC methods based on a linear combination of predictors bring few positive effects in the assimilation of cloudy pixels [18]. The lack of improvement may have occurred because the cloud-related biases were too complex and nonlinear to be removed using a linear combination of BC predictors [19].

The above static and online BC methods typically assume a linear correlation between the OMB departures and some selected predictors. For satellite data assimilations, nonlinear sources of biases commonly exist, especially in the all-sky assimilation experiments [20,21]. Therefore, Otkin et al. (2018) implemented a nonlinear correction scheme with a Taylor series polynomial expansion, which removed the linear and nonlinear conditional biases from the Spinning Enhanced Visible and Infrared Imager (SEVIRI) all-sky radiances [22]. Their results also suggested the benefits of including scene brightness temperature as a BC predictor. Nowadays, machine learning technologies have been widely applied to the nonlinear issues in NWP [23,24]. Wang et al. (2021) proposed a machine learning-based correction scheme with the Random Forest (RF) technique for the Geostationary Interferometric Infrared Sounder (GIIRS) observations, proving that the RF scheme with the added predictors of longitude and latitude outperforms the traditional offline method [25]. A brightness temperature remapping technique based on a deep convolutional neural network (CNN) demonstrated the added benefits to the decreased representativeness error in FengYun-3D (FY-3D) microwave radiation imager (MWRI) observations assimilation [26].

As one of the advancements of Fengyun-4A (FY-4A), the AGRI (Advanced Geosynchronous Radiation Imager) is attracting attention due to its high spectral, temporal, and spatial resolutions compared to its previous generation (Fengyun-2) [27]. Assimilating the two water vapor channels 9–10 of AGRI has been proven to benefit the initial value of the model and improve the prediction accuracy of “21·7” Henan extremely persistent heavy rainfall [28]. Most efforts have been made focusing on the bias characteristics of each channel for AGRI. For instance, it was verified that the biases of AGRI channel 9 and channel 10 are smaller than the biases of AGRI infrared window channels [29,30]. Geng et al. (2020) proved that the BC predictors of satellite zenith angles (satzen) have less effect on AGRI radiance bias correction based on the VarBC method [31]. Similarly, it is noted that the OMB variance of AGRI has less dependence on satellite zenith angles on the basis of linear regression tests but has stronger dependence on the scene brightness temperature [32]. A common issue of these scene-dependent biases is also noticed by Zou et al. (2016) in the Advanced Himawari Imager Infrared (AHI), owing to the nonlinearity of instrument calibration [33]. In addition, for the GIIRS observations from the FY-4A satellite, a scanpos-dependent bias is observed [34].

Although the above productive achievements have been made in terms of the application of the FY-4A AGRI radiance data, there has been few explorations in comparisons of the non-linearity and the linearity BC methods for the geostationary infrared observations assimilation. Meanwhile, the traditional BC schemes based on a linear combination of predictors may not be able to remove those systematic biases caused by the nonlinear processes. Thus, a nonlinear BC scheme is developed based on the machine learning technology of RF against the linear BC scheme by utilizing the observations of AGRI channels 9–10 under clear-sky in the Weather Research and Forecasting Data Assimilation (WRFDA) system. The proposed BC scheme is performed based on a nonlinear combination of predictors as well as the nonlinear fitting relationship between the predictors and OMB biases. Moreover, the quantification and the contribution of several BC predictors are investigated through sensitivity experiments. This study aims to apply a nonlinear correction method of OMB biases for FY-4A AGRI radiance data assimilation. The potential positive effect of the RF scheme is checked to enhance the accuracy of data assimilation. It may be conducive to improving the numerical weather forecasting through this nonlinear BC method in data assimilation systems.

The outline of this study is organized as follows. Detailed descriptions of two different BC schemes are provided in Section 2 for FY-4A AGRI radiances assimilation. The experimental design is described in Section 3. Section 4 gives the results and analysis before summarized conclusions and a plan of future work is elaborated in Section 5.

2. Materials and Methods

2.1. The Data Assimilation System

In order to obtain positive impacts in NWP from the satellite radiances, different BC methods are coupled into the WRF three-dimensional variational data assimilation system (WRF-3DVar), which minimizes the so-called variational cost function $\mathit{J}\left(\mathit{x}\right)$ as Equation (1) [35].

$$\mathit{J}\left(\mathit{x}\right)=\frac{1}{2}{\left(\mathit{x}-{\mathit{x}}_b\right)}^{\mathrm{T}}{B}^{-1}\left(\mathit{x}-{\mathit{x}}_b\right)+\frac{1}{2}{\left[{\mathit{y}}_o-\mathit{H}\left(\mathit{x}\right)+\mathrm{O}\mathrm{M}{\mathrm{B}}_{\mathrm{b}\mathrm{i}\mathrm{a}\mathrm{s}}\right]}^{\mathrm{T}}{R}^{-1}\left[{\mathit{y}}_o-\mathit{H}\left(\mathit{x}\right)+\mathrm{O}\mathrm{M}{\mathrm{B}}_{\mathrm{b}\mathrm{i}\mathrm{a}\mathrm{s}}\right]$$

(1)

where $\mathit{x}$ is the atmospheric state vectors, ${\mathit{x}}_b$ is the first guess (background) states, and ${\mathit{y}}_o$ the observation vector. $\mathbf{R}$ and $\mathbf{B}$ are the error covariance matrices for the observations and background, respectively. $\mathit{H}$ stands for the observation operator by using the Radiative Transfer for TIROS Operational Vertical Sounder (RTTOV). It is noted that the observation minus the observation operator is an essential component of the variational cost function. Therefore, it is necessary to recognize the systematic bias prior to data assimilation (Dee and Uppala, 2009). Usually, the systematic bias here denoted as ${\mathrm{O}\mathrm{M}\mathrm{B}}_{\mathrm{bias}}$ could be calculated using the OMB statistics before correcting the $\mathit{H}(x)$ [36]. In this study, the BC procedures are conducted with the linear and nonlinear schemes, respectively.

2.2. Linear Bias Correction

In the linear BC scheme, the ${\mathrm{O}\mathrm{M}\mathrm{B}}_{\mathrm{bias}}$ for each channel are removed through two terms as

$$\mathrm{O}\mathrm{M}{\mathrm{B}}_{\mathrm{b}\mathrm{i}\mathrm{a}\mathrm{s}}={\beta }_0+{\displaystyle {\sum }_{i=1}^{{\mathit{N}}_p}{\beta }_i}{p}_i(\mathit{x}),$$

(2)

where ${\beta }_0$ represents the constant component of the total bias, a linear combination of i^th state-dependent predictors $p_i$ here account for the weather condition and the characteristics of AGRI observations, and i denotes the number of used predictors. Two correction coefficients of ${\beta }_i$ and ${\beta }_{{}_0}$ are computed with the least squares fitting in the linear experiments. In addition, the BC coefficients are assumed to be channel-dependent [37].

2.3. Nonlinear Bias Correction

A bagging-based ensemble learning algorithm called “Random forest” was chosen to construct the nonlinear BC process for the following advantages. RF technology is designed to be efficient and adaptable to high-dimensional datasets. It has better performance in the aspect of generalization ability [38].

The detailed flowchart of the RF-based nonlinear BC scheme is described in Figure 1. First of all, in the RF training model, the statistics of normalized predictors $p_{train}$ and OMB for AGRI channels 9–10 were extracted as the feature variables and the target variables, respectively. The last two-week samples were randomly divided into a training set and a validating set with 80 and 20 percent ratios, respectively. To improve the robustness of the RF model, the RF training model first fits the nonlinear relationship between the OMB and predictors based on the training sets. Then, the outputs of the OMB_RFtrain are calculated by the RF training model based on the information of predictors from the other validation sets. The outputs of the OMB_RFtrain represent the bias estimates of each pixel. Afterwards, two basic parameters (“number of trees” and “depth of trees”) in the RF model need to be adjusted at each node. Through two metrics of MAE and RMSE, the RF model was constantly adjusted to obtain the optimized parameters. Another unique advantage of RF is that the influence of each predictor on the model performance, termed as ‘feature importance’, can be quantified by Gini index [39]. The selection of predictors could be adjusted according to the feature importance scores.

Secondly, once the model optimal parameters have been confirmed, the nonlinear relationship between the OMB departures and several predictors fitted by the RF training model will be inherited to the RF predicting model. Meanwhile, the predictors from testing datasets valid at the analysis time will be extracted as inputs to the RF prediction model. Then, the outputs of the RF prediction model denoted as OMB_RFpred in Figure 1 are applied as the estimates of the systematic bias for the radiance data assimilation in the WRFDA system. The final prediction of OMB_RFpred is made based on the mean of all predictions from each individual tree on test samples $\mathit{x}\prime$, as,

$$\mathrm{O}\mathrm{M}\mathrm{B}\_\mathrm{R}\mathrm{F}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}=\frac{1}{T}{\displaystyle \sum _{t=1}^{T}RF( {\mathit{x}}^{\prime })},$$

(3)

where T is the total number of regression trees.

2.4. Experimental Design

Three parallel experiments are designed to evaluate the impact of different BC schemes for AGRI channels 9–10 under clear-sky. These experiments differ in predictors and BC algorithms with their detailed configurations shown in

Table 1. The setting of three experiments.

Experiment	Predictors Used	Bias Correction Method
RF_pre4	p0: 1 (constant) p1: 1000–500 hPa thickness p2: 300–50 hPa thickness p3: Skin temperature p4: Total column water	Random forest
RF_pre7	p0: 1 (constant)	Random forest
Linear_pre7	p1: 1000–500 hPa thickness p2: 300–50 hPa thickness p3: Skin temperature p4: Total column water p5: Longitude p6: Scene brightness temperature p7: Scan position	Least square fitting

. The RF_pre4 and RF_pre7 experiments are configured consistently with the same nonlinear correction method of RF but differ in the settings of BC predictors. Unlike the other experiments, Linear_pre7 uses a linear solver of the least square fitting to correct the OMB biases. Apart from the traditional predictors p₀–p₄, the predictors p₅—longitude, p₆—scene brightness temperature, and p₇—scan position (scan pixels number in the latitudinal direction) are additionally used for the RF_pre7 and Linear_pre7 experiments. The three newly added predictors p₅–p₇ are more sensitive to the OMB biases as shown in Section 3.2.1. Predictors p₁–p₄ generally depend on the atmospheric state, while predictors p₅–p₇ are to allow for representing the bias characteristics of the satellite observations.

For all the experiments, the statistical correlations based on both the linear and nonlinear methods are obtained on a representative set of OMBs and predictors over the domain. The local area concerned is covering from the west of China to the western Pacific Ocean (15°N–55°N, 90°E–140°E) centered at 35°N, 115°E, where the “21·7” Henan extremely heavy rainfall occurred. The offline statistics are generated by the WRF-3DVar system based on the datasets of National Centers for Environmental Prediction (NCEP) operational 0.25° × 0.25° Global Forecast System (GFS) analysis and the full disk FY-4A AGRI observations. The training sets are available from 1800 UTC 4 July 2021 to 1800 UTC 18 July 2021, every 6 h. As for the RF predicting model, the testing datasets are valid at 0600 UTC 19 July 2021. The background is generated from a 6-h spin-up forecast as the cold starts.

Successful radiance assimilation requires high-quality datasets for the RF model and the WRFDA system. The following four steps of quality control (QC) are adopted in all the experiments, including (1) rejecting all channels with mixture surface types of observation data; (2) rejecting the satellite zenith angels are larger than 60°; (3) rejecting the observations if the absolute OMB innovation exceeds 3 times the observation error under clear-sky conditions, and if it exceeds 15 K; (4) for the clear-sky radiance data assimilation, rejecting the cloudy pixels with a Particle Filter (PF) cloud detection scheme proposed by Xu et al. (2016) [40].

3. Results

3.1. Predictive Capability of RF Model

The optimal parameters can be confirmed based on the error metrics of out-of-bag data (OOBD) without cross-validations, which is a highly efficient way to assess the model generalization capability for the operational forecast. In each round of bootstraping, about one-third of the training sets called “out-of-bag data” are left out and averaged as an out-of-bag score (OOBS) when the “bootstrap” is set to true. The details of model optimization are described as following.

To optimize the RF model, the model predictive capability is evaluated by adjusting the model configurations in terms of parameters and features. Figure 2 shows the OOBS based on the different model parameters (number and depth of trees). A larger OOBS indicates better predictive capability. As shown in Figure 2a, the OOBS for channel 9 is significantly increased until the depth reaches 25, when the number of trees is 130. Moreover, the trees numbers from 180 to 220 obtain a higher score when the depth of trees is smallest. In Figure 2b, a relatively higher score is found after the depth reaches 30 for the number 170. Accordingly, we set 180 trees and a maximum depth of 25 for channel 9 and 170 trees and 30 depth for channel 10, respectively, to avoid the over-fitting issue. Based on the optimal parameters, the predictors in the first analysis valid at 0600 UTC 19 July 2021 are formed as the new inputs, to obtain the OMB_RFpred through the RF predicting model.

As a premise, it is of great importance to diagnose the contributions of each predictor, whether in the proposed scheme or in the traditional BC schemes. Figure 3 illustrates the feature importance of several predictors for AGRI channel 9 and channel 10, respectively, in the RF training model. The model normalized the feature importance so that the sum of all importance scores is 1. The higher the score, the greater the correlation between predictors and OMB biases. It is evident that the predictors of scene brightness temperature, scan position, and longitude contribute more to the RF model compared to other traditional predictors. Moreover, the importance score of the scene brightness temperature is significantly higher than the other predictors for both AGRI channels 9 and 10.

To evaluate the predictive capability of the RF model, the differences between the real OMB and the predicted OMB from validation sets are measured using the mean absolute error (MAE) and root mean squared error (RMSE) for two experiments with two different bundles of predictors. One bundle consists of predictors p₀–p₄ and the other includes predictors p₀–p₇ for BC. As is shown in

Table 2. The RF model prediction scores of two different experiments.

Experiments	Channels	MAE (K)	RMSE (K)	Data Counts
RF_pre4	9	1.72	2.17	101,845
	10	1.48	1.91	99,845
RF_pre7	9	0.93	1.23	101,845
	10	1.02	1.35	99,845

, the MAE and RMSE scores of RF_pre7 for AGRI channels 9–10 are significantly lower than those of RF_pre4. The results indicate a strong generalization capability of the RF model based on the extended predictors. The statistical dependence of OMB on the newly added three predictors p₅–p₇ will be further investigated in the next sections for a better understanding of the model performance. It is worth noting that the error scores for channels 9–10 are rather different. This is probably due to the fact that the statistics are calculated based on the datasets with different data counts. To be specific, the data of channel 10 are more likely to be eliminated by the quality control processes due to the influence of the surface emissivity.

3.2. Comparison of Different BC Schemes

The comparison of three experiments is elaborated in this section to assess the performance of linear and nonlinear BC schemes in the assimilation of the observations of AGRI channels 9–10. Meanwhile, the contribution of those additional predictors p₅–p₇ to the BC effectiveness will be further evaluated by adding the sensitivity experiments in the framework of the same BC method.

3.2.1. Variations in Bias with Predictors

The OMB biases are characterized with the nonlinearity variations along with several predictors. As seen from Figure 4, the biases of OMB noticeably increase as the p₅ scene brightness temperature (scene temperature) increases in all the experiments except for the RF_pre7, especially when the scene temperature is less than 238 K (250 K) for channel 9 (channel 10). The OMB_noBC results without BC have a large amplitude of more than 6 K for both AGRI channels. An apparent scene temperature-dependence bias of OMB exits in the OMB_noBC, RF_pre7 and Linear_pre7 experiments, caused by the instrument nonlinearity. The practical nonlinearity problem is an important factor affecting the calibration accuracy of infrared imaging instruments [41]. As expected, the nonlinear correction ability of RF would be desirable to overcome this assimilation issue when taking p₅–p₇ as one of the predictors. The OMB biases of AGRI channels 9–10 in RF_pre7 are both weak or negligible dependent on the scene temperature according to the smallest variation amplitude in a range of −2 K to 2 K.

Variations in the OMB biases with respect to the predictors of longitude is elaborated for AGRI channels 9–10 in Figure 5. A finite range of values for the scan position and longitude is relevant to the regional statistics applied for the limited-area model of the WRFDA system (Figure 5 and Figure 6). The biases variation along with the longitude display relatively small fluctuations near 0 K in the RF_pre4 experiment compared to those from other experiments, especially for the AGRI channel 9. The Linear_pre7 and OMB_noBC results show a clearly negative bias with an amplitude of more than 1 K and 2 K, respectively. In the region from 100°E to 130°E, the corrected bias from the RF_pre7 experiment is rather significant. Furthermore, it is found that the bias correction effects are different between channel 9 and channel 10. A possible reason is that the predictive capability of the RF model for channel 9 and channel 10 is different, as shown in Table 2.

The OMB biases before BC exhibit negative biases of AGRI channels 9–10 along with the latitudinal scan position values from 700 to 1100 (Figure 6a,c). It seems that these cold biases are efficiently revised by the RF method since the corrected biases are mostly closer to 0 K in the RF_pre4 and RF_pre7 experiments in Figure 6. Nevertheless, the cold biases still partly exist in the experiments of Linear_pre7. As is well known, the assimilation of undetected cloudy pixels will likely lead to a cooling observed field, for which the OMB biases are negative for the infrared radiances [42]. Results indicate that the nonlinear correction scheme has the potential ability to account for the prevalence of nonlinear biases caused by clouds.

3.2.2. OMB Distributions

Figure 7 shows the QC-passed scatters of the simulated brightness temperature (B) versus the observed brightness temperature (O) before and after the BC for AGRI channel 9 and channel 10 radiance data along with the density distribution of pixels. It can be seen that the mean of the OMB is notably large before BC, which is effectively reduced by all the experiments except for channel 10 in the Linear_pre7 experiment. The best agreement between the observation and the background after BC is provided by the RF-based experiments, with the smallest OMB mean nearly at 0.233 K for channel 9 and 0.105 K for channel 10. Additionally, the scatter distribution of RF_pre7 around the isoline is denser than the other experiments (Figure 7c,g). It is noted that the standard deviation (std) of corrected-OMBs in the experiments of RF_pre7 and Linear_pre7 are smaller than those from the experiments with only the predictors p₀–p₄.

The probability density function (PDF) of the normalized OMB is shown in Figure 8 based on different BC schemes after the quality control. Approaching the Gaussianity distribution (black dotted curves) is expected to statistically lead to more accurate analyses. The kurtosis of OMB distributions before BC (orange curves) are obviously sharper than the Gaussian curve. As compared to the reference, the positive skewness of OMB_noBC is symmetrically shifted to near 0 K by all the BC experiments for both AGRI channels. However, it is observed that the OMB from Linear_pre7 experiment displays too-pronounced peaks at the left tails near around −2 K, as is marked with the black boxes. This cold departure is probably corresponding to the cloudy pixels not being completely rejected. On the contrary, the frequencies of OMB from the RF_pre7 and RF_pre4 experiments match the Gaussian distribution relatively better, especially for those left tails. All the results suggest that the RF method is capable of measuring the nonlinear relationship between the OMB biases and the BC predictors.

In addition, the brightness temperature (BT) distributions of OMBs for AGRI channels 9–10 are illustrated with and without the BC by comparing the experiments of RF_pre7 and Linear_pre7 (Figure 9). It can be seen that the OMB BT distribution without the BC in Figure 9a,b shows large negative biases and positive biases in some areas, which could be modified by the BC experiments of RF_pre7 and Linear_pre7, respectively. However, there is a significant cold bias of OMB BT in the Linear_pre7 experiment, especially over the continental area (Figure 9e,f). This is consistent with the finding of more negative OMB biases from Linear_pre7, indicated by the black boxes dotted in Figure 8. In contrast, those cold biases have been removed to some extent in the RF_pre7 experiment (Figure 9c,d). The otherness of OMB BT distributions from RF_pre7 is relatively insignificant, with a smaller std of OMB against the linear experiments.

3.2.3. Verification of Analyses

It is important to verify the potential effects of different BC methods on the subsequent assimilation of AGRI radiances. If the model background or observations contains systematic errors, the analysis will diverge from the true state [43]. Figure 10 and Figure 11 show the bias and root-mean-square error (RMSE) vertical profiles of the analyses verified against a set of conventional observations (sounding and surface synoptic). The dots marked in Figure 10 and Figure 11 show the results above a 95% significant level.

Among the results of different BC experiments, the best initial estimate is achieved by the proposed RF_pre7 scheme with the smallest bias for all variables at almost all levels, except for a slightly positive impact on the temperature analyses at the higher levels above 400 hPa (Figure 10). However, the Linear_pre7 scheme is not as effective as the nonlinear scheme in the improvements of the initial conditions. Figure 11 displays the consistent conclusions as Figure 10 but for the verification metrics of RMSE. The results suggest that the analyses after BC are generally improved for the synoptic variables, though the RF nonlinear scheme has a little direct impact on the reduced RMSE of analyses for specific humidity, as shown in Figure 11d. Yet, in the RF_pre7 experiment, the variables of winds and temperatures below 400 hPa show some improvements in terms of the reduced RMSE scores (Figure 11a–c).

To further assess the robustness of the RF scheme, the experiments are conducted for another analysis time that is far from the training period. Figure 12 and Figure 13 show the bias and RMSE vertical profiles, same as Figure 10 and Figure 11, but are valid at 0600 UTC 20 July 2021. It is observed that the bias and RMSE scores from Linear_pre7 experiments are characterized by the smallest errors of U-winds in the upper troposphere and higher errors of other variables as a comparison to RF_pre7. The errors in RF_pre7 experiments are generally reduced, especially for the temperature. The smaller errors from the BC experiments relative to the OMB_noBC experiment indicate that the quality of the synoptic variables from analyses could be improved by the BC. Generally, the largest improvements exist in the experiments with the proposed RF scheme.

4. Discussion

In this study, a nonlinear BC method based on the RF machine learning technology was developed and implemented in order to remove the nonlinear conditional biases to enhance the efficiency of AGRI radiance data assimilation. Compared with other machine learning technologies, one of the advantages of RF is that it is suitable for high-dimensional datasets and has the capability to evaluate the feature importance of BC predictors. Unlike the traditional BC schemes, such as the VarBC method, the proposed scheme has the advantage of fitting the nonlinear relationship between the predictors and OMB biases to better correct the nonlinear conditional biases.

First, the performance of the RF-based correction model was evaluated with two different settings of BC predictors. According to the results, it is confirmed that the configuration of p₀–p₇ predictors is able to enhance the generalization capability of the RF model. The additional predictors of p₅–p₇ were also diagnosed in the RF model as contributing significantly to the prediction of objects. To be specific, the traditional predictors p₀–p₄ mainly represent the bias characteristics in the model space, while the newly added predictors p₅–p₇ could reflect the information of observed space errors. Moreover, those results can be explained by the fact that the RF model is suitable for the high-dimensional and multi-feature datasets.

The application effects of the BC nonlinear scheme were investigated and compared with the linear BC scheme for the observations of AGRI channels 9–10. As expected, the OMB biases could be effectively decreased by the RF nonlinear scheme against the linear scheme as well as a more idealized error distribution of OMB shown in the RF-based experiments. The nonlinear BC method based on RF outperforms the linear BC method, especially for correcting the biases of AGRI observation caused by the nonlinear processes. An apparent scene temperature-dependence bias of OMB is observed for AGRI channels 9–10, which is caused by the nonlinearity process of the instrument calibration. Furthermore, the significant negative biases of OMB may be related to a cooling observed field resulting from the simulation of cloudy pixels. The cloud-related processes are generally non-linear and complicated. Thus, the simulation of cloud-related processes is always simplified and linearized in most numerical models, causing non-negligible nonlinear biases.

In addition, according to the verification of analyses, it is speculated that the proposed nonlinear BC scheme with p₀–p₇ predictors has the potential to increase the quality of the analyses after recognizing the system biases from AGRI observations. All of those results shown in Section 3 suggest that identifying the nonlinear correlations among the OMB and the predictors through the RF method is conducive to BC with the subsequent positive impacts on data assimilation.

5. Conclusions

BC is an essential step to remove the systematic biases before radiance data assimilation. In this study, the AGRI clear-sky observations from channels 9–10 were utilized to explore the ability of a nonlinear BC method based on the RF technology in the WRF-3DVar system. The experiments are conducted using a comparative assessment, using a traditional linear method as the benchmark to assess the proposed RF scheme performance. According to the feature importance of BC predictors, two different settings of BC predictors are designed in order to further enhance the application effect of the RF model.

It is found that the configuration of predictors p₀–p₇ provides a consistent improvement in systematic bias correction. Utilizing the newly added predictors p₅–p₇ based on the RF nonlinear method may effectively contribute to the reduced scene temperature-dependent biases. The mean and std of OMB with the RF scheme are significantly decreased in comparison to the traditional linear scheme. Further, the proposed nonlinear scheme yields an optimized estimate of the real atmospheric state by taking advantage of the reduced system biases of AGRI channels 9–10.

Despite the above encouraging results, more efforts will be extended to assess the positive impact on data assimilation from the infrared observations, upgrading the numerical forecasting skill by applying the machine learning method in the procedure of BC. On the other hand, the inherent nonlinear interactions are largely found in the cloud-related processes between the local thermodynamic environment and various cloud hydrometers. Thus, nonlinear BC methods are also planned for the all-sky radiance data assimilation.