Tracing the Source of the Errors in Hourly IMERG Using a Decomposition Evaluation Scheme

Abstract

Integrated Multi-satellite Retrievals for Global Precipitation Measurement (IMERG) is an important satellite precipitation product of Global Precipitation Measurement (GPM) mission. Quantitative information about the errors of IMERG has great significance for the data developers and end users. In order to investigate the characteristics and the source of the errors contained in IMERG, a bias-decomposition scheme was employed to evaluate the hourly IMERG over the eastern part of Mainland China during the warm season. First, the total bias of IMERG before and after calibration (termed as precipitationUncal and precipitationCal) was calculated using rain gauge measurements as reference. Then the bias was decomposed into three independent components including false bias, missed bias, and hit bias. Finally, the hit bias was further decomposed according to the rainfall intensity measured by rain gauges. The results indicate that (1) the bias of precipitationUncal over the north part is dominated by hit bias and false bias, leading to the serious overestimation for the precipitation over this area, but it underestimates the precipitation over the south part with the false bias and missed bias acting as major contributors; (2) the precipitationCal overestimates the precipitation over more than 80% of the study areas mainly as a result of a large amplitude of false bias; (3) the calibration algorithm used by IMERG could not reduce the missed bias and enlarges the false bias over some regions, revealing a shortcoming of this algorithm in that it could not effectively alleviate the bias resulting from the rain areas delineation; (4) the hit bias of IMERG is strongly related with the rainfall intensity of rain gauge measurements, which should be beneficial for reducing the errors of IMERG. This study provides a deep insight into the characteristics and sources of the biases inherent in IMERG, which is significant for its utilization and possible correction in future.

1. Introduction

Accurate precipitation observations have great significance for preventing nature disasters, managing water resources, and simulating hydrological process [1,2]. However, it is still a great challenge to monitor precipitation over many developing and mountainous regions where the rain gauge network is still very sparse and the coverage of radar stations is seriously restricted by high mountains [3,4]. In contrast, satellite remote sensing could provide vast-coverage and cost-effective information about the spatial distribution and physical characteristics of rainfall particles in the clouds (such as the distribution of ice crystals and droplets), which have quantitative or qualitative relationships with the occurrence and intensity of precipitation [5,6,7]. Therefore, great efforts have been taken to retrieve precipitation from satellite observations at visible/infrared and microwave channels over the past decades, generating numerous satellite precipitation datasets (termed as SPDs) at both global and regional scales [8,9,10].

Global Precipitation Measurement (GPM) mission is an international program mainly performed by National Aeronautics and Space Administration (NASA) and Japan Aerospace Exploration Agency (JAXA) to generate reliable and high-resolution global precipitation estimates [1,11]. The dual-frequency precipitation radar (DPR; the Ku-band at 13.6 GHz and Ka-band at 35.5 GHz) and a multichannel GPM microwave imager (GMI; frequency range between 10 and 183 GHz) loaded on GPM Core Observatory provide unprecedented knowledge about light rainfall and snow which could significantly advance the development of satellite-based precipitation retrieval algorithms [12,13]. The first global precipitation product of GPM was released by NASA in January 2015 termed as Integrated Multi-satellite Retrievals for GPM (IMERG). Currently, several research studies have been conducted to assess the performance of IMERG, revealing that IMERG could bring better estimates of precipitation than existing SPDs (such as Tropical Rainfall Measuring Mission (TRMM) 3B42) [14,15,16]. As such, this is anticipated to significantly promote the studies of hydrology, meteorology, and ecology.

However, extensive studies have pointed out that SPDs usually contain notable biases and random errors, which could be inherited and exaggerated by hydrological models driven by SPDs because of the highly nonlinear relationship between the hydrological process and precipitation [17,18,19,20]. Therefore, it is critical to quantitatively evaluate the performance of SPDs both for the algorithm developer finding the source of the errors and for guiding external users in selecting suitable SPD for their application(s) [21,22]. Conceptually, retrieving precipitation from satellite observations mainly consist of two steps, including rain/no-rain areas delineation and rainfall rate estimation, and each step is the source of the errors in SPDs. In the first step, errors will arise if some raining pixels are not detected or some pixels under no-raining condition are misclassified as raining pixels. Even for the correctly identified raining-pixels, the second step will generate errors when the rainfall rates are not accurately estimated [23]. Traditional evaluation metrics (such as bias, determination of coefficients) used by previous studies are unable to assess these errors separately from various sources, making them insufficient to comprehensively reflect the characteristics of the errors in SPDs.

To resolve this problem, Tian et al. [23] proposed an evaluation scheme which decomposed the bias of SPDs into three independent components (i.e., false bias, missed bias, and hit bias), thereby providing a new perspective to identify the sources of the biases. Tang et al. [24] applied this scheme to analyze the errors in TRMM over the continental United States and found that the bias of TRMM over this region is mainly from hit bias during warm seasons and false bias and missed bias during cold seasons. Furthermore, decomposing the total bias of SPDs is beneficial for understanding the uncertainty in hydrological models driven by SPDs. For example, Gebregiorgis et al. [25] investigated the relationship between the hydrologic model simulation error and the three decomposed components, revealing that the hit bias and missed bias are responsible for runoff error and soil moisture error, respectively. As revealed by these research studies, decomposing the bias of SPDs could provide quantitative evidence to trace the source of errors (from rain areas delineation step or from rainfall rate estimation step) as well as in hydrological models driven by SPDs. Therefore, it is a significant issue to decompose the bias of IMERG which has not been well investigated until now.

In this study, the total bias of the hourly IMERG before and after calibration by Global Precipitation Climatology Centre precipitation datasets (termed as precipitationUncal and precipitationCal, respectively) are assessed using the rain gauge measurements over the eastern part of mainland China. Subsequently, the biases of the two datasets are decomposed into false bias, missed bias, and hit bias according to the algorithm proposed by [23], with the aim of providing comprehensive characteristics of the errors in IMERG and revealing the strengths and limitations of the calibration algorithm used by IMERG. Finally, the hit bias is further decomposed according to the precipitation intensity measured by rain gauges, which may provide quantitative information for the possible correction of this datasets.

2. Data and Method

2.1. Study Area

This study focuses on the eastern part of mainland China between 18° N and 54° N, 97° E and 135° E (as shown by Figure 1). The climate of this region presented high variation in space. The north part is dominated by medium latitude monsoon climate characterized by wet summers and dry winters, while the south part is mainly affected by the monsoons from the Indian and Pacific Oceans which bring abundant precipitation to this area. In contrast, the northwest part presented less precipitation than other parts because of the long distance away from oceans.

2.2. Data

2.2.1. Rain Gauge Data

In this study, the hourly rain gauge observations from more than 30,000 automatic weather stations (AWSs) located in mainland China were used as the reference data. These observations undergo serious quality control including analysis of extremes, contiguous values, and spatial consistency checking. The quality-controlled data is then interpolated onto regular grid dataset with 0.1° using a modified climatology-based OI interpolation algorithm [26]. Currently, this dataset has been successfully used to validate high-resolution SPDs and generate multi-source precipitation datasets over mainland China [27]. In this study, only the grids with at least one rain gauge were selected to evaluate the IMERG in order to keep the reliability of these reference data. The spatial distribution of the rain gauges is heterogeneous (as showed by Figure 1). More than 60% of the rain gauges are located over the south part of the study area, whereas there are sparse rain gauges over the north part.

2.2.2. IMERG Satellite Precipitation Dataset

IMERG is generated by an algorithm based on the studies of producing the Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA) [8], CMORPH [10], and Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Cloud Classification System (termed as PERSIANN-CCS) [28], respectively. The main procedures of the algorithm used in IMERG include the following steps: (1) the precipitation estimates based on PMW observations from various satellites are inter-calibrated with the GPM Combined Instrument product; (2) the PMW precipitation datasets are propagated by the motion vectors derived from the infrared brightness temperatures; (3) the results of the second step are calibrated by Global Precipitation Climatology Centre precipitation datasets (termed as GPCC). It should be noted that IMERG has three runs for different applications (including Early, Late, and Final run). The Early run and Late run are about 4 h and 12 h after the observation time, which are suitable for time-sensitive applications (such as prediction of flash floods). The Final run is about two months after the observation time with GPCC adjustment to alleviate bias, which can be applied in research studies concerning climatology and ecology. In this study, we focus on considering the Final run IMERG with half-hour scale from April to October in 2014 (downloaded from https://storm-pps.gsfc.nasa.gov/storm/ (in which both of the calibrated and un-calibrated datasets are included)).

2.3. Method

This study focuses on evaluating and comparing the total bias and decomposed bias of IMERG before and after calibration by GPCC (termed as precipitationUncal and precipitationCal, respectively) over the eastern part of mainland China during the warm season. Additionally, the hit bias of IMERG is further decomposed according to rainfall intensity of the rain gauge measurements. The detailed steps of the evaluation process are as follows:

(1)

The rain gauge measurements and IMERG are pre-processed. First, the precipitationUncal and precipitationCal are summed up from half-hour to hourly scale. Second, the rain gauge measurements, precipitationUncal, and precipitationCal at same location and time are matched together.

(2)

The total biases of the precipitationUncal and precipitationCal are calculated according to the Equation (1):

$$\mathrm{bias}={\displaystyle \sum _{k=1}^n\left(I_k-O_k\right)}/n$$

(1)

where I_k is the precipitation estimated by IMERG; O_k is the observation of rain gauge;

(3)

The bias calculated by Equation (1) is decomposed into three independent components including missed bias (rain areas which are incorrectly determined as no-rain areas by IMERG, termed as MB), false bias (no-rain areas which are incorrectly determined as rain areas by IMERG, termed as FB), and hit bias (the rain areas correctly determined by IMERG, but the precipitation intensity is inaccurately estimated, termed as HB) by Equations (2)–(4), respectively. It is obvious that the sum of the FB, MB, and HB is equal to the bias, but the magnitude of some components could exceed the total bias because the three components could cancel each other.

$$FB={\displaystyle \sum _{f=1}^F\left(I_f\right)}/n$$

(2)

$$HB={\displaystyle \sum _{h=1}^H\left(-O_h\right)}/n$$

(3)

$$MB={\displaystyle \sum _{m=1}^M\left(I_m-O_m\right)}/n$$

(4)

where FB, HB, and MB are the false bias, hit bias, and missed bias, and F, H, and M are the number of false alarmed grids, hit grids, and missed detected grids, respectively. O_h, and O_m are the rain gauge observation at hit grids and missed detected grids, respectively. In contrast, I_f, and I_m are the precipitation estimated by IMERG at false alarmed grids and missed detected grids, respectively. O_k is the observation measured by rain gauge k, and n is the number of all grids over the study area.

(4)

The HB of IMERG is further decomposed according to the precipitation intensity of the rain gauge measurements, with the aim of investigating whether any quantitative or qualitative relationships exist between them. To perform this analysis, all samples of hit bias are separated into 49 groups using the following thresholds: first, the samples with Rain gauge measurements values above 0 and no more than 3 mm/hour are separated into 30 groups with 0.1 mm/hour per step. Second, the samples with rain gauge measurements between 3 mm/hour and 10 mm/hour are separated into 14 groups for each 0.5 mm/hour step. Third, the samples with IMERG between 10 mm/hour and 15 mm/hour are separated into 5 groups at each 1 mm/hour step. Then the bias of each group is calculated according to the Equation (1).

3. Results

3.1. Spatial Pattern of the IMERG

In this study, we calculated the mean precipitation intensity of the gauge, precipitationUncal, and precipitationCal to analyze whether the IMERG could reflect the spatial pattern of the precipitation over the study area. Additionally, the values of the precipitationUncal and precipitationCal were compared grid by grid to investigate the influence of the calibration algorithm for different regions. As shown in Figure 2a, the spatial pattern of the precipitation over the study area presents an obvious regional characteristic whereby the precipitation increases from north to south. Both precipitationUncal and precipitationCal capture this characteristic relatively well (as shown in Figure 2b,c), but the former tends to overestimate the precipitation over the north part (as marked by red rectangles in Figure 2b). In contrast, the precipitationCal presents much lower precipitation than precipitationUncal over these regions, indicating that these errors in precipitationUncal could be effectively alleviated after the calibration process. Additionally, we compared the values of the precipitationUncal and precipitationCal grid by grid (as shown by Figure 2d), and the result indicates that the calibration algorithm used by IMERG tends to elevate the precipitationUncal over the north part but reduces it over the south part.

Figure 3 presents the scatter plots between the mean hourly rain gauge measurements and precipitationUncal and precipitationCal, respectively. The solid line shows the linear least squares relationship between the rain gauge measurements and the IMERG, and the dash line is the 1:1 line. Considering the precipitation presented obvious regional characteristics (as shown in Figure 2), this research was conducted by dividing the study area into a north part and south part with 36° N serving as the line of demarcation. It can be seen in Figure 3a that the precipitationUncal and rain gauge measurements did not present a very strong relationship over either the north (R² = 0.43) or south part (R² = 0.47), and that precipitationUncal showed better performance over the north part of mainland China than the south part. A notable characteristic of precipitationUncal is that it tends to underestimate the precipitation over the north part while it overestimates the precipitation over the south part. In contrast, precipitationCal shows much better performance than precipitationUncal with more than a 50% higher R² and about 30%~40% lower root-mean-square-error (termed as RMSE). It should be noted that precipitationCal overestimates the precipitation over most of the southern part of mainland China, which is different from precipitationUncal.

3.2. Bias Decomposition

The total bias of precipitationUncal and precipitationCal was calculated and decomposed into three independent components according to the methods illustrated in Section 2.3. The spatial patterns of the total bias contained in the two datasets are presented by Figure 4. It can be seen from Figure 4a that the total bias of precipitationUncal exhibited obvious regional characteristics, for example, it overestimates the precipitation in the north part while it underestimates precipitation over the south part of the study area. In contrast, the precipitationCal presented overestimations across most grids of the study areas (about 80%), but it showed less bias than precipitationUncal for the north part (Figure 4b).

Figure 5 shows the decomposed components of the total bias contained in precipitationUncal and precipitationCal. It can be inferred from this figure that the calibration algorithm used by IMERG brings different influences to the three decomposed components. The FB of precipitationUncal (Figure 5a) mainly ranges from 0.02 mm/hour to 0.04 mm/hour and presented smooth variation in space. However, we note the relatively high FB for the North China Plain where serious air pollution happens (about 35°~40° E, 112°~120° E, denoted by the red rectangle in Figure 5a), revealing potential evidence that the air pollution may be an important inducement of false bias. By visually comparing the FB of precipitationCal (Figure 5b) and precipitationUncal, it can be concluded that the calibration algorithm of IMERG could alleviate the FB over the North China Plain but enlarge the FB over the south part of the study area. Figure 5c,d illustrated the spatial pattern of the MB of precipitationUncal and precipitationCal. It should be pointed out that the calibrated algorithms used by IMERG could not correct the rain areas delineation, therefore, the MB of precipitationUncal and precipitationCal shared the same spatial pattern. Generally, it can be seen in these figures that the MB increases from north to south with the range between 0.01 and 0.15 mm/hour. The HB of precipitationUncal and precipitationCal is presented in Figure 5e,f, respectively. The precipitationUncal yielded positive HB over 51% of the grids which are mainly distributed over the north part of the study area, while this ratio increased to 82% for precipitationCal. Especially, the calibration algorithm of IMERG seriously enlarged the HB over some regions over the south part (denoted by red rectangles in Figure 5f).

Further analysis was performed to investigate the sources of the bias inherent in IMERG, with the aim of providing quantitative information which is indispensable for its application and possible correction.

Table 1. Averaged error components of IMERG before and after calibration (termed as precipitationUncal and precipitationCal) as percentages of total precipitation measured by rain gauges for north and south parts of mainland China

Region	Dataset	Total Bias	False Bias	Missed Bias	Hit Bias
South part	PrecipitationUncal	58.2%	38.6%	−20.8%	40.4%
	PrecipitationCal	27.7%	32.5%	−20.8%	16.0%
North part	PrecipitationUncal	−13.6%	17.0%	−23.8%	−6.8%
	PrecipitationCal	12.9%	23%	−23.8%	13.7%

presents the average amplitudes of the total errors and three decomposed components for the north part and south part of mainland China (with 36° N serving as the line of demarcation). It can be seen in this table that both PrecipitationUncal and PrecipitationCal overestimate the precipitation over the south part, but that the calibration algorithm could effectively account for the hit bias contained in PrecipitationUncal in this region. In the north part, the amplitude of the total bias contained in PrecipitationCal is smaller than PrecipitationUncal mainly because the hit bias changed from negative to positive after the calibration process.

The HB of the precipitationUncal and precipitationCal was further decomposed according to the precipitation intensity measured by rain gauges (Figure 6). We found that both precipitationUncal and precipitationCal tend to overestimate the light precipitation while underestimating the heavy precipitation. Additionally, the relationship between the rain gauge measurements and the HB could be described by a three parameters power function with the determination of coefficient above 0.99, which provided useful information to alleviate the HB of IMERG.

4. Discussion

Although previous studies [14,29,30] have revealed that the IMERG showed more notable progress than existing SPDS, it can be inferred from the results of this study that both precipitationUncal and precipitationCal present notable bias over some regions of the study area. In our opinion, two reasons may be responsible for the uncertainty contained in IMERG: (1) Retrieving precipitation from PMW observations using Goddard Profiling Algorithm (GPROF) is the key process to produce IMERG, and the performance of GPROF largely relies upon the quality of the database composed by the pairs between the hydrometeor profile and PMW brightness temperature [31]. The DPR loaded on the GPM Core Observatory is designed to retrieve a high-quality hydrometeor profile which is synchronous with the PWM brightness temperature observed by the GMI. However, the hydrometeor profile acquired by the DPR loaded on the GPM Core Observatory is not used in the first version of IMERG because the number of DPR observations is not enough to finalize this database. Currently, the observations from national oceanic and atmospheric administration (termed as NOAA) National Mosaic and Multi-Sensor quantitative precipitation estimation (termed as QPE) radar are used as substitutes over land and divided into ten classes according to the land surface characteristics, but they could not provide comprehensive characteristics for hydrometeor profiles all over the world, which may lead to uncertainty associated with IMERG; (2) Michaelides et al. [32] revealed that the retrieval algorithm of SPD is usually optimized for particular regions, which could make the SPD less adapted for other regions. To our knowledge, there has been no evidence about the adjustment of the IMERG for mainland China until now. This may be one possible reason for the uncertainty of the IMERG over this area.

The results of this research also reveal a shortcoming of the calibration algorithm used by IMERG which calibrates the precipitation retrieved from satellite observations (termed as SP) by multiplying the ratio of the SP and the satellite-gauge merged precipitation datasets (termed as SG) at a monthly scale [8,9]. The advantage of this algorithm is that it creates a correction coefficient for each grid and for each month, which are beneficial for reducing the regional and seasonal bias contained in SPD. However, it does not change the rain gauge rain areas delineation, so the missed precipitation and false precipitation which are only contained in high resolution SPDs could not be alleviated [33]. This limitation may lead to various influences on the performance of this algorithm. First, for the missed detected precipitation, this algorithm could not influence the missed bias because the value of IMERG over the missed detected grids is zero. For the false alarmed grids, if the SG/MG ratio is smaller or equal to 1, the false bias will be alleviated or kept unchanged, but if the SG/MG ratio is above 1, the multiplication of this ratio and the un-calibrated IMERG will enlarge the magnitude of the false alarmed precipitation. For the grid in which false alarmed precipitation is serious and the SG/MG ratio is above 1, the enlarged false alarmed precipitation may offset the improvement of the hit bias, leading to an even poorer performance of the calibrated IMERG than the un-calibrated IMERG. It can be inferred from Figure 4 that the calibration algorithm tends to elevate the values of the precipitationUncal over the south part of the study area, leading to the deterioration of the false bias. Therefore, we strongly suggest that there is a significant need to pay more attention to eliminate the false alarmed and missed detected precipitation in the IMERG before its application. However, it should be clearly pointed out that this is a preliminary conclusion about the characteristics and sources of the errors contained in the IMERG over mainland China because this dataset was retrieved using the at-launch algorithm. More investigations will be conducted for the future versions of IMERG.

5. Conclusions

In this study, the bias of the hourly IMERG before and after calibration with GPCC was evaluated and compared with each other with supporting rain gauge data for eastern mainland China. Then, the total bias was decomposed into three independent components including false bias, missed bias, and hit bias to investigate the characteristics and the sources of the biases. Additionally, the hit bias was further decomposed according to the precipitation intensity. Several conclusions can be drawn from the results of this study:

(1)

Both precipitationUncal and precipitationCal could capture the spatial pattern of the precipitation over the study, but the former could better estimate the precipitation volume than the latter.

(2)

The calibration algorithm used by IMERG could significantly reduce the hit bias over the south regions, but it exaggerated the false bias over the south part. Additionally, this algorithm could not alleviate the missed bias of IMERG, which is largely responsible for the bias over the south part of the IMERG.

(3)

The false bias and hit bias were responsible for a large part of the total bias of both precipitationUncal and precipitationCal. Therefore, considerations concerning how to retrieve more accurate rain areas delineation using satellite observations deserve more attention in order to better apply and improve the application of IMERG in future work.

(4)

There was a strong non-linear relationship between the hit bias of IMERG and the rainfall intensity measured by rain gauges, which could be beneficial for the further correction of this new satellite precipitation dataset.