Evaluation of Gridded Multi-Satellite Precipitation Estimation (TRMM-3B42-V7) Performance in the Upper Indus Basin (UIB)

Abstract

The present study aims to evaluate the capability of the Tropical Rainfall Measurement Mission (TRMM), Multi-satellite Precipitation Analysis (TMPA), version 7 (TRMM-3B42-V7) precipitation product to estimate appropriate precipitation rates in the Upper Indus Basin (UIB) by analyzing the dependency of the estimates’ accuracies on the time scale. To that avail, various statistical analyses and comparison of Multisatellite Precipitation Analysis (TMPA) products with gauge measurements in the UIB are carried out. The dependency of the TMPA estimates’ quality on the aggregation time scale is analyzed by comparisons of daily, monthly, seasonal and annual sums for the UIB. The results show considerable biases in the TMPA Tropical Rainfall Measurement Mission (TRMM) precipitation estimates for the UIB, as well as high numbers of false alarms and miss ratios. The correlation of the TMPA estimates with ground-based gauge data increases considerably and almost in a linear fashion with increasing temporal aggregation, i.e., time scale. There is a predominant trend of underestimation of the TRMM product across the UIB at most of the gauge stations, i.e., TRMM-estimated rainfall is generally lower than the gauge-measured rainfall. For the seasonal aggregates, the bias is mostly positive for the summer but predominantly negative for the winter season, thereby showing a slight overestimation of the precipitation in summer and underestimation in winter. The results of the study suggest that, in spite of these discrepancies between TMPA estimates and gauge data, the use of the former in hydrological watershed modeling undertaken by the authors may be a valuable alternative in data-scarce regions like the UIB, but still must be taken with a grain of salt.

1. Introduction

The continued improvements in computation capabilities and the subsequent increase in the development of spatially explicit and distributed models for expressing environmental phenomena have necessitated the provision of more intensive and improved data for environmental variables both in space and time. Two major issues, particularly in hydro-meteorological studies, are the possible sparsity of data sampling points (gauge stations), and the discontinuities in the data and in the quality of the temporal records. These issues are more frequent in mountainous regions with high altitudes, because they are immensely challenging environments for measurements of precipitation through remote sensing or traditional ground-based methods due to the difficult topography and the highly-variable weather and climatic conditions [1,2]. These factors have proved to be the main hurdles due to which many developing countries are unable to achieve consistent spatial and temporal coverage for ground-based precipitation measurements [2,3], therefore making it difficult for them to achieve an effective spatial coverage of rainfall [4,5]. The consequent lack of good quality precipitation data is thus a big hurdle for properly assessing impacts of climate change on water resources in these regions [1].

As data with an acceptable gridded resolution of daily climatic variables are critical for hydrological and water resources modeling [6,7], managing the gaps in the data appropriately is then the first stage of most climatological, environmental, and hydrological studies [2]. This step is also necessary to improve the spatial resolution for sparse gauge station data sets before using them as an input for spatially-distributed rainfall-runoff models, because the gauge-based interpolation methods, commonly used in hydrologic models, usually do not cover the spatial heterogeneity of the variability of climatic variables in the catchment. These errors in the interpolated data field then have the potential to significantly bias model calibrations and water balance calculations [6].

Fortunately, continued scientific developments are also providing new prospects for addressing these issues. For example, the advancements in gathering and deriving climate data through satellite remote sensing could provide a possible opportunity to address some of the issues with regard to the spatial coverage of climate data. That is why the use of satellite-based precipitation products individually or in combination with land-based gauge data has been increasingly recognized as a very promising alternative to address the aforementioned problems [5]. Such precipitation products have proven to be of great value, especially in developing countries with remote and high-altitude locations where conventional rain gauge or weather data are of bad quality or have low coverage [8].

There are numerous satellite-based precipitation products currently available, with varying degrees of accuracy. These include the Climate Prediction Center (CPC) morphing algorithm (CMORPH) [9], Global Satellite Mapping of Precipitation [10,11,12], Naval Research Laboratory Global Blended Statistical Precipitation Analysis [13], the Tropical Rainfall Measuring Mission (TRMM), Multisatellite Precipitation Analysis (TMPA) [14,15], and a few others. Since their inception, most of these gridded datasets have been evaluated for their suitability and usability for specific regions or intended uses. In general, such investigations are less frequently carried out for mountainous regions and even less so for our study area, the Hindukush, Karakurm, and Himalays (HKH) region. In the HKH and the Upper Indus Basin (UIB) region, most of the reported works related to the evaluation of gridded precipitation products are suggestive of considerable biases in the gridded products in comparison to the gauge records [16,17,18,19].

Additionally, the quality, coverage, and representativeness of the available observed gauge records in the HKH have also been questioned and have sometimes been regarded as having considerable underestimations of regional precipitation amounts [20,21,22], especially at higher altitudes [23]. The spatial distribution of estimated real precipitation by Khan and Koch (unpublished) over the study area is given in Figure A1, while the vertical meteorological and cryspheric regimes in the UIB (modified from Hewitt 2007) are given in Figure A2.

While in most cases these gridded global precipitation data sets are some interpolated version of point measurements (most often through geo-statistical procedures), they may only be useful for regions where dense network of rain gauges are available, because otherwise, in the absence of a dense enough networks or over regions of complex topographies, the interpolated precipitation present a very generalized restitution which is not able to reflect properly the prevalent orographic, surface, or atmospheric processes [19].

In comparison with the sparse gauge observations or gridded data products based on them, satellite-based precipitation products such as Tropical Rainfall Measurement Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA), version 7 (TRMM-3B42-V7) have an inherent advantage due to their higher spatial coverage. However, they also have certain limitations because they are indirect estimates of rainfall, which depend on the cloud height and the properties of the cloud’s surfaces (IR-algorithms) and on the integrated sparse and multi-source hydro-meteorological content (passive microwave algorithms) [15,24,25]. Therefore, before such satellite-based data can be used with confidence, it is important to evaluate their accuracy or error characteristics by comparing them with data from ground-based observations.

Based on these considerations, the current study aims at assessing the skill of the TRMM precipitation dataset in matching the magnitudes and occurrences at different temporal scales at all the points of the observational network available in order to evaluate its further processing and correction requirements or suitability for subsequent use in distributed hydrological modeling.

2. Materials and Methods

2.1. Study Area and Data

2.1.1. Study Area: The Upper Indus River Basin (UIB)

The Indus River, one of the largest rivers in Asia with a total length of about 2880 km, has a drainage area of about 912,000 km² which extends across portions of India, China, Pakistan, and Afghanistan. The portion of the Indus that comprises the Upper Indus River Basin (UIB), with a logical lower boundary at Tarbela Dam (Figure 1), is about 1125 km long and drains an area of about 170,000 km² [26].

Being a high-mountain region, the UIB contains the largest area of perennial glacial ice cover (22,000 km²) outside the polar regions of the earth, and which extends even further during the winter season [27]. The altitude within the UIB ranges from as low as 455 m to a high of 8611 m and, as a result, the climate varies greatly within the basin [28].

The summer monsoon has no significant effect on the basin, as almost 90% of its area lies in the rain shadow of the Himalayan belt [20,29]. Except for the south-facing foothills, the intrusion of the Indian Ocean monsoon is limited by the mountains so that its influence weakens northwestward. Subsequently, the climatic controls in the UIB are quite different from that in the Himalayas on the eastern side. In fact, over the extent of the UIB, most of the annual precipitation originates in the west and falls in winter and spring, whereas occasional rains are brought by the monsoonal incursions to the trans-Himalayan areas. Even during the summer months, the trans-Himalayan areas do not obtain all their precipitation from the monsoons [30,31,32,33].

Climatic variables are usually strongly influenced by topographic altitude. Several studies have reported that precipitation in the HKH region exhibits large changes over short distances and has a considerable vertical gradient [30,31,34,35,36,37]. Thus, the northern valley floors of the UIB are arid, with annual precipitation of only 100–200 mm, but these totals increase with elevation and reach up to 600 mm at 4400 m height and lead to annual glacier accumulation rates of up to 1500 to 2000 mm at 5500 m altitude, according to some glaciological studies [30]. The average snow cover area in the Upper Indus River Basin fluctuates between 10% and 70%. Snow cover in the area is at a maximum of 70–80% in the winter snow accumulation period (December to February) and at a minimum of 10–15% in the summer snow melt period (June to September) [28]. Stream flow is generated by the combination of the storm runoff in the lower parts of the upper Indus basin and the snow and glacier runoff from the higher parts of the UIB [26,38].

2.1.2. Data

TMPA Data (TRMM-3B42-V7)

In this study the TRMM-3B42-V7 precipitation product is used. This product is basically a calibration-based combination scheme for precipitation estimates from multiple satellites and space-borne sensors, including infrared, microwave, radar data, and gauge measurements. Though the dataset has very good spatio-temporal resolution (0.25° × 0.25° grid, 3-hourly) and good global coverage (latitude band 50° N to 50° S) and has data available from 1998 to the recent past [1,15]. It also has some uncertainties, because the inputs on which it is based are indirect estimates of rainfall, depending on the cloud height and the properties of the cloud surface (IR algorithms) and on the integrated sparse and multi-source hydro-meteorological content (passive microwave algorithms) [14,15,25].

During the current study, 3-hourly data from 1 January 1998 to 31 December 2008 were summed to daily accumulated precipitation for each of the 0.25° × 0.25° grid boxes (which have a gauge station), and evaluated for matches with the corresponding gauge station’s observed daily accumulated precipitation. As the observational network is scant, no TRMM grid box included more than one in situ gauge station.

Observed Ground-Based Data

In the HKH region of Pakistan, observed in situ data are limited and operated by different organizations, mainly the Pakistan Meteorological Department (PMD) and Water and Power Development Authority (WAPDA). The stations operated by PMD (Figure 1) have daily-time-step climate data available for longer periods (1947 to date), but with huge gaps and missing data in the records and with only monthly data available freely for research purposes. Furthermore, all the PMD stations are valley-based, at elevations below 3000 m.a.s.l. in altitude and, therefore hardly represent the frequency and amount of precipitation in the high-altitude areas. The climate stations, operated by the WAPDA, are fairly new and have considerably consistent data over the time period, coinciding with the TRMM product. These gauge stations are distributed almost evenly across the UIB inside Pakistan and cover a wide range of elevations.

During the current study, daily precipitation records of 14 meteorological stations operated by WAPDA are utilized for the evaluation of the TRMM estimates. Their geographical attributes are given in

Table 1. Geographical attributes of the precipitation gauge network.

Description	No.	Station	Latitude (◦)	Longitude (◦)	Altitude (m)
High Altitude (2367–4440 m.a.s.l.) stations operated by Water and Power Development Authority (WAPDA), Pakistan	1	Burzil	34.91	75.90	4030
	2	Deosai	35.09	75.54	4149
	3	Hushey	35.42	76.37	3075
	4	Khot	36.52	72.58	3505
	5	Khunjrab	36.84	75.42	4440
	6	Naltar	36.17	74.18	2898
	7	Rama	35.36	74.81	3179
	8	Rattu	35.15	74.8	2718
	9	Shendoor	36.09	72.55	3712
	10	Shigar	35.63	75.53	2367
	11	Ushkor	36.05	73.39	3051
	12	Yasin	36.45	73.3	3350
	13	Zani	36.33	72.17	3895
	14	Ziarat	36.77	74.46	3020

. The evaluation is limited to the duration of 1998 to 2008, as the observed precipitation data could not be acquired for the period beyond 2008.

2.2. Methods

The quantitative comparison of the TRMM estimates with the ground rain-gauge station observations is done by employing various widely used statistical indicators. These include the correlation coefficient (r), the mean relative bias error (rBIAS), the mean bias error (MBE), the mean absolute error (MAE), and the root mean square error (RMSE), defined in the following equations:

$$r=\frac{{\displaystyle \sum _{i=1}^n(T_i-\overline{T})(G_i-\overline{G})}}{\sqrt{{\displaystyle \sum _{i=1}^n{(T_i-\overline{T})}^2}}\sqrt{{\displaystyle \sum _{i=1}^n{(G_i-\overline{G})}^2}}}$$

(1)

$$rBIAS=\frac{1}{n}{\displaystyle \sum _{i=1}^n(\frac{T_i-G_i}{G_i})}$$

(2)

$$MAE=\frac{1}{n}{\displaystyle \sum _{i=1}^n|T_i-G_i|}$$

(3)

$$MBE=\frac{1}{n}{\displaystyle \sum _{i=1}^n(T_i-G_i)}$$

(4)

$$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{(T_i-G_i)}^2}}$$

(5)

where n is the number of samples, T_i refers to satellite-based precipitation, G_i is gauge-based precipitation, and $\overline{T}$ and $\overline{G}$ are the corresponding means. Among these statistical indices, r shows the degree of linear correlation between TRMM precipitation estimates and gauge observations. MBE, MAE, and rBIAS are used to assess the systematic bias, i.e., the deviation of the satellite precipitation from the gauge observations, and the RMSE gives the magnitude of the average error in relative terms.

In addition, evaluations were also made for the daily TRMM estimates and gauge data over the Indus river basin, based on a 2 × 2 contingency table (

Table 2. Contingency Table 2 × 2. TRMM: Tropical Rainfall Measurement Mission.

		Observed Values (Gauge Data)		Total
		YES	NO
Estimated Values (TRMM estimates)	YES	-a -Hits	-b -False Alarms	Total—Yes Estimated
	NO	-c -Misses	-d -Correct Negative	Total—No Estimated
Total		Total—Yes Observed	Total—No Observed	Totala + b + c + d

), by detecting rain events (Hits), no events (Correct Negative), Misses by the TRMM, and False Alarms by the TRMM. More specifically, we used a threshold of 0.3 mm/d to differentiate precipitation and no precipitation events, since lower precipitation values may be the result of noise, as indicated by [31,39] etc.

Based on these four indicators, with orders as shown in the table, several categorical statistical indices are derived, including accuracy (Ac), bias score or frequency bias index (FBI), probability of detection (POD), false alarm ratio (FAR), critical success index (CSI), and true skill statistics (TSS) [40,41], defined in the following equations:

$$Ac=\frac{a+d}{Total}$$

(6)

$$FBI=\frac{a+b}{a+c}$$

(7)

$$POD=\frac{a}{a+c}$$

(8)

$$FAR=\frac{b}{a+b}$$

(9)

$$CSI=\frac{a}{a+b+c}$$

(10)

$$TSS=\frac{a}{a+b}-\frac{b}{c+d}=\frac{ad-bc}{\left(a+b\right)\left(c+d\right)}$$

(11)

where a represents the number of rainfall events that have been successfully estimated by TRMM data (hits), b is the number of events incorrectly predicted as rain events by TRMM (false alarms), c is the number actual events that are missed by TRMM (Misses), and d is the number of dry days or no-rainfall events identified successfully by the TRMM dataset. For each day, depending on how the estimated and observed precipitation behave, any event above the given threshold (0.3 mm) is scored either as a Hit, Miss, False Alarm or Correct Negative, so that the rainfall is a Hit if both TRMM and observed precipitation reach the threshold; False Alarm if only the TRMM estimate reaches the threshold; Miss if only the observed precipitation reaches it; and Correct Negative if both are below the threshold. The number of Hits, False Alarms, Misses and Correct Negatives are used in Equations (5)–(10) to calculate the above mentioned statistical indices.

Each of these indices provides a specific information on the two data sets compared. Thus, Ac indicates the fraction of estimates which is correct (range: 0 to 1, perfect score: 1); FBI indicates whether the estimated dataset has a tendency to underestimate (FBI < 1) or to overestimate (FBI > 1) rain events; POD quantifies the fraction of rain occurrences that is estimated correctly (range: 0 to 1, perfect score: 1); FAR measures the fraction of false alarms in the satellite rain estimates (range from 0 to 1, perfect score: 0); and CSI measures the fraction of estimated events that are correctly predicted (range from 0 to 1, perfect score: 1). Unlike all the aforementioned indices, TSS does not depend on the frequency of the climatological event and uses all elements in the contingency table (Table 2). Thus, TSS provides a measure of the accuracy of the estimates in terms of the probability of correct detection of events or no events. In this case the range is from −1 to 1, with a perfect score being 1, with 0 showing no skills, and a negative score signifying that the estimates are worse than a random forecast.

3. Results and Discussion

The assessment of the reliability of the TRMM estimates and their comparisons with the rain data from gauge station presented in this section has been done by three different methodologies: (1) a statistical analysis, based on r, rBIAS, MBE, MAE, and RMSE for daily, monthly, annual and seasonal data aggregates; (2) categorical statistics of daily data by computing Ac, FBI, POD, FAR, CSI, and TSS; and (3) a visual comparison for monthly, annual, and seasonal data.

3.1. Statistical Analysis for Daily, Monthly, Annual, and Seasonal Agreggates

The results of the TRMM-assessment based on the statistical measures r, rBIAS, MBE, MAE, and RMSE are given for daily data aggregation in

Table 3. Statistical analysis for daily aggregates. UIB: Upper Indus Basin; r: correlation coefficient; rBIAS: mean relative bias error; MBE: mean bias error; MAE: mean absolute error; RMSE: root mean square error.

Sub-Basin	Station	Daily
		r	rBIAS	MBE (mm)	MAE (mm)	RMSE (mm)
Southern UIB	Burzil	0.22	−0.42	−11.77	16.20	20.76
	Deosai	0.10	0.99	4.41	14.53	26.33
	Rama	0.23	−0.22	−16.20	18.31	27.44
	Rattu	0.14	0.69	−7.90	18.12	29.24
Eastern UIB	Shigar	0.08	1.31	−3.99	10.24	17.81
	Hushey	0.14	−0.07	−5.73	10.79	14.73
Northwestern UIB	Khot	0.19	0.70	0.49	4.93	8.08
	Naltar	0.25	0.24	−11.94	15.39	21.10
	Shendoor	0.16	1.48	1.22	9.46	15.67
	Ushkor	0.21	0.70	−0.51	8.16	12.74
	Yasin	0.10	5.27	24.24	28.68	46.57
	Zani	0.13	−0.14	−15.23	19.16	26.96
Northern UIB	Khunjrab	0.15	−0.27	−5.50	10.52	15.40
	Ziarat	0.14	0.71	−0.94	6.05	9.48
Basin average		0.16	0.78	−3.53	13.61	20.88
Maximum		0.25	5.27	24.24	28.68	46.57
Minimum		0.08	−0.42	−16.20	4.93	8.08

, for monthly and annual data aggregation in

Table 4. Statistical analysis based on monthly and annual data aggregation.

Sub-Basin	Station	Monthly					Annual
		r	rBIAS	MBE (mm)	MAE (mm)	RMSE (mm)	r	rBIAS	MBE (mm)	MAE (mm)	RMSE (mm)
Southern UIB	Burzil	0.55	−0.28	−56.80	58.73	112.8	0.43	−1.05	−391.5	391.5	407.7
	Deosai	0.22	0.17	21.65	43.62	78.95	−0.37	0.24	146.6	201.4	234.6
	Rama	0.54	−0.36	−78.26	79.54	165.9	0.78	−2.05	−538.5	538.5	574.8
	Rattu	0.20	−0.20	−39.31	61.32	119.10	0.30	−0.58	−264.9	274.1	353.8
Eastern UIB	Shigar	0.02	−0.21	−19.42	36.56	78.26	−0.11	−0.62	−133.2	179.7	249.8
	Hushey	0.09	−0.24	−29.19	39.72	101.9	−0.15	−0.79	−193.2	230.3	339.6
Northwestern UIB	Khot	0.51	−0.21	−26.52	39.47	74.14	0.29	−0.65	−182.0	242.7	250.9
	Naltar	0.60	−0.31	−58.62	60.17	120.6	0.12	−1.32	−395.1	395.1	416.3
	Shendoor	0.42	0.08	6.53	22.08	37.34	0.54	0.13	40.4	66.4	99.1
	Ushkor	0.52	−0.03	−1.82	19.19	39.41	0.62	−0.05	−14.7	79.1	110.5
	Yasin	0.21	1.36	118.3	126.7	241.2	0.10	0.72	806.0	806.0	827.5
	Zani	0.49	−0.34	−75.05	77.98	154.91	0.52	−1.69	−508.3	508.3	532.6
Northern UIB	Khunjrab	0.39	0.05	2.26	13.13	23.40	−0.28	0.09	18.3	52.0	70.0
	Ziarat	0.38	−0.07	−5.39	15.23	32.62	0.55	−0.15	−30.6	73.4	104.7
Basin average		0.61	−0.23	−9.09	14.15	20.98	0.57	−0.24	−117.3	117.3	134.1
Maximum		0.60	1.36	118.3	126.7	241.2	0.78	0.72	806.0	806.0	827.5
Minimum		0.02	−0.36	−78.26	13.13	23.40	−0.37	−2.05	−538.50	51.96	69.96

, and for seasonal aggregation in

Table 5. Statistical analysis for summer and winter season data aggregation.

Sub-Basin	Station	Summer Season					Winter Season
		r	rBIAS	MBE (mm)	MAE (mm)	RMSE (mm)	r	rBIAS	MBE (mm)	MAE (mm)	RMSE (mm)
Southern UIB	Burzil	0.35	−0.48	−190.8	190.8	201.5	0.05	−0.50	−200.7	200.7	226.0
	Deosai	−0.42	0.27	68.1	101.0	123.5	−0.14	0.31	78.6	116.9	139.6
	Ramma	0.68	−0.54	−198.7	198.7	213.9	0.60	−0.92	−339.9	339.9	371.4
	Rattu	0.49	0.09	25.8	101.0	127.9	−0.01	−1.00	−290.8	290.8	357.1
Eastern UIB	Shigar	−0.09	−0.22	−36.9	106.0	156.6	−0.10	−0.57	−96.3	105.4	127.9
	Hushey	0.22	−0.35	−83.2	103.7	156.3	−0.55	−0.46	−109.9	132.2	189.3
Northwestern UIB	Khot	−0.11	−0.03	−4.1	38.1	48.3	0.33	0.16	22.2	29.2	34.8
	Naltar	0.48	−0.50	−203.8	203.8	220.6	0.17	−0.47	−191.6	191.6	205.9
	Shendoor	0.57	0.42	56.1	56.7	79.6	−0.02	−0.12	−15.9	61.1	65.6
	Ushkor	0.49	0.17	31.0	71.9	84.1	0.65	−0.25	−45.9	55.3	85.7
	Yasin	−0.35	2.94	593.6	593.6	615.2	0.22	1.05	212.5	212.5	244.1
	Zani	0.52	−0.66	−268.8	268.8	301.3	0.57	−0.59	−239.8	239.8	256.1
Northern UIB	Khunjrab	0.09	−0.21	−41.3	91.8	109.2	0.56	−0.72	−141.1	157.5	166.9
	Ziarat	0.56	−0.02	−2.5	36.2	47.4	0.54	−0.20	−28.4	44.8	65.3
Basin average		0.60	0.00	−5.18	30.1	38.7	0.36	−0.4	−96.51	96.5	109.1
Maximum		0.68	2.94	593.6	593.6	615.2	0.65	1.05	212.5	339.9	371.4
Minimum		−0.42	−0.66	−268.8	36.2	47.4	−0.55	−1.00	−339.9	29.2	34.8

. The summer seasons include the months of April, May, June, July, August, and September, while the remaining 6 months, which are October, November, December, January, February, and March, are aggregated to represent the winter season.

It is evident from a first glance at the two tables that the TRMM performs overall rather poorly for estimating the observed rain amounts for the study region at all temporal scales, as the average r values are only 0.16, 0.22, 0.22 and 0.20 for monthly, annual, and seasonal (summer and winter) aggregation, respectively. Further specific results are discussed in the subsequent sub-sections.

3.1.1. Skill Statistics for TRMM Precipitation Estimates (Daily Aggregates)

The daily aggregates of the TRMM precipitation estimates show poor skill in matching observed precipitation, with an average r of only 0.16 (Table 3). The comparison of the observed and TRMM daily rainfall data indicates highly variable MAE values across the UIB, with a range ≥23 mm/day (Table 3). Values of MAE are high throughout most of compared locations in the UIB, with MAE ≤ 13 mm for all stations across the UIB. The northwestern parts of the UIB show the highest and most variable MAE.

Based on the values of the MBE in Table 3, one can notice that the TRMM data have huge underestimations across most of the UIB (average MBE of −3.53 mm), while they show at the same time a distinct spatial pattern across the study area, with a clear underestimation of the TRMM estimates for all the studied locations in the eastern and northern UIB, as well as for the southern UIB at all locations except one. On the other hand, the stations located in the northwestern UIB experience a mixed trend, with TRMM data indicating a moderate to high under- or overestimation at half (three) of the locations each.

The mean relative bias (rBIAS) at the different gauge locations also follows a similar pattern, with huge variations and range from a negative −0.42 to a high of +5.27 at station “Yasin”. The TRMM precipitation estimate is thus more than 5 times the gauge-based observed value, which means that there is tremendous overestimation of the former.

For the relative bias rBias, the TRMM estimates show underestimation only at five locations, unlike for the mean bias MBE discussed before, where 10 out of the 14 locations display underestimations. After a more detailed examination of both the TRMM- and observed time series, it was found that, at some locations, very large number of light precipitation events were generally overestimated, while a smaller number of heavy precipitation events were underestimated by the TRMM estimates. This behavior may have resulted in the overall average positive rBIAS at most of the locations, discussed earlier, while in reality the overall mean bias MBE of the TRMM estimates is negative (underestimation) at more than two-thirds of the locations.

The RMSE values for the daily time series are also very high and show large variations, ranging from as low as 8.08 mm/day to as high as 46.57 mm/day, with an average basin-wide value of RSME = 20.88 mm/day.

These results are in general agreement with previous studies [23,42], as most of them have reported the TRMM product to underestimate the gauge-based rainfall amounts over the HKH region in general and, even more so over the western parts of HKH.

3.1.2. Skill Statistics for TRMM Precipitation Estimates (Monthly and Annual Aggregates)

The skill statistics of the monthly and annually aggregated TRMM precipitation estimates listed in Table 4 shows overall also poor skill in matching the observed ground-based precipitation, but with considerably improved values for the Pearson correlation coefficient r for all the studied locations, namely, with r values of 0. 61 and 0.57 for the basin average rainfall for monthly and annual aggregates, respectively.

The MAE for these long-term aggregated precipitation data is also highly variable across the UIB, ranging from 13.13 mm/month to 126.68 mm/month for the monthly aggregates and from −538.5 mm/year to 806.0 mm/year for annual aggregates. The values for MAE are generally high throughout most of the locations in the UIB, with a basin-wide average of 14.15 mm/month and 117.3 mm/year for monthly and annual aggregated rainfall, respectively. The spatial pattern of the errors observed in both cases as well as the predominant underestimations observed at most locations is similar to that observed for the daily aggregates above. Thus the northwestern parts of the UIB show the highest and the most variable MAE.

The TRMM estimates are also significantly underestimated across most of the UIB, with and average basin-wide MBE of −9.09 mm/month and −117.3 mm/year for monthly and annual aggregated rainfall, respectively. The eastern part of UIB shows a distinct underestimation of the TRMM rainfall for all stations there and this underestimation is even higher in the southern UIB, where three out of the four locations experience this effect, while at one location (Deosai), an overestimation of 21.65 mm/month for the monthly aggregate is observed. The stations located in the northwestern UIB have a mixed trend, with moderate-to- high underestimations at four locations and an opposite behavior in the remaining two, and this holds for both monthly and annual aggregates. For the northern UIB, for the MBE of the two locations evaluated there have also been mixed results, with one station (Khunjrab) indicating a slight overestimation with MBE of 0.05 mm/month and 18.3 mm/year, and the other (Ziarat) an underestimation with a negative MBE of −0.07 mm/month and −30.6 mm/year for monthly and annual aggregated rainfall, respectively.

3.1.3. Skill Statistics for TRMM Precipitation Estimates (Seasonal Aggregates)

The various statistical indices of the two seasonally (summer and winter) aggregated TRMM estimates are shown in Table 5.

One may notice from the table that the various skill indices have comparable trends to those of the monthly- or annually aggregated TRMM series, in terms of magnitude, but show a different pattern for the two seasons. For example, for summer, the TRMM estimates show positive rBIAS for a few locations where the monthly and annual aggregates earlier indicated a negative one (i.e., stations Rattu, Ushkor). In contrast, for the winter season predominantly negative rBIAS values are obtained, similar to those of the monthly and annual aggregations.

The overall range of the MBE for the stations evaluated varies from −268.8 mm to 593.6 mm for the summer season and from −339.9 mm to 212.5 mm for the winter, with an average MBE across the UIB of −5.18 and −96.51 mm for summer and winter, respectively. These comparatively lower MBE values for the summer season are indicative of a situation where the under- or overestimation occurring in the different months of the season, cancel each other partly out.

The correlations coefficient r range from −0.42 to 0.68 and −0.55 to 0.65 for the summer and winter seasons, respectively, with the average r for the UIB having better values for the summer- (r = 0.6) than for the winter (r = 0.36) season, most likely due to the fact that the precipitation is much higher in summer than in winter.

3.2. Categorical Statistics

The results of the TRMM-precipitation analysis for the six categorical indices described in Section 2.2 are listed in

Table 6. Categorical statistics for daily TRMM estimate and gauge rain data. Ac: accuracy; FBI: frequency bias index; POD: probability of detection; FAR: false alarm ratio; CSI: critical success index; TSS: true skill statistics.

Sub-Basin	Station	Ac	FBI	POD	FAR	CSI	TSS
Southern UIB	Burzil	0.57	0.75	0.42	0.45	0.31	0.12
	Deosai	0.57	1.01	0.61	0.39	0.44	0.14
	Ramma	0.55	1.27	0.50	0.61	0.28	0.08
	Rattu	0.51	1.48	0.56	0.62	0.29	0.04
Eastern UIB	Shigar	0.60	1.30	0.41	0.68	0.22	0.08
	Hushey	0.57	0.83	0.40	0.52	0.28	0.10
Northwestern UIB	Khot	0.65	1.34	0.57	0.58	0.32	0.25
	Naltar	0.62	0.86	0.40	0.53	0.28	0.14
	Shendoor	0.56	1.16	0.40	0.65	0.23	0.04
	Ushkor	0.61	1.07	0.42	0.60	0.26	0.12
	Yasin	0.67	1.03	0.44	0.58	0.27	0.20
	Zani	0.55	0.86	0.35	0.59	0.24	0.03
Northern UIB	Khunjrab	0.55	1.04	0.43	0.58	0.27	0.06
	Ziarat	0.60	0.73	0.35	0.52	0.25	0.10
Basin average		0.58	1.05	0.45	0.56	0.28	0.11
Maximum		0.67	1.48	0.61	0.68	0.44	0.25
Minimum		0.51	0.73	0.35	0.39	0.22	0.03

. Basically these indices show how the TRMM data match the ground-based gauge data at daily time scales.

Thus, the values for the first index, accuracy (Ac) are well above 0.50 for all stations, with an average of 0.58.

The frequency bias index FBI has neither very high positive nor negative values, but varies on both sides, with nine stations showing overestimation, and the remaining five underestimation. The basin-wide average FBI is 1.05, which indicates a slight overestimation of the TRMM rainfall, vindicating the results of the general statistical skill analysis of the previous sections.

The other categorical indices (see Equations (8)–(11)) do not show very good results either. Thus, for most of the stations the values of the probability of detection (POD) is below 0.5, with only four stations having values above it. The false alarm ratios (FAR) for all stations but one are generally too high, with a basin-wide average of FAR = 0.56. In the same way, both the CSI and the TSS values are also not very promising, as only three stations have values above 0.30 for the former and only one station a value of about 0.20 for the latter.

Thus, overall, these results of the categorical statistics indicate that the TRMM rainfall estimates in the UIB do not have a good match with the ground-based gauge data and, therefore, should only be used after some corrections and adjustments have been made.

3.3. Visual Comparison

For visual comparison, monthly, annually, and seasonally aggregated time series of the TRMM rainfall estimates and of the various gauge stations are plotted.

Figure 2 and Figure 3 show these time series plots for two stations, Yasin and Khunjrab, respectively. One may notice from these plots that for station Yasin (Figure 2) huge biases and errors at all three time scales considered are obtained, whereas for station Khunjrab a better match, especially at the annual and seasonal aggregations, is obtained. The corresponding plots for the others stations analyzed reveal a somewhat similar pattern.

The monthly aggregated TRMM- and gauge observed rainfalls averaged over all stations and the full length of period considered (1998–2008), are plotted in Figure 4. The figure also has a demarcation of the seasons. From the figure a systematic underestimation of the TRMM rainfall in the winter months and a mix of under and overestimation in the summer months can clearly be seen, corroborating to a large extent the statistical results of the previous sections.

Finally, the monthly, annually, and seasonally aggregated time series of both the averaged TRMM- rainfall estimates and of the observed gauge data are plotted in Figure 5. Though there is almost a persistent underestimation of the TRMM estimates, the peaks and troughs of both series follow, in most instances, similar patterns.

4. Discussion and Conclusions

In this study, data from a TMPA product—TRMM 3B42 V7—for the Upper Indus Basin (UIB), Pakistan over the period 1998–2008 has been assessed and evaluated on a point-to-point basis by comparison with rain gauge data from 14 stations. These assessments have been performed at monthly, seasonal, and annual aggregation scales. The results indicate that the TMPA product has considerable errors in estimating the rainfall amounts at the various gauge stations throughout the study area and throughout the total time period studied. There is a predominant trend of underestimation of the TRMM product across the UIB at most of the gauge stations; TRMM-estimated rainfall is generally lower than the gauge-measured rainfall. The seasonal TRMM rainfall, though, shows a specific pattern, with the summer rainfall slightly overestimated, and winter rainfall predominantly underestimated at almost all locations and all aggregation time scales.

These results conform overall with those of previous studies, which, in most cases, suggest that neither the sparsely observed station data and gridded data products based on them, nor the sensors-based data fully represent the precipitation regime of the region [42], with strong non-representation or underestimation [16] of regional precipitation amounts, especially for higher altitudes by [20,22,42]. In fact, the in situ meteorological observations in the UIB are sparse and mostly taken at valley-based stations. These data provide low spatial coverage and are scant for higher altitudes. Furthermore, the complex orography of the UIB region also affects the amounts, spatial patterns and seasonality of the precipitation. Additionally, most of the authors [20,21,22,23,42] indicated that the observation network across the UIB also shows underestimation of precipitation amounts, with an average of around 166%, reaching in excess of 300% over some parts of the basin [23]. This means that the TRMM product may even be underestimating the true areal precipitation by a much greater margin, as the true areal precipitation is estimated to be much higher [23] than the gauge observation records.

The comparison of any gridded or sensor-based datasets with observed precipitation may not be taken, therefore, as a conclusive evidence for declaring the evaluated data is unappropriated in terms of usability, but rather shows the degree to which these data sets match the magnitudes or occurrences of the observed precipitation, which by no means is perfect, and a better match may also indicate that the evaluated data have tendencies to underestimate the real areal precipitation over the UIB. Furthermore, the spatial resolution of the TRMM product (0.25° × 0.25°), may also pose limitations, especially for distributed hydrological modeling and investigations [23,43], as at this resolution, the orographic influences on the precipitation regime cannot be mapped, while the hydrological models usually require precipitation data at a much finer scale.

The main conclusion to be drawn from this study may be then resumed as follows:

(1)

The TRMM-3B42-V7 product has an overall poor agreement with the observed rainfall gauge data in the study area, and this holds for all temporal scales considered.

(2)

The results, eventually, mean that the TMPA TRMM-3B42-V7 product may only be regarded as suitable for further rainfall analyses and subsequent hydrological applications [23,43] in the study region if some improvements, down-scaling, and local calibrations of its output data are carried out first.