1. Introduction
Reliable climate change scenarios are needed in assessing the potential impacts of climate change in different socioeconomic sectors, as well as in identifying suitable adaptation measures. Global Climate Models (GCMs) and Regional Climate Models (RCMs) are used to derive future scenarios of different hydro-climatic variables. RCMs are used to dynamically downscale coarser resolution climatic variables into finer resolution ones and to better represent meso-scale climatic phenomena. These models simulate complex interactions between atmospheric, oceanic and land processes, and as such exhibit a lot of uncertainties in projected climates. The regional models can add value to GCM projections, but only for certain variables and locations [
1] as RCMs develop their own climatology which is not consistent with the driving global datasets of precipitation [
2]. The downscaling strategy also introduces uncertainties associated with model physics [
3]. The choice of model domain is important in applying an RCM, since a sub-optimal domain may not represent regional climate realistically [
4,
5,
6,
7]. The optimum domain size may also vary from place to place depending on local driving factors. With the increase in spatial and temporal resolutions of the regional models, the uncertainties are expected to increase. However, these spatial and temporal scale dependencies of the models have not been systematically investigated in climate change studies.
Providing REgional Climates for Impacts Studies (PRECIS) is an RCM developed by the Hadley Center of the United Kingdom. It has 19 vertical levels in the atmosphere and obtains its lateral boundary conditions from a coupled atmosphere–ocean general circulation model, HadCM3 [
8,
9,
10]. A PRECIS model description is provided in detail in
Supplementary Section S1. Since the release of PRECIS, the model has been extensively used in different parts of the world to generate climate change scenarios. It appeared from the review of published literature (see details in
Section 2) that, though the PRECIS model has been used to project the monsoon rainfall over Bangladesh, India, Nepal and Bhutan, there has not been any systematic study checking the reliability of the model. The only check that has been done is a comparison of aggregated (both spatial and temporal) rainfall between the observation and model simulation. It has been frequently reported in the literature that there is a bias in simulated rainfalls [
11]. It is likely that this bias in the projected (annual, seasonal and monthly) rainfalls would adversely affect the projection of rainfall extremes, such as annual maximum, 99th percentile and 95th percentile. In fact, if a model including the PRECIS fails to simulate the base climate adequately, its projected climates, particularly projected rainfall extremes, are expected to be uncertain. Therefore, with continued and increasing use of the model, more stringent tests, such as frequency distribution of quantities over different spatial scales, even on a specific grid, are warranted [
12].
In this study, the performance of the PRECIS model in simulating the rainfall over Bangladesh is evaluated. The selection of Bangladesh as a case study area is guided by the fact that (a) the country is identified as one of the most vulnerable countries to climate change impacts, and (b) a number of vulnerability studies are currently being conducted for the country using projected climate data (for example, the Ecosystem Services for Poverty Alleviation [ESPA] project funded by the UK′s Department for International Development [DfID], and the DEltas, vulnerability and Climate Change: Migration and Adaptation [DECCMA] project funded jointly by DfID and Canada′s International Development Research Center [IDRC]; both the ESPA and DECCMA projects are using RCM projection).
The paper is organized as follows. In the next section, the current state of the research field on PRECIS model is reviewed. The study area, methodology and the data collected and used in the study are described in
Section 3. The climate model used and the statistical analyses employed are also described in this section. The findings of the study—particularly the comparison between observed and model-simulated rainfalls at various spatial and temporal scales—are then discussed in
Section 4. Finally, some conclusions are drawn in the final section of the paper.
2. Literature Reviews
Since the release of PRECIS, the model has been widely used to generate climate change impact scenarios for South Asia including Bangladesh. For example, Islam et al. [
13] and Islam [
14] generated rainfall and temperature scenarios for Bangladesh by running the model at a horizontal resolution of 50 km. It was reported that the model overestimated the pre-monsoon season rainfall and underestimated the monsoon season rainfall and, therefore, the model-generated rainfall could not be used directly in application purposes. A linear regression technique between simulated and observed rainfalls was also proposed there, to adjust the model-generated rainfall. Nowreen et al. [
15] studied the changes in rainfall extremes in the northeastern part of Bangladesh using the PRECIS generated rainfalls. However, the inherent limitation of the PRECIS model is also apparent from the results therein. The model had difficulty in capturing important topographical and physical processes responsible for precipitation patterns at the regional scale [
10]. For example, the simulated rainfalls of the model from March to October were lower than the observed rainfalls indicating that the model underestimated the rainfalls in these months. The underestimation was also higher for the months having higher rainfalls. This is in line with the observations in [
16] on the underestimation of rainfall by the PRECIS as a model for Pakistan. Rajib [
4] studied the changes in monthly rainfalls in three time periods (i.e., 2011–2040, 2041–2070 and 2071–2100) and compared those with the observed rainfall of a base period (1971–2000) over Bangladesh. The monthly rainfalls for future periods were obtained from four GCMs and the PRECIS outputs at the country level as 30 years’ average. Though the PRECIS outputs were not reported separately, it was reported that the rainfalls in all the months would increase in the future.
The PRECIS model was used to develop high resolution climate change scenarios over India [
17]. Here, the model was found to be overestimating all-India summer monsoon rainfall and underestimating the variability. Furthermore, there was significant positive bias in the rainfall during the onset phase of the monsoon (May and June). Revadekar et al. [
18] studied precipitation extremes over India based on a comparison of baseline and future simulations of PRECIS. The authors, however, did not report on the performance of the model in simulating baseline extremes, which could have been done by a comparison of the observed and simulated rainfall extremes during the base period. Projections of the summer monsoon rainfall over India were made in [
19] with the PRECIS simulation. Here, it was found that the model could simulate the 30-year average rainfall magnitude well, but it underestimated the seasonal rainfall variability. The latter indicates that there is a smoothing tendency in the rainfall response in the model. The precipitation over the Bay of Bengal region was also reported as being overestimated. The spatial pattern of the projected number of rainy days was found to differ among the ensembles, suggesting uncertainties in the projections.
The uncertainties in the two regional climate model simulations (RegCM4 and PRECIS) of the South Asian summer monsoon were evaluated in Syed et al. [
2]. Their results show large differences in rainfalls between the two RCMs and the observations. The inter-annual variability over the six sub-regions studied, which included Bangladesh, was not well captured during the summer monsoon. They also mentioned that both GCMs and RCMs were impacted by uncertainties that weakened the confidence in end projections, and thereby limited the usefulness thereof in planning and policy decisions. Zacharias et al. [
10] reported that the PRECIS generated rainfalls across India had a bias of +34% to −89% in the annual scale, indicating ‘poor performance of the model in simulating the baseline rainfall patterns’. The model also simulated more rainy days in the baseline period than those observed. The PRECIS simulation was found to capture the shape of the annual rainfall cycle over the western and central Himalayas [
20]. However, for the eastern Himalayas, where Bangladesh is located, the peak of the rainfall hydrograph was simulated two months earlier in May or June. A wet bias in model simulation was also reported for the three regions.
3. Study Area, Methodology and Data
Bangladesh is extremely vulnerable to climate change and, in fact, it is one of the most vulnerable countries in the world [
21]. Having a huge population (about 180 million) spread over a small geographic area (about 147,000 km
2), the country is located at the confluence of the three mighty rivers—the Ganges, the Brahmaputra and the Meghna—and at the northern margin of the Bay of Bengal. It experiences a tropical monsoonal climate with distinct dry and wet seasons. Mean annual rainfall varies between 1500 to 4400 mm (
Figure 1). About 80% of the country is floodplain and 33% is in the coastal zone with a low altitude. The major livelihood activities of the country are dependent on natural resources. The economic condition of the country is poor and there is wide-spread poverty and unemployment. All these factors have made the country vulnerable to different hydro-climatic hazards. Floods, cyclones, river erosion, soil and water salinization, groundwater contamination with arsenic, and waterlogging are among the major hazards affecting the country. Climate change led by global warming is expected to further aggravate the situation. To assess the potential impacts of climate change on different socioeconomic sectors and to plan for possible adaptation and mitigation measures, the reliable projection of climatic variables for the country is essential.
The reliability of the PRECIS model in simulating rainfall of Bangladesh was assessed by employing a number of statistical techniques. Among the techniques, a pattern similarity statistic suggested in Santer et al. [
22] was used to compare the observed time-evolving change patterns of a climatic variable with the model-predicted signal patterns. Such pattern/map correlation between the model-simulated and observed rainfalls was computed for a base period of 1986–2005. The same period was used in the fifth assessment report of the Intergovernmental Panel on Climate Change (IPCC) [
23] as the base period. The pattern correlation was computed for annual, seasonal, monthly, annual maximum, 99th percentile and 95th percentile rainfalls. The pattern correlation method was used by IPCC in its second and third assessment reports. From the 20 yearly values of a variable corresponding to a model grid point, a grid point average value of the variable was obtained. The pattern correlation was obtained from the different grid point average values between the model and observed series.
In addition to the spatial-pattern correlation, temporal correlation, root mean square difference (RMSD), the mean absolute difference (MAD) and probability density functions (PDFs) between the model and observed rainfalls were computed to check the appropriateness of the model projections. RMSD and MAD were expressed in percentages by dividing those with the corresponding observed mean rainfalls. For each model grid, the yearly rainfall time series was generated first for each rainfall variable (annual, monthly or extreme rainfall), and then the presence of temporal trends in each variable was investigated and, where present, it was removed by detrending. The detrending was done by subtracting and adding the respective rise and fall of the trend line above and below a horizontal base line from the observed or simulated rainfall. The detrending was done to avoid spurious correlation solely due to the presence of trends in the series. The correlation between the detrended model and observed rainfalls was then computed. The correlation facilitates a better evaluation of the capacity of a climate model to reproduce the observed inter-annual variability [
24]. The inter-annual correlation/variability between climate model-simulated and observed rainfalls was also investigated in earlier studies [
2,
10,
24,
25,
26,
27]. The statistical significance of the correlation was determined by applying the standard Student’s t-test. In addition, PDFs were calculated for rainfall indices namely, annual, monthly and annual maximum between observed and modelled simulations over the base period at different locations/grids near 28 BMD stations.
The simulated daily rainfall was collected from the Met Office, UK for a period of 1971–2099 at an approximate horizontal resolution of 0.22° × 0.22° (25 km × 25 km). The data were generated from the PRECIS regional climate model driven by atmosphere–ocean coupled HadCM3 boundary conditions. The basic aspects, such as dynamical flow, the atmospheric sulphur cycle, clouds and precipitation, radiative processes, the land surface and the deep soil, are explicitly handled by the model and are described in Jones et al. [
12]. The validation of the model is done by Caesar et al. [
28]. The HadCM3 simulation used here is from the standard original parameter settings in the atmospheric component of the model. The simulation utilizes observed greenhouse gas forcing for the historical period and A1B scenario of the Special Report on Emissions Scenarios (SRES) for the future period. The SRES A1B scenario [
29] represents a medium-high emissions scenario which assumes a future of strong economic growth leading to an increase in the rate of greenhouse gas emissions with no explicit climate change mitigation policy. The atmospheric carbon dioxide equivalent concentration rises throughout the 21st century to around 900 ppm by 2100. This scenario provides overlap and consistency with much of the existing climate modeling work, and is fairly consistent with observed carbon emissions since the 1980s [
30,
31]. IPCC has used one ensemble from each model in its latest 5th Assessment Report [
23].
Measured daily rainfall data were collected from the Bangladesh Meteorological Department (BMD), Dhaka. BMD is the sole government organization which measures and maintains credible climatic data. There are 28 observational stations of BMD within the study area of about 147,000 km
2. Daily rainfall monitored at all 28 gauges of BMD successfully passed the homogeneity test by various literatures [
32,
33,
34,
35]. Notably, for all 28 BMD stations Khan et al. [
34] found almost straight lines with no breakpoint in the double mass curves and Mahmud et al. [
35] observed homogenous using Standard Normal Homogeneity test, Von Neumann Ratio test, Buishand Range test and Pettitt test. Thereby, the BMD rainfall data are considered reliable and is widely used in scientific studies [
36,
37,
38] and government documents. We used data from all 28 stations within the study area for the purpose of generating a baseline map, and for the quantitative comparison of observed and model-simulated rainfalls. A point-to-point comparison was made between the two data sets and the results are discussed in the next section. The model grid in which a BMD station was located was considered as a point corresponding to that station [
14].
4. Results and Discussion
The average annual rainfall over the study area for the base period (1986–2005) was found to be 2025 mm from the model simulation; the same was found to be 2535 mm from the observed data. Thus, the average rainfall from the model simulation was about 20% less than the observed rainfall. The spatial distribution of both observed and model-simulated mean annual rainfalls are given in
Figure 1. It is noticed from the figure that the low rainfall domain (1500–2000 mm) is larger in the model-simulated rainfall as compared to that of the observed one. Thus, the low rainfall areas were overestimated in the model. Furthermore, the high rainfall domain (3900–4400 mm)—particularly in the southeastern part—in the simulated rainfall was lower than that in the observed rainfall. Thus, there is a dry bias in the simulated rainfall over Bangladesh in the model. Similar findings were also reported in [
14]. This could be due to the poor representation of the topographical and physical processes responsible for monsoon precipitation in the model [
10]. In the case of pattern correlation in mean annual rainfall between the observed and simulated rainfalls, a high correlation coefficient of 0.78 was found. This high correlation may be spurious and is probably due to the similar spatial variation pattern in the two data sets. The MAD and RMSD were found to be 23% and 28%, respectively.
The point-to-point scatter plots and correlations between the observed and corresponding simulated annual rainfalls are given in
Supplementary Figure S1 and
Table 1, respectively. It is seen from the
Table 1 that the correlation coefficient varies widely—from −0.52 at Chittagong to +0.61 at Patuakhali. Furthermore, it is evident that the correlation coefficients are relatively small except for Patuakhali. The MAD and RMSD of the simulated rainfalls vary from 16%–45% and 20%–51%, respectively. Thus, there is a large error in simulated annual rainfall over the study area.
Comparisons of probability density functions (PDFs) between the observed and simulated rainfalls over the base period at different locations of the study area were also made (
Supplementary Figure S2). Such comparisons for two locations (Dinajpur in the northwest and Mymensingh in the north-central Bangladesh) are shown in
Figure 2 as an example. It is seen from the figure that neither the location parameter nor the shape parameter of the observed rainfall is captured by the simulated rainfall. The location parameter is underestimated in model simulation and the shape parameter always has a higher peak and smaller base in simulated rainfall. Thus, the model fails to capture the spread of observed rainfalls and provides rainfalls within a narrow range. Furthermore, the PDFs from the observed rainfalls are almost symmetrical, whereas that from the simulated rainfalls are either positively (Dinajpur) or negatively (Mymensingh) skewed.
The performance of the model in simulating monthly rainfalls was assessed. The assessment was made only for those months which have monthly rainfalls greater than zero in all the years. The results are shown in
Figure 3 (and also reported in
Supplementary Table S1) which shows that the model fails to capture the monthly rainfall characteristics at individual stations as the correlation coefficients are often low or negative, and MADs and RMSDs are high. As an example, the correlation coefficients are negative for the month of September at 21 (out of 28) stations in the study area. The other stations also had lower values of correlation coefficients. This indicates that the model is not able to capture the inter-annual rainfall variability in September, the last month of the monsoon season. This could be due to the failure of the model in capturing the monsoon recession from Bangladesh adequately. The MAD varies from 21%–78% and RMSD from 44%–93% for the month of September. The inter-annual variability of rainfall is poorly captured for the months of July and August of the monsoon season. However, it is interesting to note that the pattern correlations are quite high for these months. For example, the pattern correlations for the months of June, July and August were found to be 0.88, 0.69 and 0.67, respectively, which are very high. Thus, it appears that the pattern correlation may not be a good indicator to judge the performance of a climate model. This was also echoed earlier in [
39].
Comparisons of the PDFs between observed and simulated rainfalls for different months at different locations were also made. The results were more or less similar for different months. As an example, the comparison for the month of June for Rangpur in the northwest and Patuakhali in south-central Bangladesh indicated that there was a negative bias in scale parameter and a positive bias in shape parameter (
Figure 4). The areal average observed rainfall in June was 491 mm and simulated rainfall was 404 mm. Thus the simulated rainfall was grossly underestimated by the model.
A further comparison was made between the observed and simulated annual maximum daily rainfalls with their PDFs (
Supplementary Figure S3). The results are shown in
Figure 5 for Sylhet in the northeast and Dinajpur in the northwest Bangladesh. It is seen from the figure that the shape parameter of the PDF for Sylhet and the scale parameter of the PDF for Dinajpur are poorly simulated by the model. The underestimation of the scale parameter was found to be almost universal throughout the study area. This indicates that the annual maximum rainfall was underestimated by the model. In fact, the average annual maximum observed rainfall was found to be about 170 mm and that of the model-simulated rainfall was found to be about 138 mm. The underestimation is also evident in the double mass curves of annual maximum rainfalls (
Figure 6). The MAD between the observed and simulated maximum rainfalls was found to vary between 35% and 69%, and the RMSD between 44% and 93% depending on the locations. Thus, the error in simulated annual maximum rainfall was very high. More importantly, the correlation coefficients were negative in 17 out of 28 locations for which observed data were available. The other 11 stations also had very low values. It thus appears that the uncertainty in simulated annual maximum rainfall is higher than that of annual average rainfall.
The consistency—and hence reliability—of the model results were also assessed by comparing the simulation results among different future time periods. The results for mean annual rainfall are shown in
Figure 7. It is seen from the figure that the rainfall is projected to be mainly increased during the period of 2021–2040. The major increase is expected to be over the Chittagong region. The pattern of increased rainfall is also evident for the period of 2041–2060. However, there is a shift in the zone of maximum change. For 2021–2040, the zone of maximum change is near the south-east side, whereas for 2041–2060, it is almost all over Bangladesh. The reason for this shift in the zone is not understood. Moving to the period of 2061–2080 in the figure, we see that a decrease in rainfall manifests over the entire region. The same is also true for the period of 2081–2100. Thus, one can deduce that the rainfall would decrease in the long term. However, it is not clear whether there is any physical basis of this decrease or if it is simply the result of the incapability of the model to simulate the rainfall.
The spatial distribution of the linear trends and their statistical significance is given in
Figure 8. It is seen from the figure that the majority of the study areas have positive trends in annual rainfalls. The trends are negative only towards the northern margin and southwestern coastal zone. The
p (significance level) values are >0.05, which indicates that the trends are not statistically significant at 95% confidence level.
The spatial distribution of the change in the 95th percentile rainfall (
Figure 9) indicates that the change is entirely positive for the period of 2021–2040, mainly positive for 2041–2060, mainly negative or weakly positive for 2061–2080, and very weakly positive for 2081–2100 except for the coastal region. This temporal change in the sign is difficult to understand physically and may simply be due to the inadequacy of the model in capturing the important rainfall processes and features of Bangladesh. Overall, the trend is found to be small with its
p value ˃0.05. The same analyses for the annual maximum rainfall, 99th percentile, 90th percentile and 80th percentile were carried out and the results were more or less similar to the above results in relation to the 95th percentile.
As discussed earlier, there is a high pattern correlation between the observed and simulated rainfalls of the base period, but the correlation coefficient is small and sometimes negative at individual locations. This indicates that the model captures the overall spatial pattern of temporal mean rainfall, but fails to capture the inter-annual variability of rainfall. To see if this failure is due to the spatial resolution of the model, the PDFs of annual rainfalls were estimated for each cell, for averaged rainfalls of five cells and for averaged rainfalls of nine cells. A visual comparison of the one-, five- and nine-cell PDFs (not shown) indicated that there was no significant difference among the three types of PDFs. This indicated that even after aggregation, the inter-annual variability could not be captured. This is further evident from the double mass curves of one-, five- and nine-cell average annual rainfalls (
Figure 10). Thus, it appeared that there was no added benefit of spatial aggregation of rainfall in terms of producing inter-annual variability. Inter-annual variability had not been captured in the simulated rainfall. The problem could be due to the poor boundary conditions of the PRECIS model obtained from the driving GCM, or due to the internal structure of the PRECIS, which fails to capture the dynamic characteristics of the rainfall over Bangladesh. These aspects are worthy of investigation in future research.
Since the model failed to capture the variability in rainfall at the annual time scale as well as at the monthly time scale (discussed earlier), it is highly unlikely that it had captured the rainfall extremes. This is also evident from
Figure 11, which shows the PDFs of the rainfalls of two nearby grid cells in the central coastal region of Bangladesh. While the PDF for one grid cell in
Figure 11 has the lowest peak in the 2070s, the PDF for another nearby grid cell has the same lowest peak in the 2050s. The same is also true for the right tails. Thus, any projection of future rainfall extremes based on PRECIS-simulated projection is likely to be highly uncertain.
Our findings suggest that there are large uncertainties in simulated baseline and future rainfalls over Bangladesh when using the PRECIS model. Perhaps this is true for other climate models and other regions as well. The uncertainties could have stemmed from the poor representation of various complex physical, chemical and biological processes and their interactions in the model itself, the input data used to run the model, driving GCM boundary conditions, etc.; however, we did not attempt to assess the sources of these uncertainties. The uncertainties in simulated rainfall grow as the spatial and temporal scales become finer. However, finer data are needed in climate change impact studies involving hydrologic, hydrodynamic, agricultural, ecological, and other models. The PRECIS outputs at the present state are not reliable, particularly in representing rainfall variability and temporal correlation at a finer scale (e.g., daily). Furthermore, there is a bias in the model-simulated rainfall over Bangladesh. Thus, an improvement in the model skill is warranted before using it in practical decision making. One way to achieve this is to involve local scientists in model development so that local context is better understood and captured, and to incorporate locally available observed data in model calibration and validation processes. In the meantime, other potential options to deal with uncertainties could be adaptive water management, institutional capacity development, the development of robust and ‘no regret’ strategies, reliance on multiple models, downscaling methods and scenarios, etc. [
40,
41].