Assessing the Performance of CMIP5 Global Climate Models for Simulating Future Precipitation Change in the Tibetan Plateau

Abstract

In this study, the performance of 33 Coupled Model Intercomparison Project 5 (CMIP5) global climate models (GCMs) in simulating precipitation over the Tibetan Plateau (TP) was assessed using data from 1961 to 2005 by an improved score-based method, which adopts multiple criteria to achieve a comprehensive evaluation. The future precipitation change was also estimated based on the Delta method by selecting the submultiple model ensemble (SMME) in the near-term (2006–2050) and far future (2051–2095) periods under Representative Concentration Pathways (RCP) scenarios RCP4.5 and RCP8.5. The results showed that most GCMs can reasonably simulate the precipitation pattern of an annual cycle; however, all GCMs overestimated the precipitation over TP, especially in spring and summer. The GCMs generally provide good simulations of the temporal characteristics of precipitation, while they did not perform as well in reproducing its spatial distributions. Different assessment criteria lead to inconsistent results; however, the improved rank score method, which adopts multiple criteria, provided a robust assessment of GCMs performance. The future annual precipitation was projected to increase by ~6% in the near-term with respect to the period 1961–2005, whereas increases of 12.3% and 16.7% are expected in the far future under RCP4.5 and RCP8.5 scenarios, respectively. Similar spatial distributions of future precipitation changes can be seen in the near-term and far future periods under the two scenarios, and indicate that the most predominant increases occurred in the north of TP. The results of this study are expected to provide valuable information on climate change, and for water resources and agricultural management in TP.

1. Introduction

The Tibetan Plateau (TP), also referred to as the “Third Pole” of the world [1], is the highest plateau in the world. It has an average elevation of more than 4000 m and features complex terrains [2]. TP is the source of major rivers in Asia, and as such is also referred to as the “Water Tower of Asia” [3]. Owing to its unique geographical characteristics, TP plays an important role in the East Asian and global atmospheric circulation through dynamic and thermal processes. On the other hand, TP is identified as a region that is sensitive and vulnerable to climate change. It is experiencing a warming trend [4,5,6], which has profound impacts on the hydrological cycle and the ecological system. Therefore, it is imperative to understand how climate will change in TP in the future.

Precipitation is a key climate variable in the global climate system and has an important impact on the hydrological cycle and the ecological system [7,8,9]. Scientific evidence has shown that precipitation patterns change in a warmer climate, and the likelihood of extreme precipitation is expected to increase due to global warming [10,11,12,13]. Thus, understanding the effect of climate change on precipitation has a great significance, as such information can be important for devising strategies to cope with such change.

Global climate models (GCMs) have been valuable tools for assessing climate change and produce climate projections [14]. The Coupled Model Intercomparison Project 5 (CMIP5) has provided numerous GCMs, which have more complicated mechanisms than before, and incorporate carbon cycle models and a dynamic vegetation module [15]. Many studies have found that the CMIP5 models perform better than the CMIP3 ones, have a smaller bias, and an overall improved capability of simulating climate change [16,17,18,19]. The same results were also demonstrated in characterizing the climatology of the East Asian monsoon [20,21]. However, CMIP5 models are still affected by major bias in simulating the magnitude and trend of precipitation, especially in regions with complex topography [22,23].

Many studies have been conducted to assess the performance of precipitation simulations of the CMIP5 GCMs using various methods in different regions [24,25,26,27,28,29]. Fu et al. [30] evaluated the performance of 25 GCMs using a score-based method in southeastern Australia. Das et al. [31] adopted a Multiple Imputation Chained Equation technique to cope with the limited data availability and poor data quality in the Western Himalayan Region, and assessed 34 GCMs based on three criteria: mean seasonal cycle, temporal trends, and spatial correlation. Li et al. [32] used the technique for order preference by similarity to ideal solution model to assess 31 GCMs in mainland Southeast Asia. Moreover, multimodel ensembles of GCMs have been proven to perform better than individual GCMs, especially after rejecting the most poorly performing models based on the assessment of all GCMs, and were shown to provide more reliable information on climate change [33,34]. Thus, an accurate assessment of GCMs with appropriate methods is necessary.

Because of the coarse resolution of the GCMs, the outputs provide inadequate climate change information at the regional or local scale [35,36]. Downscaling methods, including dynamic downscaling and statistical downscaling, have been used to bridge the gap between the coarse resolution of GCMs and the need for climate information at a local scale [37,38]. Statistical downscaling has advantages in its convenient implementation and lower computational requirements without considering the effects of boundary conditions [39]. The Delta method, which is one of the statistical downscaling methods, has been widely applied in future climate projections [40,41,42].

In view of the complexity of the East Asian monsoon system and the complex influence of topography, the uncertainty about future climate change is amplified in TP [43,44]. In addition, the limited number of observation stations and data availability aggravates the uncertainty in precipitation projections for TP. Previous studies assessed the simulation capabilities of the CMIP5 GCMs using several evaluation indicators for TP [2,8,22,45]. However, for TP, only a few studies have adopted a multiple-criteria evaluation method to assess the CMIP5 GCMs comprehensively, and simulated future precipitation in combination with downscaling methods. Therefore, in this study, an improved rank score method is applied for assessing the performance of the CMIP5 GCMs in simulating precipitation over TP. The Delta method is then used to project the future precipitation change during two periods under Representative Concentration Pathway 4.5 (RCP4.5) and Representative Concentration Pathway 8.5 (RCP8.5) scenarios for TP based on a multimodel ensemble, which selects the best performing GCMs. The improved rank score and Delta downscaling method can be used to evaluate the GCMs and simulate future climate change. This study is expected to provide valuable information on climate change, and for water resources and agricultural management throughout TP.

2. Materials and Methods

2.1. Study Area

TP is located in East Asia. The study area in this work is limited to 26–40° N and 74–106° E (Figure 1). TP features a complex topography, snow mountains, glaciers, permafrost, and lakes. TP is called the roof of the world because of its high elevation, especially in the western TP. The complex terrain and the presence of the Asian monsoon, which exerts an important influence on the regional climate, are the major factors controlling the amount and patterns of precipitation [46].

2.2. Data

2.2.1. GCM Data

The assessment and projection presented in this study are based on the 33 GCMs of the CMIP5 (

Table 1. Description of the CMIP5 climate models used in this study.

ID	Model Name	Modeling Centre (or Group)	Nation	Resolution (Lon × Lat)
1	ACCESS1.0	Commonwealth Scientific and Industrial Research Organization/Bureau of Meteorology	Australia	1.88° × 1.25°
2	ACCESS1.3	Commonwealth Scientific and Industrial Research Organization/Bureau of Meteorology	Australia	1.88° × 1.25°
3	BCC-CSM1.1	Beijing Climate Center, China Meteorological Administration	China	2.81° × 2.79°
4	BNU-ESM	Beijing Normal University	China	2.81° × 2.79°
5	CanESM2	Canadian Centre for Climate Modeling and Analysis	Canada	2.81° × 2.79°
6	CCSM4	National Center for Atmospheric Research	USA	1.25° × 0.94°
7	CESM1(CAM5)	National Center for Atmospheric Research	USA	1.25° × 0.94°
8	CESM1(WACCM)	National Center for Atmospheric Research	USA	2.50° × 1.88°
9	CMCC-CMS	Centro Euro-Mediterraneo sui Cambiamenti Climatici	Italy	1.88° × 1.88°
10	CNRM-CM5	Centre National de Recherches Météorologiques Centre Européen de Recherche et Formation Avancée en Calcul Scientifique	France	1.41° × 1.40°
11	CSIRO-Mk3.6.0	Commonwealth Scientific and Industrial Research Organization/Queensland Climate Change Centre of Excellence	Australia	1.88° × 1.88°
12	EC-EARTH	EC-EARTH consortium published at Irish Centre for High-End Computing	Netherlands /Ireland	1.13° × 1.13°
13	FGOALS-g2	Institute of Atmospheric Physics, Chinese Academy of Sciences	China	2.81° × 2.81°
14	FIO-ESM	The First Institute of Oceanography, SOA	China	2.80° × 2.80°
15	GFDL-CM3	NOAA Geophysical Fluid Dynamics Laboratory	USA	2.50° × 2.00°
16	GFDL-ESM2G	NOAA Geophysical Fluid Dynamics Laboratory	USA	2.00° × 2.02°
17	GFDL-ESM2M	NOAA Geophysical Fluid Dynamics Laboratory	USA	2.50◦ × 2.02°
18	GISS-E2-H	NASA/GISS (Goddard Institute for Space Studies)	USA	2.50° × 2.00°
19	GISS-E2-R	NASA/GISS (Goddard Institute for Space Studies)	USA	2.50° × 2.00°
20	HadGEM2-AO	National Institute of Meteorological Research, Korea Meteorological Administration	Korea	1.88° × 1.25°
21	HadGEM2-CC	Met Office Hadley Center	UK	1.88° × 1.25°
22	HadGEM2-ES	Met Office Hadley Center	UK	1.88° × 1.25°
23	INMCM4.0	Russian Academy of Sciences, Institute for Numerical Mathematics	Russia	2.00° × 1.50°
24	IPSL-CM5A-LR	Institute Pierre-Simon Laplace	France	3.75° × 1.89°
25	IPSL-CM5A-MR	Institute Pierre-Simon Laplace	France	2.50° × 1.27°
26	IPSL-CM5B-LR	Institute Pierre-Simon Laplace	France	3.75° × 1.89°
27	MIROC5	Atmosphere and Ocean Research Institute (The University of Tokyo), National Institute for Environmental Studies, and Japan Agency for Marine-Earth Science and Technology	Japan	1.41° × 1.40°
28	MIROC-ESM	The same as 27	Japan	2.81° × 2.79°
29	MIROC-ESM-CHEM	The same as 27	Japan	2.81° × 2.79°
30	MPI-ESM-LR	Max Planck Institute for Meteorology	Germany	1.88° × 1.87°
31	MPI-ESM-MR	Max Planck Institute for Meteorology	Germany	1.88° × 1.87°
32	MRI-CGCM3	Meteorological Research Institute	Japan	1.13° × 1.12°
33	NorESM1-M	Bjerknes Centre for Climate Research, Norwegian Climate Center	Norway	2.50° × 1.89°

), including historical monthly precipitation data from 1961 to 2005 and future outputs data from 2006 to 2100 under the RCP4.5 and RCP8.5 scenarios. Table 1 provides summary information on the GCMs, whereas more details are available on the CMIP5 website. In view of the different spatial resolutions of the GCMs, precipitation data of the GCMs were all resampled to a 2° × 2° grid using a bilinear interpolation approach [29].

2.2.2. Observed Data

The observed data from 1961 to 2005 used in this study were obtained from the China Meteorological Data Sharing Service System. The dataset was interpolated into a high spatial resolution grid of 0.5° × 0.5° on the basis of records from Chinese meteorological stations, accounting for the elevation effect using the Thin Plate Spline interpolation method [47], and verified through data quality control. To facilitate assessment and comparisons, the observed data were also resampled to a 2° × 2° grid to be consistent with the GCMs data. TP contains 89 of such grid cells (Figure 1), all of which were utilized in the comparisons.

2.3. Methods

2.3.1. Assessment of CMIP5 GCMs

Considering that a single assessment criterion can lead to a biased result [30,48], multiple criteria were adopted for the assessment, including the mean value (M), standard deviation (SD), temporal correlation coefficient (r_t), spatial correlation coefficient (r_s), Mann–Kendall test statistics (Z and Slope), and probability density function (PDF) statistics (Brier score (BS) and Significance score (Sscore)). Then, accounting for all criteria, we used an improved rank score method to comprehensively assess the performance of GCMs in simulating precipitation for TP.

M and SD reflects the homogeneity of the GCMs simulations. The monthly and seasonal M and SD were calculated as follows.

$$M=\frac{1}{n}{\displaystyle {\sum }_{i=1}^{n}Pi}$$

(1)

$$SD=\sqrt{\frac{{\displaystyle {\sum }_{i=1}^n(Pi-\overline{P})}}{n-1}}$$

(2)

where P_i represents the monthly or seasonal precipitation for the GCMs and the observed value at time i, $\overline{P}$ represents the mean precipitation, and n represents the number of time series.

The correlation coefficient (r) was used to measure the correlation of temporal and spatial distributions between the GCMs and observations. Monthly and seasonal correlation coefficients were calculated in order to perform the time series correlation analysis. For the spatial correlation coefficient, the r was calculated based on the mean value of monthly and seasonal data in each grid (89 cells in total). The calculation is performed according to the following formula.

$$r\_t=\frac{{\displaystyle {\sum }_{i=1}^n\left(P{o}_i-P\overline{o}\right)\left(P{m}_i-P\overline{m}\right)}}{\sqrt{{\displaystyle {\sum }_{i=1}^n{{\displaystyle \left(P{o}_i-P\overline{o}\right)}}^2\times {\displaystyle {\sum }_{i=1}^n{{\displaystyle \left(P{m}_i-P\overline{m}\right)}}^2}}}}$$

(3)

where Po_i and Pm_i represent the precipitation of observation and GCMs at time i, respectively; $P\overline{o}$ and $P\overline{m}$ represent the corresponding mean precipitation of observation and GCMs, respectively; and n represents the length of the time series.

$$r\_s=\frac{{\displaystyle {\sum }_{g=1}^n\left(P{o}_g-P\overline{o}\right)\left(P{m}_g-P\overline{m}\right)}}{\sqrt{{\displaystyle {\sum }_{g=1}^n{{\displaystyle \left(P{o}_g-P\overline{o}\right)}}^2\times {\displaystyle {\sum }_{g=1}^n{{\displaystyle \left(P{m}_g-P\overline{m}\right)}}^2}}}}$$

(4)

where Po_g and Pm_g represent the precipitation of observation and GCMs at cell g, respectively; $P\overline{o}$ and $P\overline{m}$ represent the corresponding mean precipitation of observation and GCMs, respectively; and n represents the number of cells in the grid.

The long-term trend of precipitation was identified using the Z statistic and the Slope of the Mann–Kendall test based on annual time series, so that the ability of the various GCMs in representing the actual precipitation trend could be compared. The Z and Slope statistics are defined as follows [49,50,51].

$$\mathrm{Z}=\{\begin{array}{cc}\frac{S-1}{\sqrt{\mathrm{var}\left(S\right)}},& S>0\\ 0,& S=0\\ \frac{S-1}{\sqrt{\mathrm{var}\left(S\right)}},& S>0\end{array}$$

(5)

$$S={\displaystyle {\sum }_{i=1}^{n-1}{\displaystyle {\sum }_{k=i+1}^n\mathrm{sgn}\left(x_k-x_i\right)}}$$

(6)

$$\mathrm{sgn}\left(\theta \right)=\{\begin{array}{cc}1,\hfill & \hfill \theta >0\\ 0,\hfill & \hfill \theta =0\\ \hfill -1,& \hfill \theta >0\end{array}$$

(7)

$$\mathrm{var}\left[S\right]=\left[n\left(n-1\right)\left(2n+5\right)-{\displaystyle {\sum }_{t}t\left(t-1\right)\left(2t+5\right)}\right]/18$$

(8)

where x_k and x_i represent sequential precipitation values, t represents the extent of any given time, and n is the length of the time series.

$$Slope=\mathrm{median}\left(\frac{x_i-x_j}{i-j}\right)$$

(9)

where 1 < j < i < n and the slope is considered as the median of the entire dataset.

The BS and Sscore of PDF statistics were used to evaluate the effect of the GCMs simulations on the probability density distribution of monthly precipitation [52,53].

$$BS=\frac{{\displaystyle {\sum }_{i=1}^n{\left(P{m}_i-P{o}_i\right)}^2}}{n}$$

(10)

$$Sscore={\displaystyle {\sum }_{i=1}^n\mathrm{Minimum}\left(P{m}_i,P{o}_i\right)}$$

(11)

where Pm_i and Po_i are the probabilities related to GCMs and observed precipitation, respectively, at the i-th bin, and n is the number of bins. BS is the mean square error of the measurement in probability prediction. The Sscore describes the degree of overlap between the observed and simulated probability distributions.

The improved rank score method distinguishes relative error indices from the nonrelative error indices, which could provide consistent results in multiple criteria assessment [29,30]. In this study, M, SD, Z, Slope, and BS are relative error indices, while r_t, r_s, and Sscore are nonrelative error indices. The GCM rank score of each criterion can be calculated as follows.

$$RSe=\{\begin{array}{c}1-\frac{Te-T\min }{T\max -T\min },T\mathrm{represents}\mathrm{the}\mathrm{relative}\mathrm{error}\mathrm{index}\\ \frac{Te-T\min }{T\max -T\min },T\mathrm{represents}\mathrm{the}\mathrm{non}-\mathrm{relative}\mathrm{error}\mathrm{index}\end{array}$$

(12)

For the relative error indices of M, SD, Z, and Slope, T_e represents the absolute error of the statistics between the GCM and observation, and a smaller T_e indicates a better performance of the GCM. For the relative error indices of BS, T_e represents the absolute value of the statistics for a GCM, and a smaller T_e indicates a better performance of the GCM. On the other hand, for the nonrelative error indices of r_t, r_s, and Sscore, T_e represents the absolute value of the statistics, and a larger T_e indicates a better performance of the GCM. T_max and T_min are the corresponding maximum and minimum values of all GCMs, respectively.

For M, SD, r_t, and r_s, the RS is calculated on both monthly and seasonal data. The weights for each season are set to 0.25, whereas the weight assigned to the whole year is 1. For Z, Slope, BS and Sscore, the overall GCM rank score of each criterion is RS_e. For M, SD, r_t, and r_s, the overall GCM rank score of each criterion can be calculated as follows.

$$R{S}_{eo}=\frac{R{S}_{spr}\times 0.25+R{S}_{sum}\times 0.25+R{S}_{aut}\times 0.25+R{S}_{win}\times 0.25+R{S}_a\times 1}{2}$$

(13)

where RS_spr represents the rank score in spring, RS_sum represents the rank score in summer, RS_aut represents the rank score in autumn, RS_win represents the rank score in winter, and RS_a represents the rank score based on the data of the whole year.

According to the calculated RS values for each criterion, the overall RS value of all criteria for precipitation can be obtained using the following formula.

$$R{S}_p={\displaystyle {\sum }_{e=1}^n\frac{R{S}_p\times W_e}{W_s}}$$

(14)

where RS_e represents the rank score of the e-th criterion, W_e represents the corresponding weight, W_s represents the overall weight, and n represents the number of criteria (eight in this study). For the two criteria describing the same characteristic, the weights are set to 0.5, while the weights of the single criterion describing characteristics are set to 1. Therefore, the weights of M, SD, Z, Slope, BS, and Sscore are set to 0.5, and the weights of the temporal and spatial r are set to 1.

2.3.2. Projection of Future Precipitation Change

According to the assessment of the performance of the GCMs, the multimodel ensemble (MME), based on the arithmetic mean method, is utilized to forecast future precipitation under RCP4.5 and RCP8.5 scenarios. In addition, the sub-multimodel ensemble, which selects the GCMs that perform better in the study region, is also used in this study. The formula is as follows.

$$MME=\frac{1}{n}{\displaystyle {\sum }_{k=1}^n{P}_k}$$

(15)

where P_k is the future precipitation outputs of the GCMs, and n represents the number of selected GCMs.

The Delta method is then applied to project future precipitation changes based on the historical MME data from 1961 to 2005 (45 years) and future outputs with the same duration: 2006–2050 and 2051–2095. The changes can be calculated as follows.

$$P_c=\frac{P_f}{P_h}$$

(16)

where P_f and P_h are the mean precipitation of the future and historical MME outputs.

3. Results

3.1. Annual Cycle of Precipitation

The mean monthly precipitation across an annual cycle (observed data and GCMs outputs) from 1961–2005 is shown in Figure 2. It is apparent that most GCMs can reasonably simulate the annual cycle of precipitation, but they generally overestimate the amount of precipitation over TP. All GCMs exhibit the same single-peak precipitation pattern as in the observations, which show that precipitation is concentrated in June–July–August–September (JJAS). For TP, the mean monthly precipitation during 1961 to 2005 over the TP is 32.1 mm (ranging from 2.9 mm to 87.6 mm), while that of the GCMs varies from 10.5 mm to 212.3 mm, with an average of 80.3 mm. The GCMs overestimate the precipitation, on average, by 48.2 mm, especially in spring (March–April–May (MAM)) and summer (June–July–August (JJA)), for which the average mean monthly errors are 57.1 mm and 66.0 mm, respectively, whereas the errors are 45.4 mm and 24.2 mm in autumn (September–October–November (SON)) and winter (December–January–February (DJF)), respectively.

3.2. Comparison of GCMs Based on Statistical Criteria

Table 2. Statistics of the assessment criteria for the observations and the GCMs over TP. The symbols “▲”, “△”, “▼”, and “▽” represent a significant increasing trend, an increasing trend, a significant decreasing trend, and a decreasing trend, respectively, where “significant” means p-value<0.05.

Model	Mean (mm)	SD (mm)	r_t	r_s	Mann–Kendall			PDF
					Z		Slope (mm/y)	Sscore	BS
Observations	32.1	30.3			1.81	▲	0.41
Access1.0	78.1	52.3	0.94	0.72	−1.24	▽	−0.42	0.59	0.044
Access1.3	91.8	55.6	0.92	0.76	0.32	△	0.10	0.54	0.048
BCC-CSM1.1	89.9	35.9	0.95	0.44	1.52	△	0.91	0.50	0.050
BNU-ESM	117.8	45.1	0.99	0.38	3.71	▲	2.84	0.49	0.050
CanESM2	59.8	44.8	0.97	0.87	3.51	▲	1.71	0.63	0.041
CCSM4	84.5	55.9	0.95	0.68	0.44	△	0.16	0.59	0.045
CESM1(CAM5)	92.9	64.0	0.95	0.69	1.13	△	0.53	0.63	0.042
CESM1(WACCM)	107.6	64.1	0.90	0.83	2.22	▲	1.32	0.57	0.045
CMCC-CMS	79.6	46.1	0.97	0.75	0.64	△	0.43	0.55	0.048
CNRM-CM5	68.0	43.6	0.99	0.78	3.24	▲	1.52	0.61	0.041
CSIRO-Mk3.6.0	60.6	43.5	0.97	0.86	0.95	△	0.51	0.63	0.040
EC-EARTH	56.0	40.3	0.97	0.80	1.07	△	0.48	0.63	0.040
FGOALS-g2	76.3	38.5	0.94	0.60	1.83	▲	0.92	0.53	0.048
FIO-ESM	111.5	37.5	0.95	0.75	2.67	▲	1.73	0.48	0.051
GFDL-CM3	86.5	50.5	0.97	0.88	−2.49	▼	−1.08	0.56	0.048
GFDL-ESM2G	76.2	46.1	0.98	0.85	2.04	▲	0.99	0.56	0.046
GFDL-ESM2M	78.5	46.9	0.96	0.86	1.32	△	0.66	0.56	0.046
GISS-E2-H	75.0	36.0	0.95	0.54	−2.63	▼	−1.23	0.52	0.048
GISS-E2-R	72.0	31.1	0.95	0.51	−2.30	▼	−0.93	0.51	0.049
HadGEM2-AO	78.2	56.9	0.95	0.73	0.89	△	0.26	0.63	0.039
HadGEM2-CC	81.8	57.5	0.95	0.65	0.44	△	0.14	0.62	0.040
HadGEM2-ES	80.1	55.9	0.95	0.65	−0.79	▽	−0.27	0.62	0.041
INMCM4.0	79.9	44.5	0.96	0.88	1.99	▲	0.78	0.54	0.047
IPSL-CM5A-LR	56.0	25.2	0.90	0.78	1.03	△	0.42	0.52	0.048
IPSL-CM5A-MR	59.4	33.2	0.97	0.70	1.38	△	0.47	0.54	0.048
IPSL-CM5B-LR	63.5	25.6	0.90	0.78	0.28	△	0.15	0.53	0.048
MIROC5	78.3	52.5	0.96	0.82	−0.17	▽	−0.07	0.59	0.044
MIROC-ESM	92.3	54.8	0.96	0.39	2.26	▲	1.16	0.56	0.045
MIROC-ESM-CHEM	91.9	54.8	0.96	0.44	2.16	▲	1.28	0.57	0.045
MPI-ESM-LR	84.8	57.1	0.98	0.76	0.00	−	0.01	0.58	0.044
MPI-ESM-MR	84.8	56.7	0.98	0.75	−0.95	▽	−0.36	0.59	0.044
MRI-CGCM3	56.2	41.2	0.98	0.76	0.34	△	0.27	0.62	0.040
NorESM1-M	99.3	66.7	0.98	0.81	0.85	△	0.47	0.57	0.046

presents the statistics, for the observations and the GCMs, of the eight assessment criteria: M, SD, r_t, r_s, Z, Slope, Sscore, and BS. For the mean monthly precipitation, a smaller absolute error indicates a better performance of the GCMs. The EC-EARTH, IPSL-CM5A-LR, and MRI-CGCM3 demonstrate a better ability in simulating the mean precipitation for TP, and exhibit relatively small absolute errors of 24.0 mm, 24.0 mm, and 24.1 mm, respectively, while the CESM1(WACCM), FIO-ESM, and BNU-ESM perform poorly.

The SD of the observations is 30.3 mm, while that of the GCMs ranges from 25.2 mm to 66.7 mm, with a mean value of 47.3 mm. GISS-E2-R, IPSL-CM5A-MR, and IPSL-CM5B-LR show good performance in simulating the SD, whereas CESM1(CAM5), CESM1(WACCM), and NorESM1-M do not simulate SD well.

The value of r_t is above 0.90 for all GCMs, with a maximum of 0.99, which indicates that the GCMs provide superior simulations of the characteristics of the time series of annual precipitation; among them, the CNRM-CM5 and BNU-ESM models perform best. On the contrary, r_s performs inconsistently across all GCMs, as it varies from 0.38 to 0.88. The spatial correlation coefficient of 10 GCMs (approximately 30.3% of the total number of GCMs) is higher than 0.8, indicating that these 10 GCMs (e.g., the GFDL-CM3, INMCM4.0, and CanESM2 models) provide a better description of the characteristics of spatial distribution than the remaining GCMs. Figure 3 presents the correlation coefficient of the GCMs in each season. Generally, the GCMs can effectively simulate the time characteristics in spring and autumn with a mean r_t value of 0.99 and 0.99, while the r_t values in summer and winter are 0.80 and 0.80, respectively. The GCMs do not perform as well in reproducing the spatial distribution as they do in reproducing the temporal distribution, especially in winter.

The annual observed precipitation shows a significant (p < 0.05) increasing trend (0.41 mm/year) from 1961 to 2005 in TP. In total, 25 GCMs exhibit an increasing trend, which is significant in 10 of them. This indicates that most GCMs could capture the same trend as that in the observations. The FGOALS-g2, INMCM4.0, and GFDL-ESM2G exhibit Z values closest to the observations. The slope values in IPSL-CM5A-LR, CMCC-CMS, and IPSL-CM5A-MR are those that agree the most with the observations. The GISS-E2-H, GFDL-CM3, GISS-E2-R, and BNU-ESM perform relatively poor in reproducing the annual precipitation variation trend.

The Sscore of the GCMs ranges from 0.48 to 0.63, with a mean value of 0.57. This indicates, in general, a relatively limited performance, in spite of the Sscore of nine GCMs being higher than 0.6, which means a better representation of the probability density distribution compared with the observations. The BS of the GCMs varies from 0.039 to 0.051, with a mean value of 0.045. In general, a larger Sscore usually implies a smaller BS. The CSIRO-Mk3.6.0, EC-EARTH, and HadGEM2-AO show a good ability in simulating the characteristics of the probability density distribution, while the FIO-ESM, BNU-ESM, and BCC-CSM1.1 are not able to simulate them well.

3.3. Comprehensive Assessment by Rank Score

Figure 4 illustrates the rank score of the eight chosen criteria and the overall results. Different criteria produce different rank scores for the same GCM. For instance, EC-EARTH scores relatively high in the eight criteria and in the overall result, while GISS-E2-R has higher scores for M, SD, and r_t, but lower ones for r_s, Z, Sscore, and BS. Similarly, CESM1(CAM5) shows a superior ability in simulating the annual variation trend and probability density distribution, whereas it is not able to capture the characteristics of the M and SD well, but exhibits an average ability in describing the temporal and spatial distribution characteristics. This demonstrates that the ability of individual GCMs in simulating certain aspects is insufficient, thus a comprehensive assessment using multiple criteria is of great significance.

In the overall assessments (Figure 4), the rank score ranges from 0.38 to 0.89, and 22 GCMs (approximately 66.7%) score above 0.6. The five best GCMs, which exhibit better performance in terms of precipitation simulation over TP are CSIRO-Mk3.6.0 (0.888), EC-EARTH (0.886), MRI-CGCM3 (0.849), CNRM-CM5 (0.788), and CanESM2 (0.786).

3.4. Sensitivity Analysis of GCMs Scores

The sensitivity of the rank scores of the GCMs was investigated by comparing the overall score with the results after removing each statistic individually. The results (Figure 5) indicate that the rank score is insensitive to the addition or removal of a single criterion. This suggests that the improved rank score method with multiple criteria can provide a robust assessment of GCMs performance. In addition, CSIRO-Mk3.6.0, EC-EARTH, MRI-CGCM3, CNRM-CM5, and CanESM2 always provided better performance, which was unaffected by the addition or removal of a specific criterion.

3.5. Future Precipitation Projection

The multiple ensemble method is deemed capable to mitigate the uncertainty of GCMs in regional climate projection [54,55,56].

Table 3. Statistics relative to the assessment criteria of a multiple model ensemble for TP.

Model	Mean (mm)	SD (mm)	r_t	r_s	Mann–Kendall			PDF
					Z		Slope (mm/y)	Sscore	BS
observation	32.1	30.3			1.81	▲	0.41
MME	80.3	45.2	0.97	0.86	3.73	▲	0.70	0.43	0.050
SMME	65.0	44.1	0.98	0.87	4.22	▲	0.46	0.51	0.048

shows the statistics relative to the assessment criteria of MME and SMME. The GCMs that ranked within the top 20% (the SMME in this study), including CSIRO-Mk3.6.0, EC-EARTH, MRI-CGCM3, CNRM-CM5, CanESM2, HadGEM2-AO, and GFDL-ESM2G, were selected for future precipitation projection for TP. Undoubtedly, the MME and SMME can both capture the precipitation trend over an annual cycle (Figure 2), whereas the SMME performs better in simulating the precipitation (Table 3). The future precipitation change was assessed under RCP4.5 and RCP8.5 scenarios using the Delta method in the near-term (2006–2050) and far future (2051–2095) periods.

Table 4. Changes in precipitation for TP based on submultiple model ensemble (SMME) under RCP4.5 and RCP8.5 scenarios relative to the period of 1961 to 2005.

Period	RCP4.5					RCP8.5
	Annual	MAM	JJA	SON	DJF	Annual	MAM	JJA	SON	DJF
2006–2050	106.4%	109.3%	105.6%	105.8%	104.8%	105.8%	109.2%	105.3%	104.5%	102.2%
2051–2095	112.3%	117.3%	110.3%	111.1%	113.9%	116.7%	125.0%	114.3%	113.4%	116.9%

shows the change of future precipitation during 2006 to 2050 and 2051 to 2095 compared with the period 1961–2005 under both RCP4.5 and RCP8.5 scenarios. The magnitude of annual precipitation increase in the far future is much larger than in the near-term. This is particularly true for the period of 2051 to 2095 under the RCP8.5 scenario, which exhibits a value of 116.7%, representing a 16.7% increase of precipitation compared to that in the period 1961–2005. The future annual precipitation is projected to increase by ~6% in the near-term relative to 1961–2005, while increases of 12.3% and 16.7% in the far future under RCP4.5 and RCP8.5 scenarios, respectively, are expected. Moreover, a slightly larger increase in precipitation is expected to occur in spring (MAM) with respect to the other seasons. Figure 6 shows the evident increasing trend of annual precipitation, and highlights that the increasing trend under the RCP8.5 scenario is the most apparent. The difference between the two RCP scenarios is small in the near-term, but becomes more substantial with time, indicating that the projected change in precipitation is more sensitive to the emission scenarios in the far future than in the near-term (Figure 6).

Figure 7 presents the spatial distribution of mean annual future precipitation changes during 2006–2050 and 2051–2095 relative to the period 1961–2005 under RCP4.5 and RCP8.5 scenarios for TP. Similar spatial distributions can be seen in the near-term (2006–2050) and far future (2051–2095) under the two scenarios. This suggests that the most predominant increase in precipitation is expected to occur in the northern part of the study area, whereas the expected change near the eastern edge is not so drastic. Additionally, most of the grid cells show an increasing trend from the near-term to the far future, which is larger in magnitude under the RCP8.5 scenario than under the RCP4.5 scenario.

4. Discussion

In this paper, it has been shown that most GCMs can reproduce the seasonal pattern of precipitation for TP reasonably. However, all GCMs overestimate the amount of precipitation compared with the observations. Similar results have also been shown by Xu et al. and Su et al. [22,57], notwithstanding the different approaches followed by the authors for assessing the performance of the CMIP5 GCMs. In addition, the wet bias was also identified in the CMIP3 GCMs, suggesting that the CMIP5 has unsolved limitations deriving from the CMIP3 [2,17]. Moreover, these limitations appear to be more obvious in the steep mountain regions [58]. In various studies, a correction of the observed values for orographic effects was attempted, because the interpolation of station data (point data) of precipitation to assign values to the grid cells is considered to be biased by the complex topography [59,60]. Nonetheless, most GCMs still overestimate precipitation [22]. In further studies, an algorithm to modify GCMs simulations of precipitation over steep mountains was also explored [61]. Additionally, the horizontal resolution of the models is an important factor for simulated precipitation, especially for the orographic precipitation. In this study, we have found that the GCMs with higher horizontal resolution seem to perform better than the model with lower resolution. In fact, many factors control the precipitation in mountain regions. Therefore, it is always essential to better understand physical parameterization schemes and dynamical process for improving the models in simulating precipitation.

We assessed and ranked the performance of CMIP5 GCMs in simulating precipitation using the improved score-based method. Similar results can also be found in studies of Su et al. [22], which suggest that CSIRO-Mk3.6.0, MRI-CGCM3, and CanESM2 have a better performance compared to the other GCMs. However, the fact that these three models perform relatively better for TP does not means that they will perform as well in other regions. Previous studies also showed that the performance of GCMs depends on the region they are applied to [29,32]. Therefore, the performance of GCMs should be assessed at the regional scale to obtain more reliable information.

Many studies have indicated that GCMs generally do not perform as well in simulating precipitation as they do in simulating temperature at a regional scale, especially for TP [2,22,30,32,62]. Extracting the precipitation outputs of the GCMs directly may not provide an accuracy which is sufficient for actual applications. Multiple model ensemble and downscaling methods could mitigate the uncertainty of GCMs for regional climate projection. Different MME methods have been used in GCMs studies, mainly including the arithmetic mean method, the Bayesian model averaging, and the reliability ensemble averaging, all of which indicate that the MME performs better than any individual GCM in simulating climate factors [33,63]. In this study, we adopted the arithmetic mean method to obtain an MME from the GCMs that exhibited the best performance. Additionally, we chose the Delta method for projecting future precipitation change. For reducing the uncertainty on future precipitation projection systematically, multiple methods should be considered and compared in further studies.

5. Conclusions

This study focused on assessing the performance of 33 CMIP5 GCMs in simulating precipitation and projecting future precipitation changes for TP. The performance was evaluated using an improved score-based method, which adopted multiple criteria to achieve a comprehensive assessment. The future precipitation change was obtained using the Delta method and the submultiple model ensemble in the near-term (2006–2050) and in the far future (2051–2095) under RCP4.5 and RCP8.5 scenarios. The main conclusions are summarized as follows.

(1) Most GCMs can reasonably simulate the annual cycle of precipitation. They successfully reproduce a single-peak pattern of precipitation, in which most of the precipitation is concentrated in JJAS. However, all the GCMs overestimate the monthly precipitation for TP, on average by 48.2 mm, especially in spring and summer.

(2) The GCMs provide superior simulations of the temporal pattern of precipitation, with a correlation coefficient above 0.9 for the annual precipitation, while do not perform so well in reproducing the spatial distribution characteristics. Most GCMs are able to capture the same annual variation trend as in the observation, but have relatively limited performance in simulating the probability density distribution of precipitation.

(3) Different assessment criteria yield different rank scores for the same GCM, and, therefore, a comprehensive evaluation using multiple criteria is of great significance. The improved rank score method with multiple criteria achieved a robust assessment of GCMs performance.

(4) The results of the comprehensive assessment indicated that CSIRO-Mk3.6.0, EC-EARTH, MRI-CGCM3, CNRM-CM5, and CanESM2 have better performance in simulating precipitation for TP compared to the other GCMs that we have assessed.

(5) The future annual precipitation was projected to increase by about 6% in the near-term with respect to the period 1961 to 2005, while it increases by 12.3% and 16.7% in the far future under RCP4.5 and RCP8.5 scenarios, respectively. The future precipitation showed an increasing trend, which is much more evident under the RCP8.5 scenario than under the RCP4.5 scenario.

(6) Similar spatial distributions of precipitation changes can be seen in the near-term (2006–2050) and far future (2051–2095) under the two scenarios. These distributions suggest that precipitation might increase in the north of TP more than in other parts of TP, and that the precipitation change in the eastern margin of TP is not so predominant.

Assessing the performance of GCMs effectively and simulating future precipitation are important for understanding ongoing climate change and its impact on hydrology and water resources, agriculture, and ecology. The results of this study can provide valuable input for further studies focusing on TP. Moreover, in further studies, multiple methods for ensembling GCMs and for producing future precipitation projections should be considered and compared for mitigating the uncertainty of GCMs outputs at the regional scale.