Reliability of the ERA5 in Replicating Mean and Extreme Temperatures across Europe

Abstract

ERA5 is widely considered as a valid proxy of observation at region scales. Surface air temperature from the E-OBS database and 196 meteorological stations across Europe are being applied for evaluation of the fifth-generation ECMWF reanalysis ERA5 temperature data in the period of 1981–2010. In general, ERA5 captures the mean and extreme temperatures very well and ERA5 is reliable for climate investigation over Europe. High correlations ranging from 0.995 to 1.000 indicate that ERA5 could capture the annual cycle very well. However, the high mean biases and high Root Mean Square Error (RMSE) for some European sub-regions (e.g., the Alps, the Mediterranean) reveal that ERA5 underestimates temperatures. The biases can be mainly attributed to the altitude differences between ERA5 grid points and stations. Comparing ERA5 with the other two datasets, ERA5 temperature presents more extreme temperature and small outliers for regions southern of 40° latitude and less extreme temperatures in areas over the Black Sea. In Scandinavia, ERA5 temperatures are more frequently extreme than the observational ones.

1. Introduction

One of the main issues that are addressed in the largest percentage of climatological studies, and also one of the main challenges that climate scientists face to conduct their research is to have accurate and long-term datasets. Such data that lead to robust results and conclusions and can provide critical information on the monitoring of the trends of the climatic parameters, need to have a homogenous spatial resolution over each domain of interest, and should be long-term, and have no gaps. Moreover, accurate datasets are an essential aspect for the climate model evaluation, hydrological and agrometeorological model simulations, impact studies, and determination of adaptation and mitigation strategies [1,2]. Even though countries all over the world have a sufficient network of observational and/or gauge stations, in several cases, their datasets do not fulfill the aforementioned criteria.

Thus, in order to fill this “data gap”, acknowledging the need for coarse data networks, several high-resolution gridded datasets have been developed over the past few years, providing a more consistent climatic archive both from the temporal and spatial scale point of view [3]. The importance of the reanalysis datasets lies mainly in the fact that they are a contemporary and effective information source for numerous decadal time-scale climatological studies [4]. However, it should be highlighted that these reanalysis data are not used indiscriminately [5], since they may be affected by the potential inhomogeneities of the observational data, errors in satellite radiance, incomplete model physics, and insufficient representation of topography [6]. Their evaluation is fundamental to assess the potential uncertainty due to the interpolation (mainly dependent on the observation network density used) and to the data assimilation and forecast models [5,7]. Rapaić et al. [6] also mentioned that satellite radiance processing errors may also result in discontinuities and/or artificial trends in reanalysis data series underlining the importance of their evaluation, especially when they are used for long-term trend analysis [8]. It should also be taken into consideration that each dataset could have inconsistencies in its performance (in some regions it could provide very satisfactory results, and in others, its skill could be low); thus, no general and universal conclusion can be derived for the whole planet on their ability to capture the climatic characteristics of the global. Furthermore, the results of an evaluation strongly depend on the parameter that is being evaluated (e.g., temperature or precipitation), along with whether its mean or extreme values are tested. Reanalysis data are often characterized by systematic changes due to alterations in satellite/remote sensing data in terms of adjustment, sensitivity, and data network density [9]. Additionally, reanalysis data can lead to unreliable climate signals as the changes that may occur in the observational data interact with the assimilating model [10].

Recently, the European Center for Medium-Range Weather Forecasts (ECMWF) released their newest reanalysis database (ERA5 reanalysis) [11] to replace the ERA-Interim one. As the authors mention, the most important upgrade of ERA5 has been its finer spatial, as well as temporal resolution (31 km vs. 79 km and hourly vs. 3-hourly, respectively). Additionally, they highlight the fact that a much larger amount of data and a higher number of vertical levels were utilized. In comparison to its predecessor, ERA5 presents improvements both on the data assimilation system and the model physics. Nowadays, several studies have been published worldwide, aiming at the evaluation of this updated database. For example, there is increased accuracy in simulating the parameter of precipitation in different regions of the planet [2,12,13,14] by underlining the strengths and weaknesses of ERA5 over each domain of interest. Other studies focused on more specialized parameters, such as solar radiation, irradiance, land surface temperature, wind power production, sea surface temperature, thermal climate index, and others [15,16,17,18,19]. Nevertheless, regarding air temperature (T-2m), until now (to the authors’ knowledge), no studies have been conducted regarding the evaluation of this main climatic parameter over Europe. Thus, aiming to fill this literature gap and provide a useful “guide” to future researchers that may want to use ERA5 temperature data, we decided to conduct this evaluation.

Overall, our study’s main goal was to detect and examine the strengths and weaknesses of this updated reanalysis dataset (ERA5) all over the European domain. The comparisons were made not only by using previous gridded datasets, such as the European daily high-resolution gridded observational dataset (E-OBS), but also using data from 196 stations that cover the region of interest produced by the European Climate Assessment and Dataset (ECA&D). The evaluation focuses on the main climatic parameter, temperature, expanding the analysis both on its mean annual and seasonal values, but also on its extremes (95th and 5th percentile) for the assessment of the ERA5 performance and sensitivity on the temperature’s minimum and maximum values.

2. Materials and Methods

As mentioned in the previous paragraph, we utilized daily mean temperature data provided by three sources (ERA5, ECA&D, E-OBS). The evaluated dataset ERA5 is the most updated recent reanalysis data, produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) [20], with a high spatial resolution of 0.25°. The main advantage of the ERA5, in comparison with the previous versions of ECMWF reanalysis data, is that it takes into account the uncertainty of the initial data [21]. For the evaluation of the ERA5 data, daily mean temperature values derived from 196 European stations (Figure 1) produced by ECA&D are utilized [22]. The station selection was made according to their location (uniform coverage of the studied area) and data availability. Except for the station data, the E-OBS daily gridded dataset with a 0.25 × 0.25 spatial resolution [23] was used to evaluate ERA5. The E-OBS dataset derives from the European Climate Assessment and Dataset (ECA&D) after the application of the Kriging interpolation [23]. The research area is Europe, and all used datasets cover a 30-year time period starting from 1981 to 2010. Additionally, for a more thorough and detailed evaluation of the results, we divided the domain of study into eight (8) subregions (Figure 1) according to the common EURO-CORDEX analysis domain [24].

The evaluation of the ERA5 data was made both annually and seasonally. Apart from the mean daily temperature, an additional analysis was made for extreme maximum and minimum temperatures. The 95th and the 5th percentiles were selected as a threshold for extreme maximum and minimum temperature, respectively [25,26,27]. The analysis was performed in two parts. Firstly, for each station, the closest ERA5 grid point (over land) was selected. The ERA5 mean, extreme maximum (95th percentile), and minimum (5th percentile) temperatures were compared with the respective station data. Secondly, to compare ERA5 data with E-OBS data, a bilinear remapping of E-OBS data to the ERA5 data was performed. In both cases, the annual and seasonal differences between the ERA5 mean, extreme maximum and minimum temperatures, as well as the respective data from ECA&D stations and the E-OBS dataset were calculated, mapped, and compared.

3. Results

The comparison between ERA5 with the E-OBS data and daily observations of ECA&D stations is summarized in

Table 1. Correlation RMSE, mean bias, and Standard Deviation Bias (StDB) between the ERA5 dataset and E-OBS, and daily observation of ECA&D stations. Annual statistics are given for mean temperature for the eight European subregions.

ERA5–E-OBS
Subregion	Correlation	RMSE	Mean Bias	StDB
AL	0.997	0.66	−0.19	0.63
BI	0.995	0.52	0.23	0.47
EA	1.000	0.22	0.11	0.20
FR	0.998	0.38	−0.01	0.38
IP	0.998	0.37	0.04	0.37
MD	0.997	0.75	0.36	0.66
ME	0.999	0.37	0.02	0.37
SC	1.000	0.38	0.25	0.30
ERA5–ECA&D
Subregion	Correlation	RMSE	Mean Bias	StDB
AL	0.999	1.84	−1.80	0.40
BI	0.995	0.51	−0.06	0.51
EA	1.000	0.34	0.26	0.22
FR	0.998	0.61	−0.45	0.41
IP	0.999	1.10	−1.04	0.36
MD	0.998	1.39	−1.32	0.43
ME	0.998	0.40	−0.06	0.40
SC	1.000	0.37	−0.25	0.27

. ERA5 temperatures correlate with E-OBS, and daily observation temperatures exceed 0.995 at the annual base. The statistics for the comparison between ERA5 and E-OBS are much better than the one with ECA&D, small RMSE (<0.75 °C), small mean biases (<0.36 °C), and standard deviation biases (<0.66 °C). For the ECA&D database, RMSE values range between 0.34 and 1.84 and the mean biases are not higher than −2.0 °C. The mean standard deviation is close to 0.5 °C.

British Isles (BI) present the lowest correlation (0.995), while the relationship between the two databases for Scandinavia (SC) and Eastern Europe (EA) is very strong (cor = 1.000). ERA5 underestimates Annual mean Temperatures (AmT) compared to ECA&D data, but ERA5 slightly overestimates AmT related to E-OBS. Alps (AL), the Mediterranean region (MD), and the Iberian Peninsula (IP) show the highest RMSE values and biases. For example, for the Alps region, the RMSE value of ECA&D is three times higher compared to the E-OBS one, and the mean bias (ECA&D) is almost nine times greater than the corresponding E-OBS bias. Only for the EA region, the ERA5 temperatures are overestimated for both comparisons. The statistical differences in temperature between the models could be explained due to the different altitudes between the regions and the sparse observations per region. Regions with high altitudes and complex terrain present the highest biases. In order to identify which of these two characteristics have the greatest effect on the temperature differences, a correction of ERA5 temperature in terms of the average lapse rate was performed according to the following equation:

$$T=T_0+\gamma \Delta z,$$

(1)

where γ = −0.65 °C/100 m. For example, the elevation of ERA5 in the Alps region differs by about 200 m to 400 m higher or lower compared to E-OBS or the station’s one (Figure 2a,b). However, it appears to not have an important role in the temperature differences (Figure 2c,d). Additionally, in the northern coastal regions in Scandinavia and the Mediterranean coast, there are significant differences in altitudes. In this case, the correction of elevation reduces the temperature differences (Figure 2e,f), meaning that complex terrain accommodates the temperature biases of ERA5.

3.1. Spatial Resolution of Mean Temperature

ERA5 mean temperatures follow Europe’s climatology with the lowest temperatures appearing in northern regions and higher temperatures in the more southern ones (Figure 3a and Figure 4a,b). Additionally, low temperatures occur in mountainous areas due to high elevation. However, ERA5 data do not show uniform behavior throughout Europe. Additionally, the areas with low or negative temperatures present the highest standard deviation values (Figure 3b and Figure 4c,d). ERA5 data are very similar or even identical to E-OBS in many regions in Europe, but there are other regions where ERA5 datasets differ considerably (Figure 3c). Notably, for regions of latitude greater than 55° N, ERA5 data differ. Specifically, in the northern coastal zone of Scandinavia, the ERA5 is underestimated compared to E-OBS, while on the inner side of the Scandinavian peninsula there is a statistically significant temperature overestimation. In the southern part of Europe and the Mediterranean, it seems that the complex terrain is affecting data performance. Areas with high altitudes, such as the Alpine region and the Mediterranean mountain windward regions, show a statistically significant underestimation of temperatures. The Alps are characterized by the greatest underestimation. In contrast, the highest positive differences occur in the southeast of Europe, southern Turkey, and North Africa, areas where the initial station temperature data of the E-OBS dataset are quite sparse. The distribution of temperature differences for winter and summer gives the overall picture in the annual results (Figure 4e,f). The lowest temperatures of ERA5 in the coastal zone of Scandinavia are recorded in all seasons except summer, while the very high temperatures in the North African region and South-East Turkey occur in summer, and with less intensity in spring (not shown). Additionally, in all seasons, the western Iberian Peninsula, the Italian peninsula, and the south-western Balkan Peninsula show lower temperatures than the E-OBS.

At the same time, ERA5 data were also compared with station data from the ECA&D database (Figure 3d). The spatial representation of the differences between the two databases shows a similar pattern for central Europe with small non-statistical differences between ERA5 and station data. Accordingly, for Scandinavia, the Norwegian stations’ temperatures are underestimated, while daily temperatures in Sweden and Finland are overestimated. The pattern in southern Europe is different. For numerous stations, temperature is underestimated. Specifically, for about 40% of the stations, ERA5 show statistically significant lower temperatures than the observed ones, and only 20% showed statistically significant positive differences. It should be noted that for three stations in Romania and one station in the Czech Republic, ERA5 overestimates mean temperatures of more than 5 °C for all seasons (Figure 4g,h, spring and autumn not shown).

3.2. Spatial Resolution of Extreme Temperature

A corresponding analysis was also carried out for extreme temperatures. Figure 5 shows the results for the minimum extreme mean temperature defined as the 5% percentile of the mean temperature. It was found that the northern and southern regions of Europe present the greatest differences (Figure 5b). Scandinavia is once again significantly distinguished; the extreme low mean temperatures of Norway are underestimated, while throughout the rest of Scandinavia, there is a significant overestimation of extreme low mean temperatures by ERA5. A very significant underestimation of extreme low mean temperatures is also observed over the Alpine region. The windward slopes of the mountains in the Mediterranean peninsula, a great part of Turkey, and North Africa are characterized by a statistically significant underestimation of extreme low mean temperatures. This pattern is the same in all seasons except summer, where it seems that generally, in most of Europe, ERA5 overestimates extreme low mean temperatures (not shown). A significant variation occurs in the Alps, with the northern slopes showing lower extreme mean temperatures than the E-OBS and the southern slopes showing higher extreme low mean temperatures.

For extreme low mean temperatures, a comparison was also made with the data of the ECA&D stations (Figure 5c). The general pattern of differences in extreme low mean temperatures does not differ significantly from the above results, but the spatial and temporal peculiarities presented are important. In particular, on an annual basis, only 17 stations show a statistically significant positive difference in extreme low mean temperatures. In winter (not shown), positive statistically significant results are limited to eight (8) within the Scandinavian Peninsula, three (3) in Romania, two (2) in Italy, and two (2) in Great Britain. Statistically significant differences in extreme low mean temperatures occur mainly in Norway, and in countries around the Mediterranean Basin. In summer (not shown), most of the statistically significant differences were presented in the North Balkan and the Alps.

The equivalent pattern of extreme high mean temperatures (95% percentile) is different from that of low mean temperatures (Figure 6). ERA5 seems to overestimate the extreme high mean temperatures compared to E-OBS dataset, while the majority of the biases are non-statistically significant (Figure 6b). The highest differences are not in Europe but in Southern Turkey and North Africa. For winter, the general pattern changes by underestimating extreme high mean temperatures both in the northern Scandinavian region and in the Alpine region (not shown). This difference is not capable of changing the overall pattern of extreme high mean temperatures, as presented in the other seasons’ maps. Autumn and spring show the least statistically significant differences (not shown).

Comparing extreme high mean temperatures of ERA5 with the database of ECA&D stations, the pattern is a little different (Figure 6c). The southern European stations around the Mediterranean Basin continue to show statistically significant negative differences, meaning that the ERA5 model underestimates the extreme high mean temperatures in the Mediterranean and the Northern Balkans. The exception is two stations in Romania and one station on the Czech–German border showing statistically significant positive differences.

3.3. Statistical Analysis of Extreme Temperature

The previous analysis showed that the mean spatial ERA5 temperatures compared to the two databases (E-OBS and ECA&D) are quite similar. However, when the comparison is more thorough, discrepancies are revealed.

These differences are likely related to the different distributions that ERA5 temperatures seem to have. The density plots of the stations with the highest positive (Omu Peak station in Romania) and negative (Embrun station in France) difference, as well as a station with no difference are presented in Figure 7. The plots show that the mean temperature of the selected stations presents a bimodal distribution, meaning that there is a large group of days characterized by positive temperatures and a group of days characterized with negative temperatures. In the case of the Romanian station, ERA5 data show almost the same frequency of positive values but with higher temperatures, and the peak of positive values moves from about 8 °C to 18 °C (Figure 7a). In contrast, the frequency of negative temperatures decreases, and the peak of negative temperatures move from −8 °C to −3 °C (Figure 7c). For negative temperatures, there is a move towards negative values, with a lower frequency and lower temperatures. However, there is an absolute match between the station data and ERA5, as shown in Figure 7b.

For a better description of the temperatures’ distribution, the statistical measures of skewness and kurtosis are used. Distributions with positive kurtosis excess exhibit tail data exceeding the tails of the normal distribution, while with negative kurtosis excess, the tail data are less extreme than the normal distribution tails. Skewness measures extreme values in one versus the other tail. Negative skew refers to a longer or fatter tail on the left side of the distribution, while positive skew refers to a longer or fatter tail on the right.

Figure 8 presents the kurtosis of the three datasets. Central and northern Europe, except for Scandinavia, appear to have negative kurtosis excess values, meaning that the extreme temperatures will be less extreme than usual. In southern European regions, the kurtosis has high negative values, implying that the regions will experience occasional extreme temperatures. Scandinavia is the only region presenting positive kurtosis with large temperature outliers. The higher the negative kurtosis, the flatter the tails. Comparing ERA5 with the other two datasets, regions south of 40° latitude present higher negative kurtosis values, meaning that ERA5 temperature presents more extreme values and small outliers. On the other hand, regions above the Black Sea present lower negative values, implying less extreme temperature. The most interesting aspect of the kurtosis results is in Scandinavia. ERA5 fails to represent the positive kurtosis values. ERA5 temperatures are more often extreme than normal, while the large temperature outliers are not detected.

The results of the skewness can be compared in Figure 9. A large part of Europe has distributions with almost zero skew (−0.2 to 0.2). The northern European regions have negative skewness that refers to a longer or fatter tail in low temperatures. However, the southern regions show a positive skew, meaning that their distributions have a longer or fatter tail in high temperatures. It should be pointed out that there are some differences between the ERA5 temperature and the two datasets. Scandinavian stations, for which ERA5 data show even lower negative skewness values, give lower extreme negative temperatures. For stations in Europe that change the signal of skewness, that is, from positive to negative, the pattern is more complex. According to ERA5 temperature, some stations show more low temperatures (e.g., part of Great Britain). However, there are stations, for example, Rijeka of Croatia (45.33 N–14.45 E), where there is a completely different distribution with more high temperatures than the observed data, but also more extreme low temperatures.

4. Discussion

From our analysis on the simulation of daily air temperature (T-2m) over Europe of the latest version of ERA datasets (ERA5), it is obvious that, in general, their performance is quite satisfactory, and they have been substantially improved in comparison to their previous versions. Analogous results were found in the recent study of Bandhauer et al. [28], in which daily precipitation was evaluated over several parts of the European region. The authors mention, as their concluding remark, that even though ERA5 data are undoubtfully useful and can capture the mesoscale precipitation patterns over Northern and Central Europe, they present some limitations and weaknesses that should be taken under serious consideration. Even though the analysis of Mahto and Mishra [29] took place in a very different region (India) than in our case, especially regarding its climatic characteristics (as well as topography, geographical positions but also data availability), an interesting result was found for maximum and minimum temperatures. It is mentioned that during the monsoon season, ERA5 overestimates the Tmin positive trends and outperforms for maximum temperature. Additionally, they point out that even though ERA5 can reproduce most of the hydrological variables over their domain of interest, they should be used cautiously, since they were not able to reproduce temperature and precipitation trends [29].

Moreover, remarkably enough, Gleixner et al. [21], while assessing the performance of ERA5 for the temperature simulation over East Africa, concluded that the correlation of the database with station data is very high, a result that agrees with our findings over Europe. However, the authors mention that while ERA5 data are clearly improved, in comparison to predecessor datasets, they were not able to capture the observed long-term trends, both for temperature and precipitation values. Exploring the ERA5 performance on the reproduction of temperature from another point of view (mean diurnal temperature cycle), over the Canadian Prairies, the reanalysis dataset was found to be significantly improved, and is recommended to be used for agricultural modeling over the domain of study. In comparison to observational data, ERA5 mean monthly temperature biases are small and do not exceed ±0.3 °C [30]. Nonetheless, it is pointed out that the assessment of precipitation is much more challenging with different biases depending on the seasonal scale of the analysis. Comparing the examined reanalysis dataset with its previous version, as well as with buoy data over the Arctic Sea near-surface air temperature, Wang et al. [31] found positive (warm) biases with the larger ones in winter, spring and autumn, and smaller ones during summer which, according to the authors, could be attributed to the difference in height with the observations. This is a common finding in our study, since ERA5 performs differently for the summer months over the European domain. The evaluation and intercomparison of different reanalysis datasets from Keller and Wahl [32] led to the conclusion that ERA5 performs quite satisfactorily for most examined climate indices in the European region. Finally, the higher resolution of ERA5 seems to be a major improving factor in the representation of temperature in mountain zones and valleys according to Rakhmatova et al. [33] over the Uzbekistan region. It is also concluded that the temperature characteristics of the examined area are in good agreement with the observations of ground stations, and the authors suggest that ERA5 can also be used for the computation of drought indices.

Overall, regarding the evaluation of ERA5 in different regions from a climatological point of view, and for different meteorological parameters, all authors agree that this updated reanalysis dataset has been substantially improved to its previous versions. However, aside from the strengths, detailed analysis of ERA5 revealed some limitations as well, mainly over areas with sharp orography, a complex terrain, and sparse observational network. Additionally, several studies concluded that the time scale (e.g., annual, seasonal, monthly) with which each analysis is conducted plays an important role in the performance of ERA5.

5. Conclusions

The main concluding remarks of the present study are the following:

• In general, ERA5 captures the spatial distribution of mean temperature over Europe, however in latitudes higher than 55° N, ERA5 presents some weaknesses in simulating the temperature values (especially over the Scandinavian region).
• Moreover, the complex topography of certain areas affects the performance of ERA5 (e.g., the Alps and Mediterranean). The results must be interpreted with caution in areas with complex terrain, since the alternation between land and sea plays an important role in the observed differences between ERA5 and E-OBS. In case there is a need to correct these temperature datasets, and especially extreme temperatures, statistics of extremes should be utilized [34,35,36].
• Analogous to the annual ones were the results when the analysis was conducted on a seasonal scale.
• The comparison with the station network revealed that in southern Europe, the examined parameter is in many cases underestimated.
• Regarding extreme low temperatures, the weakest performance of ERA5 was noted over the northern and southern regions of Europe.
• The results of the 95th percentile are different, and ERA5 generally overestimates the high mean temperatures (not statistically significant differences).
• The examination of the temperature statistics among the compared datasets showed that ERA5 presents more extreme values (southern to 40° latitude) and less extreme temperatures in areas over the Black Sea.
• In Scandinavia, ERA5 temperatures are often more extreme than the observational ones.