Next Article in Journal
Prediction of Indoor Air Temperature Using Weather Data and Simple Building Descriptors
Previous Article in Journal
Influenza Vaccination of Pregnant Women and Serious Adverse Events in the Offspring
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Excess Mortality in Istanbul during Extreme Heat Waves between 2013 and 2017

1
Department of Public Health, Cerrahpasa Faculty of Medicine, Istanbul University-Cerrahpasa, Kocamustafapasa, Fatih, 34098 Istanbul, Turkey
2
Sabanci University Istanbul Policy Center, Bankalar Caddesi, No:2 Karaköy Minerva Han, 34420 Istanbul, Turkey
3
Institut national de santé publique du Québec, 945 Avenue Wolfe, Québec, QC G1V 5B3, Canada
4
Department of Public Health, Istanbul Medipol University School of Medicine, Kavacık mah. Ekinciler cad. No:19 Kavacık Kavşağı, Beykoz, 34810 Istanbul, Turkey
5
Cerrahpasa Faculty of Medicine, Istanbul University-Cerrahpasa, Kocamustafapasa, Fatih, 34098 Istanbul, Turkey
6
Institut national de santé publique du Québec and Université Laval, 945 Avenue Wolfe, Québec, QC G1V 5B3, Canada
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2019, 16(22), 4348; https://doi.org/10.3390/ijerph16224348
Submission received: 19 September 2019 / Revised: 5 November 2019 / Accepted: 5 November 2019 / Published: 7 November 2019
(This article belongs to the Section Climate Change)

Abstract

:
Heat waves are one of the most common direct impacts of anthropogenic climate change and excess mortality their most apparent impact. While Turkey has experienced an increase in heat wave episodes between 1971 and 2016, no epidemiological studies have examined their potential impacts on public health so far. In this study excess mortality in Istanbul attributable to extreme heat wave episodes between 2013 and 2017 is presented. Total excess deaths were calculated using mortality rates across different categories, including age, sex, and cause of death. The analysis shows that three extreme heat waves in the summer months of 2015, 2016, and 2017, which covered 14 days in total, significantly increased the mortality rate and caused 419 excess deaths in 23 days of exposure. As climate simulations show that Turkey is one of the most vulnerable countries in the Europe region to the increased intensity of heat waves until the end of the 21st century, further studies about increased mortality and morbidity risks due to heat waves in Istanbul and other cities, as well as intervention studies, are necessary.

1. Introduction

Heat waves are one of the most common direct impacts of anthropogenic climate change. Frequency, intensity, and duration of warm spells and heat waves have increased due to global temperature rise [1]. The number and intensity of heat waves in the Mediterranean region, including western Turkey and the Black Sea coastline, have significantly increased as well [2,3]. Excess mortality due to heat wave episodes has been documented after several long and widespread heat wave episodes over the last 30 years, for example: the 1995 heat wave in the upper midwestern US, with more than 700 excess deaths in Chicago only [4]; the 2003 European heat wave episode with more than 70,000 total excess deaths in 16 European countries [5]; and the 2010 heat wave in India with 1344 excess deaths in the city of Ahmadabad alone [6].
Scientific research, local preparedness, and public health interventions clearly reduce the risk of mortality during heat waves. In France, for example, where a severe 2006 heat wave caused about 2065 excess deaths, the model predictions for expected mortality were much higher [7].
The number of hot days and frequency and duration of heat waves increased in the western part of Turkey between 1965 and 2006 [8]. The total number of heat wave episodes increased all over Turkey between 1971 and 2016, according to an analysis using the data of meteorological stations [9]. The heat waves in 2000, 2007, and 2010 in Turkey were investigated from a meteorological perspective [10], and there were several news reports regarding intensive heat waves and health-related deaths in southeastern Europe, including Turkey [11,12,13]. Temperature rise and increasing inter-annual temperature variability, especially during the summer seasons in the Mediterranean Basin until the end of the century, are evident from climate models simulations, and therefore, the quantity and intensity of the heat waves are expected to increase in Turkey due to global anthropogenic climate change [14].
However, there are very few studies on mortality related to heat waves available in Turkey. A study on the effects of heat on mortality in European and Eastern–Southern Mediterranean cities including Istanbul shows that a 1 °C increase above threshold causes a 2.4% increase in mortality on all ages [15]. Oray et al. [16] reported in an observational study that an increase in mortality was documented in an emergency department in Izmir during a nine-day period when the air temperature values were higher than the seasonal average of the city in June 2016. Heat-related mortality studies based on health statistics have been very limited, although Turkey has been affected by many well-studied Euro-Mediterranean heat waves, including one in the summer of 2003. Turkey is one of the most vulnerable countries to the increased intensity of heat waves until the end of the 21st century. Amengual et al. states that “By 2075–2094, projected extreme heat wave amplitude increases range from 2 °C to 4 °C per heat wave day in southern Spain, ... Italy, Greece, and Turkey...” [17].
In this context of increasing meteorological risks, a description of current impacts of heat waves is a first necessary step to begin addressing this public health risk now and in the future. Most existing research in this domain is North American and European, while the higher risks are probably in countries where such studies remain to be performed; however, they often are difficult to implement as detailed and high quality data availability remains a serious problem [18]. In this research, excess mortality in Istanbul attributable to extreme heat wave episodes between 2013 and 2017 is examined using total excess deaths in general and using mortality rates across different categories, including age, sex, and cause of death, in order to begin identify potential subgroups for preventative public health actions.

2. Methods

2.1. Study Area

Istanbul is the most populated city in Turkey, with 15 million inhabitants representing 18.5% of Turkey’s total population, according to the 2017 census [19]. The city is located in the northwestern part of the country at 41°01′ N, 28°58′ E, divided between the European and Asian continents by the Bosporus Strait. Despite its large population, the city covers only 5461 square kilometers; thus, its population density is high at 2747 inhabitants per km2. The southern half of the city, stretched along the coasts of the Bosporus Strait and Marmara Sea, is where most of the urban settlements and commercial areas are located. The climate in the city’s southern settlements has the general characteristics of the Mediterranean type climate, characterized by mostly warm and dry days in the summer and cold and wet days in the winter [20]. The mean annual temperature of the city is 13.8 °C, with the highest monthly average in August of 22.8 °C and the lowest monthly average in February of 5.3 °C. The warming trend in Istanbul is clear, as the observed mean annual temperature increased 0.94 °C between 1912 and 2016 [21]. Recently, however, settlements in the city have started to expand northward into rural and forested areas where the climate is modified by the cooler Black Sea patterns, which bring colder and rainier weather than a typical Mediterranean climate both for winter and summer seasons [20].

2.2. Mortality Data

Daily mortality records for the three summer months (June–August) every year from 2013 to 2017 were obtained from the Istanbul Health Directorate with their permission. The data starting from 2013 only was used because reliable information on the daily number of deaths in Istanbul was only available from this year. It included the deaths of Istanbul residents of all ages, while abortions and in-utero deaths were excluded, although reported by the same Death Notification System. The daily number of deaths was analyzed according to sex, age, and cause of death. The age at death was categorized into four groups: 0–14, 15–64, 65–74, and ≥75. The 75+ is a separate age group in order to see the vulnerability of elderly people. The cause of death was coded according to the International Classification of Diseases, Revision 10 (ICD 10). All non-accidental and non-violent deaths were classified as deaths due to cardiovascular diseases (I10–I99), respiratory diseases (J00–J99), or others. Crude, sex-specific, age-specific, and cause-specific mortality rates were calculated using the population data, which was obtained from the Turkish Statistical Institute, in order to estimate the excess death caused by extreme heat wave events.

2.3. Meteorological Data

The meteorological data used for this study included daily mean, maximum, and minimum temperatures for the summer months (June–August) of 2013–2017. The First Region Directorate (Istanbul) of the Turkish State Meteorological Service provided the meteorological data collected from 24 weather stations in the metropolitan city of Istanbul. The average of the mean daily temperatures of all 24 stations was used for the analysis, since the death records were collected for all districts of Istanbul in total, regardless of the place of death.

2.4. Definition of Heat Wave

Although there is no universal definition of a heat wave, periods of extreme temperatures in which the average daily temperature is greater than the 90th or 95th percentile for the region over a period of two or more days are usually considered as heat wave conditions [22,23]. In this study, an extreme heat wave episode was defined as a period with a daily average of the mean temperature above the 95th percentile for at least three consecutive days. Three days (lag days) were added at the end of the defined heat wave period in order to take into account the delayed impact of heat waves on mortality [24]. The reference period used for comparison was determined using the same days of the nearest week corresponding with the defined extreme heat wave in other years. If these dates coincided with any other heat wave episode, these days were excluded and these reference periods had 3 or 4 years, depending on relevant available data.

2.5. Statistical Analysis

SPSS v21.0 (SPSS Inc., Chicago, IL, USA) and Microsoft Office Excel (Microsoft Corporation, Redmond, WA, USA) were used for statistical analysis. Crude, sex-specific, age-specific, and cause-specific death rates were calculated by dividing the number of deaths over the multiplication of the population with the number of days. The daily excess number of deaths were thus calculated as the difference in death rates during the heat wave (including the lag days) and the comparison period, multiplied by the total population during the heat wave. Z tests of the difference of natural logarithms were used to compare death rates during the heat wave (including three added lag days) to the death rates during the reference period.
Z = ln ( T h e a t   w a v e ) ln ( T r e f e r e n c e   p e r i o d ) v a r ( T h e a t   w a v e ) ( T h e a t   w a v e ) 2 + v a r ( T r e f e r e n c e   p e r i o d ) ( T r e f e r e n c e   p e r i o d ) 2 ,
where
v a r ( T ) = m ( P J ) 2 ,
where Theat wave is the crude daily death rate for a heat wave; Treference period is the crude daily death rate for the reference period; m is the number of deaths during the period; and PJ is the number of persons-days at risk during the period.
Normal approximation was used to calculate the p-value. Risk ratio (RR) was calculated by dividing the death rate in the heat wave by the death rate in the reference period, and the CI95% was calculated. A p-value < 0.05 was accepted for statistical significance.

3. Results

For summer months, the average daily number of deaths in Istanbul hovered between 140 and 161 from 2013 to 2017. Depending on the year, the average of summer months daily mean temperatures was between 23.4 °C and 24.2 °C. Annual population numbers, daily average number of deaths according to sex and age group, and the daily minimum and maximum average temperatures are shown in Table 1.
Four extreme heat wave episodes were identified between 2013 and 2017, based on the definition above; one heat wave happened in 2015, one in 2016, and two in 2017. There were no extreme heat wave episodes in 2013 and 2014. The dates and average of daily mean temperatures of the heat waves and their reference periods, crude death rates in each heat wave episode, and the risk ratios (RR) compared to the crude death rates in reference periods for each heat wave episode are shown in Table 2.
The first heat wave (Heat Wave 1) started on 27 July 2015 and lasted four days, and the three-day lag period ended on 2 August 2015. Risk ratio for all deaths during Heat Wave 1 was 1.11 (CI95% 1.04–1.18) and significantly increased (p = 0.002). This ratio corresponded with 17 daily and 118 total excess deaths in Heat Wave 1 in 2015.
The second heat wave (Heat Wave 2) started on 5 August 2016 and lasted seven days, and the three-day lag period ended on 14 August 2016. Risk ratio for all deaths during Heat Wave 2 was 1.06 (CI95% 1.00–1.12) and significantly increased (p = 0.037). This ratio corresponded with 10 daily and 96 total excess deaths in Heat Wave 2 in 2016.
The third heat wave (Heat Wave 3) started on 29 June 2017 and lasted three days, and the three-day lag period ended on 4 July 2017. Risk ratio for all deaths during Heat Wave 3 was 1.21 (CI95% 1.14–1.30) and significantly increased (p < 0.001). This ratio corresponded with 34 daily and 205 total excess deaths in Heat Wave 3 in 2017.
The fourth heat wave (Heat Wave 4) started on 5 August 2017 and lasted three days, and the three-day lag period ended on 10 August 2017. Risk ratio for all deaths during Heat Wave 4 was 0.99 (CI95% 0.92–1.07) but was not significantly increased unlike the other three heat wave episodes.
The analysis shows that three extreme heat waves in the summer months of 2015, 2016, and the first heat wave in 2017, which covered 14 days in total, significantly increased the mortality rate and caused 419 excess deaths in 23 days, including the three-day lags. Excess death counts, daily mean temperatures, and temperature of reference periods are shown for each heat wave in Figure 1.
Sex-specific death rates and risk ratios for men and women in the extreme heat waves between 2013 and 2017 are shown in Table 3. The death risk for men increased only in Heat Wave 3 in 2017 (RR: 1.14 (CI95% 1.04–1.26)). The death risk for women, on the other hand, increased in Heat Wave 1 in 2015 (RR: 1.25 (CI95% 1.15–1.37)) and Heat Wave 3 in 2016 (RR: 1.29 (CI95% 1.18–1.42)). Neither sex-specific death risks were significantly increased during Heat Wave 2 in 2016.
Age-specific death rates and risk ratios for the age groups 0–14 and above 75 are shown in Table 4. While the death risk did not increase for the 0–14, 15–64, and 65–74 age groups (data not presented here), there was an increased risk for the ≥75 age group in Heat Wave 1 in 2015 (RR: 1.16 (CI95% 1.06–1.27)) and Heat Wave 3 in 2017 (RR: 1.38 (CI95% 1.26–1.52)).
Cause-specific death rates and risk ratios for cardiovascular, respiratory, and others are shown in Table 5. While there was no increased death risk for respiratory diseases, death risk caused by cardiovascular diseases significantly increased in Heat Wave 1 in 2015 (RR: 1.17 (CI95% 1.05–1.31)) and Heat Wave 3 in 2017 (RR: 1.32 (CI95% 1.18–1.48)).
Risk ratios for crude, sex-specific, age-specific, and cause-specific deaths with their CI 95% in all episodes are shown in Figure 2. Although the death risk for all deaths were significantly increased also in Heat Wave 2 in 2016, increased death risk in Heat Waves 1 and 3 in 2015 and 2017 were stronger for all deaths, female deaths, elderly deaths, and deaths due to cardiovascular diseases.

4. Discussion

This study showed that three heat wave episodes in Istanbul in 2015, 2016, and 2017 caused an increased risk of mortality: 11%, 6%, and 21%, respectively. The total number of excess deaths during the heat wave episodes for three years was 419. This result is consistent with similar studies in different cities. Two studies about summer heat waves in Vienna and in Switzerland in 2003 showed a 13% and 7% increase in daily mortality, respectively [25,26]. On the other hand, the risk ratio is lower than those reported in some other studies, such as a 55% increase for France in 2003 [27], 38% increase for Belgrade in 2013 [28], and 33% for Montreal in 2010 [29]. Various results may be due in part to the duration of the heatwave and magnitude of extreme heat, as well as to the age structure, prevalence of chronic diseases, socio-economic deprivation, the level of emergency preparedness, access to air conditioning, and other factors [30,31]. Some analyses were done to try to characterize some risk factors that could be amenable to preventative public health interventions; below are discussed the importance of the characteristics of the heat wave, age, and categorical cause of death.
In an epidemiological study about the relation of heat and mortality in London between 1976 and 1996, Hajat et al. [32] found out that the “heat episodes of long duration and of highest temperature have the largest mortality effect.” For example, in the Belgrade study in which a high increase in mortality was found, the duration of the heat wave was nine days and mean daily temperature was 6.7–13.1 °C higher than normal [28]. In this study, the longest heat wave lasted seven days and the mean daily temperatures during the heat wave episodes were 2.3–4 °C higher than the monthly normal [33].
These results seem to show that women are more vulnerable to heat waves than men, but RRs are significantly different only for wave 1. Many other studies show increased mortality in women during heat waves, possibly because women are less heat tolerant than men due to different thermoregulatory and physiological mechanisms [26,27,34,35,36].
Although younger compared to Turkey’s age distribution in general, Istanbul has an aging population. People over 65 constituted 6.7% of the population of Istanbul in 2018 and 8.8% of the total population in Turkey. The ratio was 5.2% in 2008 and is expected to be more than 7% after 2020 [37]. Elderly people older than 75 were more vulnerable to heat waves: the death risk for the elderly population increased by 16% in 2015 and 38% in 2017. Death risks were not significantly increased in other age groups. This result is consistent with many other studies where the association of mortality and heat waves is much stronger for people older than 75 years [27,28,38,39,40,41].
Cardiovascular deaths seemed related to heat waves in Istanbul, while other cause-specific death rates, including those due to respiratory diseases, did not increase. The death risk for cardiovascular disease increased 17% in the 2015 heat wave and 32% in the 2017 heat wave, both higher than the increase of crude death rates. This result is also consistent with previous studies that found that while cardiovascular deaths were related to heat, especially for the elderly [42,43], heat does not seem to have an effect on respiratory mortality [44,45]. It could be related in part to air pollution levels during the heat waves. This study did not take the impact of air pollution into account, although ozone pollution may have an impact on the results as seen in Brisbane during the 2004 heat wave. Thus, this topic needs further research [46].
Although the death risk was increased in three (out of four) heat waves in Istanbul, the increase in the 2016 heat wave was not as high as the other two heat waves in 2015 and 2017. The highest risk ratio was for Heat Wave 3 between 29 June and 1 July 2017, i.e., early in the summer, although with a shorter duration. This may be caused by the period of time necessary for adaptation to heat, because people are less tolerant to extreme temperature during the first weeks of summer than the late summer [47], similar to the results in a study of heat waves in London [32]. Additionally, the temperatures during Heat Wave 1 and 3 in 2015 and 2017, respectively, were 4 °C higher than the monthly normals, while in Heat Wave 2 in 2016 the temperature was only 2.3 °C hotter than normal. The second heat wave in 2017 did not increase death rates for the general population or the elderly. This may be seen as an example of the harvesting effect: although the first heat wave in 2017 increased the mortality rate particularly for the elderly, no statistically significant mortality displacement was found. On the other hand, because of the first heat wave in June, and thanks to the warning reports by the meteorological services and media [48,49], people could have taken their own precautions, including moving out of the city, taking advantage of the August holiday season.
Some limitations of this study need explanation. There are two general methodological approaches for heat (or heat wave) and mortality studies. In time series studies, the impact of temperature on mortality is assessed in a defined geographic region over a substantial time period. In heat wave studies, relative risk of death rates in defined heat wave episodes and control episodes (i.e., several days before or after the heat wave in the same year or the similar period of previous years) are examined. An additional approach may be possible if deaths due to heat (or heatstroke) are explicitly recorded and provided as a cause of death, although this kind of data usually underestimates heat-related deaths [50,51]. In this study, the design was imposed by the data available. Although mortality data has been collected and recorded by the health authorities in Turkey for a long time, the daily number of deaths and their categorization according to basic demographics and cause of death have become available only quite recently. Therefore, the long-term trends in daily mortality data that are necessary for time series studies could not be assessed, even for crude death rates. This type of study will only be possible in the future after a sufficient amount of time has passed for time series studies. It is also important to note that heat stress or heatstroke are not recorded in the death certificates that are made available by the regional health directorates.
Another type of limitation is linked to available data, which has been collected for only a short period on a daily basis, and the lack of information on several potential confounders. As in many such ecological studies, residual confounding is certainly present but could not be estimated properly. Future improvements on data availability for longer periods on variables such as socio-economic levels, quality of dwellings, intra-urban heat islands, and the like would allow a better estimation of risks levels and improve decision-making for population health measures.
Several meteorological indicators may be used in heat wave studies, including mean temperature, apparent temperature, and heat index [23]. Daily minimum temperature is also important because of the recovery effect of low nighttime temperatures [25]. The simple average of the daily mean temperature measured by all meteorological stations in Istanbul was used here, because many factors possibly contributing to mortality, such as location and circumstances of deaths, time of the day, suddenly or after a being hospitalized, etc., could not be distinguished. Furthermore, Istanbul is a big metropolitan city, and the hospitals where the deaths took place and were recorded may be in different districts of the city than the locations in which the onset of the heatstroke, cardiac arrest, or any other events leading directly to death occurred. In addition, using the simple average of all meteorological stations may smooth the extreme measurements of temperatures in the city center, because some of the stations are close to the forested and rural areas of the city.
The definition of a heat wave period is important for the study design. Researchers may choose one of the definitions, such as taking a certain temperature threshold, which is defined by considering the city’s climate normal in comparison to the days with a temperature exceeding a certain percentile (e.g., 90th, 95th, 99th, etc.). Additionally, the extreme heat event must last more than two or three days [52,53]. Defining a warning threshold for a city or region by looking at which extreme heat events make excess death significant is important for public health interventions [54,55]. The heat wave definition in this study (temperatures above the 95th percentile sustained for more than three days) showed a significant relationship between heat events and mortality. However, Istanbul and other cities in Turkey do not have any heat wave warning system or specific local thresholds for determining a heat wave. Similar research projects in other regions of Turkey would be useful for creating a local definition of a heat wave that can highlight overmortality risks.

5. Conclusions

This study showed that three heat wave episodes in Istanbul in 2015, 2016, and 2017 caused an increased risk of all-cause mortality: 11%, 6%, and 21%, respectively, without any harvesting effect. The total number of excess deaths during the heat wave episodes for three years was 419. These results are in line with similar studies worldwide in comparable situations for amplitude of effect, age groups most affected, and the importance of cardiovascular deaths in the episodes. Some limitations due to data availability and quality highlight potential future improvements.
Since there are very limited studies about heat waves and mortality in Istanbul related to the past heat waves such as the ones in 2000, 2003, 2007, and 2010, it could not be assessed whether there were any improvement over time in terms of adaptation or behavioral change, as no historic reliable data is available. Since there are no public health intervention plans for heat waves at the local or national levels, further studies about increased death risks due to heat waves in Istanbul and other cities, as well as intervention studies, are necessary in the light of the impacts presented in this study. Other studies on heat-related ambulatory and hospital morbidity could be of great interest too for Istanbul and other regions of Turkey.

Author Contributions

Conceptualization, G.C., Ü.Ş., and P.G.; Data curation, U.S., B.K., H.C.A., B.İ., Ö.A.S., H.K., and U.G.; Formal analysis, G.C., U.S., B.K., and H.C.A.; Funding acquisition, G.C. and Ü.Ş.; Investigation, G.C., Ü.Ş., U.S., B.K., H.C.A., B.İ., Ö.A.S., and H.K.; Methodology, G.C., Ü.Ş., G.L., R.B., and P.G.; Project administration, G.C., Ü.Ş., and P.G.; Resources, G.C., Ü.Ş., U.S., B.K., H.C.A., B.İ., Ö.A.S., and H.K.; Software, M.D.; Supervision, G.C., Ü.Ş., M.D., G.L., R.B., and P.G.; Validation, M.D., G.L., R.B., and P.G.; Visualization, G.C., Ü.Ş., U.S., B.K., H.C.A., and B.İ.; Writing—original draft, G.C., Ü.Ş., U.S., B.K., H.C.A., B.İ., Ö.A.S., H.K., and U.G.; Writing—review and editing, G.C., Ü.Ş., U.S., M.D., B.K., H.C.A., B.İ., Ö.A.S., G.L., R.B., H.K., U.G., and P.G. All authors approved the final version.

Funding

This work was supported by the Scientific Research Projects Coordination Unit of Istanbul University-Cerrahpasa. Project number: BYP-2018-31650.

Acknowledgments

The authors would like to thank The First Region Directorate (Istanbul) of the Turkish State Meteorological Service and Health Directorate of Istanbul for providing data.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Perkins, S.; Alexander, L.; Nairn, J. Increasing frequency, intensity and duration of observed global heatwaves and warm spells. Geophys. Res. Lett. 2012, 39. [Google Scholar] [CrossRef]
  2. Kuglitsch, F.G.; Toreti, A.; Xoplaki, E.; Della-Marta, P.M.; Zerefos, C.S.; Türkeş, M.; Luterbacher, J. Heat wave changes in the eastern Mediterranean since 1960. Geophys. Res. Lett. 2010, 37. [Google Scholar] [CrossRef] [Green Version]
  3. Diffenbaugh, N.S.; Pal, J.S.; Giorgi, F.; Gao, X. Heat stress intensification in the Mediterranean climate change hotspot. Geophys. Res. Lett. 2007, 34. [Google Scholar] [CrossRef] [Green Version]
  4. Semenza, J.C.; Rubin, C.H.; Falter, K.H.; Selanikio, J.D.; Flanders, W.D.; Howe, H.L.; Wilhelm, J.L. Heat-related deaths during the July 1995 heat wave in Chicago. N. Engl. J. Med. 1996, 335, 84–90. [Google Scholar] [CrossRef]
  5. Robine, J.-M.; Cheung, S.L.K.; Le Roy, S.; Van Oyen, H.; Griffiths, C.; Michel, J.-P.; Herrmann, F.R. Death toll exceeded 70,000 in Europe during the summer of 2003. C. R. Biol. 2008, 331, 171–178. [Google Scholar] [CrossRef] [PubMed]
  6. Azhar, G.S.; Mavalankar, D.; Nori-Sarma, A.; Rajiva, A.; Dutta, P.; Jaiswal, A.; Sheffield, P.; Knowlton, K.; Hess, J.J. Heat-related mortality in India: Excess all-cause mortality associated with the 2010 Ahmedabad heat wave. PLoS ONE 2014, 9, e91831. [Google Scholar] [CrossRef]
  7. Fouillet, A.; Rey, G.; Wagner, V.; Laaidi, K.; Empereur-Bissonnet, P.; Le Tertre, A.; Frayssinet, P.; Bessemoulin, P.; Laurent, F.; De Crouy-Chanel, P. Has the impact of heat waves on mortality changed in France since the European heat wave of summer 2003? A study of the 2006 heat wave. Int. J. Epidemiol. 2008, 37, 309–317. [Google Scholar] [CrossRef]
  8. Unal, Y.S.; Tan, E.; Mentes, S.S. Summer heat waves over western Turkey between 1965 and 2006. Theor. Appl. Climatol. 2013, 112, 339–350. [Google Scholar] [CrossRef]
  9. Gurkan, H.; Eskioglu, O.; Yazıcı, B.; Sensoy, S.; Komuscu, A.U.; Calik, Y. Projected trends in heat and cold waves under effect of climate change. In Proceedings of the 8th Atmospheric Sciences Symposium, Istanbul, Turkey, 1–4 November 2017; pp. 111–122. [Google Scholar]
  10. Demirtaş, M. High impact heat waves over the Euro-Mediterranean region and Turkey-in concert with atmospheric blocking and large dynamical and physical anomalies. Anadolu Üniv. Bilim. Teknol. Derg. A Uygul. Bilimler Mühendis. 2017, 18, 97–114. [Google Scholar] [CrossRef]
  11. Reuters. Available online: https://www.reuters.com/article/environment-europe-heatwave-dc/heatwave-causes-deaths-in-greece-romania-idUSL2515969220070625 (accessed on 5 June 2019).
  12. Daily Sabah Europe. Available online: https://www.dailysabah.com/europe/2017/08/05/heat-wave-affects-southern-europe-turkey-gives-a-taste-of-the-future (accessed on 5 June 2019).
  13. The Guardian. Available online: https://www.theguardian.com/environment/2000/jul/09/weather.climatechange (accessed on 5 June 2019).
  14. Ozturk, T.; Ceber, Z.P.; Türkeş, M.; Kurnaz, M.L. Projections of climate change in the Mediterranean Basin by using downscaled global climate model outputs. Int. J. Climatol. 2015, 35, 4276–4292. [Google Scholar] [CrossRef]
  15. Leone, M.; D’Ippoliti, D.; De Sario, M.; Analitis, A.; Menne, B.; Katsouyanni, K.; de’Donato, F.K.; Basagana, X.; Salah, A.B.; Casimiro, E.; et al. A time series study on the effects of heat on mortality and evaluation of heterogeneity into European and Eastern-Southern Mediterranean cities: Results of EU CIRCE project. Environ. Health 2013, 12, 55:1–55:12. [Google Scholar] [CrossRef] [PubMed]
  16. Oray, N.C.; Oray, D.; Aksay, E.; Atilla, R.; Bayram, B. The impact of a heat wave on mortality in the emergency department. Medicine 2018, 97, e13815. [Google Scholar] [CrossRef] [PubMed]
  17. Amengual, A.; Homar, V.; Romero, R.; Brooks, H.E.; Ramis, C.; Gordaliza, M.; Alonso, S. Projections of heat waves with high impact on human health in Europe. Glob. Planet. Chang. 2014, 119, 71–84. [Google Scholar] [CrossRef]
  18. Campbell, S.; Remenyi, T.A.; White, C.J.; Johnston, F.H. Heatwave and health impact research: A global review. Health Place 2018, 53, 210–218. [Google Scholar] [CrossRef] [PubMed]
  19. TUIK. Available online: http://www.tuik.gov.tr/PreIstatistikTablo.do?istab_id=1590 (accessed on 19 January 2019).
  20. Ezber, Y.; Lutfi Sen, O.; Kindap, T.; Karaca, M. Climatic effects of urbanization in Istanbul: A statistical and modeling analysis. Int. J. Climatol. 2007, 27, 667–679. [Google Scholar] [CrossRef]
  21. Toros, H.; Abbasnia, M.; Sagdic, M.; Tayanç, M. Long-Term Variations of Temperature and Precipitation in the Megacity of Istanbul for the Development of Adaptation Strategies to Climate Change. Adv. Meteorol. 2017. [Google Scholar] [CrossRef]
  22. Perkins, S.E.; Alexander, L.V. On the measurement of heat waves. J. Clim. 2013, 26, 4500–4517. [Google Scholar] [CrossRef]
  23. McGregor, G.R.; Bessemoulin, P.; Ebi, K.; Menne, B. Heatwaves and Health: Guidance on Warning-System Development; World Meteorological Organization and World Health Organization: Geneva, Switzerland, 2015. [Google Scholar]
  24. Huang, C.; Barnett, A.G.; Wang, X.; Tong, S. Effects of extreme temperatures on years of life lost for cardiovascular deaths: A time series study in Brisbane, Australia. Circ. Cardiovasc. Qual. Outcomes 2012, 5, 609–614. [Google Scholar] [CrossRef]
  25. Grize, L.; Huss, A.; Thommen, O.; Schindler, C.; Braun-Fahrlander, C. Heat wave 2003 and mortality in Switzerland. Swiss Med. Wkly. 2005, 135, 200–205. [Google Scholar]
  26. Hutter, H.-P.; Moshammer, H.; Wallner, P.; Leitner, B.; Kundi, M. Heatwaves in Vienna: Effects on mortality. Wien. Klin. Wochenschr. 2007, 119, 223–227. [Google Scholar] [CrossRef]
  27. Fouillet, A.; Rey, G.; Laurent, F.; Pavillon, G.; Bellec, S.; Guihenneuc-Jouyaux, C.; Clavel, J.; Jougla, E.; Hémon, D. Excess mortality related to the August 2003 heat wave in France. Int. Arch. Occup. Environ. Health 2006, 80, 16–24. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Bogdanović, D.; Milošević, Z.; Lazarević, K.K.; Dolicanin, Z.C.; Ranđelović, D.; Bogdanović, S.D. The impact of the July 2007 heat wave on daily mortality in Belgrade, Serbia. Cent. Eur. J. Public Health 2013, 21, 140–145. [Google Scholar] [CrossRef] [PubMed]
  29. Bustinza, R.; Lebel, G.; Gosselin, P.; Bélanger, D.; Chebana, F. Health impacts of the July 2010 heat wave in Quebec, Canada. BMC Public Health 2013, 13, 56. [Google Scholar] [CrossRef] [PubMed]
  30. Reid, C.E.; O’neill, M.S.; Gronlund, C.J.; Brines, S.J.; Brown, D.G.; Diez-Roux, A.V.; Schwartz, J. Mapping community determinants of heat vulnerability. Environ. Health Perspect. 2009, 117, 1730–1736. [Google Scholar] [CrossRef] [PubMed]
  31. Rey, G.; Fouillet, A.; Bessemoulin, P.; Frayssinet, P.; Dufour, A.; Jougla, E.; Hémon, D. Heat exposure and socio-economic vulnerability as synergistic factors in heat-wave-related mortality. Eur. J. Epidemiol. 2009, 24, 495–502. [Google Scholar] [CrossRef]
  32. Hajat, S.; Kovats, R.S.; Atkinson, R.W.; Haines, A. Impact of hot temperatures on death in London: A time series approach. J. Epidemiol. Community Health 2002, 56, 367–372. [Google Scholar] [CrossRef]
  33. Tarım, T.C. Ve Orman Bakanlığı Meteoroloji Genel Müdürlüğü. İllere Ait Mevsim Normalleri(1981–2010). Available online: https://www.mgm.gov.tr/veridegerlendirme/il-ve-ilceler-istatistik.aspx?k=H&m=ISTANBUL (accessed on 5 June 2019).
  34. Li, J.; Xu, X.; Yang, J.; Liu, Z.; Xu, L.; Gao, J.; Liu, X.; Wu, H.; Wang, J.; Yu, J. Ambient high temperature and mortality in Jinan, China: A study of heat thresholds and vulnerable populations. Environ. Res. 2017, 156, 657–664. [Google Scholar] [CrossRef]
  35. Díaz, J.; Jordán, A.; García, R.; López, C.; Alberdi, J.; Hernández, E.; Otero, A. Heat waves in Madrid 1986–1997: Effects on the health of the elderly. Int. Arch. Occup. Environ. Health 2002, 75, 163–170. [Google Scholar] [CrossRef]
  36. Rooney, C.; McMichael, A.J.; Kovats, R.S.; Coleman, M.P. Excess mortality in England and Wales, and in Greater London, during the 1995 heatwave. J. Epidemiol. Community Health 1998, 52, 482–486. [Google Scholar] [CrossRef] [Green Version]
  37. TUIK. Adrese Dayalı Nüfus Kayıt Sistemi. Available online: http://www.tuik.gov.tr/PreTablo.do?alt_id=1059 (accessed on 19 January 2019).
  38. Conti, S.; Meli, P.; Minelli, G.; Solimini, R.; Toccaceli, V.; Vichi, M.; Beltrano, C.; Perini, L. Epidemiologic study of mortality during the Summer 2003 heat wave in Italy. Environ. Res. 2005, 98, 390–399. [Google Scholar] [CrossRef]
  39. Johnson, H.; Kovats, R.S.; McGregor, G.; Stedman, J.; Gibbs, M.; Walton, H.; Cook, L.; Black, E. The impact of the 2003 heat wave on mortality and hospital admissions in England. Health Stat. Q. 2004, 15, S126. [Google Scholar] [CrossRef]
  40. Kovats, R.S.; Johnson, H.; Griffith, C. Mortality in southern England during the 2003 heat wave by place of death. Health Stat. Q 2006, 29, 6–8. [Google Scholar]
  41. Canoui-Poitrine, F.; Cadot, E.; Spira, A. Excess deaths during the August 2003 heat wave in Paris, France. Rev. D’épidémiol. Santé Publique 2006, 54, 127–135. [Google Scholar] [CrossRef] [Green Version]
  42. Yin, Q.; Wang, J. The association between consecutive days’ heat wave and cardiovascular disease mortality in Beijing, China. BMC Public Health 2017, 17, 223. [Google Scholar] [CrossRef]
  43. Wainwright, S.H.; Buchanan, S.D.; Mainzer, M.; Parrish, R.G.; Sinks, T.H. Cardiovascular mortality—The hidden peril of heat waves. Prehosp. Disaster Med. 1999, 14, 18–27. [Google Scholar] [CrossRef]
  44. Ellis, F.; Nelson, F. Mortality in the elderly in a heat wave in New York City, August 1975. Environ. Res. 1978, 15, 504–512. [Google Scholar] [CrossRef]
  45. Bobb, J.F.; Obermeyer, Z.; Wang, Y.; Dominici, F. Cause-specific risk of hospital admission related to extreme heat in older adults. JAMA 2014, 312, 2659–2667. [Google Scholar] [CrossRef]
  46. Tong, S.; Ren, C.; Becker, N. Excess deaths during the 2004 heatwave in Brisbane, Australia. Int. J. Biometeorol. 2010, 54, 393–400. [Google Scholar] [CrossRef]
  47. Koppe, C.; Kovats, S.; Jendritzky, G.; Menne, B. Heat-Waves: Risks and Responses; World Health Organization: Copenhagen, Denmark, 2004. [Google Scholar]
  48. Sözcü. Available online: https://www.sozcu.com.tr/2017/gundem/hava-durumu-1-agustos-asiri-nem-rekor-sicakliklari-getirecek-1955483/ (accessed on 5 June 2019).
  49. Akşam. Available online: https://www.aksam.com.tr/guncel/istanbul-hava-durumu-meteoroloji-sicaklik-uyarisi-6-agustos/haber-649139 (accessed on 5 June 2019).
  50. Basu, R.; Samet, J.M. Relation between elevated ambient temperature and mortality: A review of the epidemiologic evidence. Epidemiol. Rev. 2002, 24, 190–202. [Google Scholar] [CrossRef]
  51. Hajat, S.; Armstrong, B.; Baccini, M.; Biggeri, A.; Bisanti, L.; Russo, A.; Paldy, A.; Menne, B.; Kosatsky, T. Impact of high temperatures on mortality: Is there an added heat wave effect? Epidemiology 2006, 632–638. [Google Scholar] [CrossRef]
  52. Tong, S.; Wang, X.Y.; Barnett, A.G. Assessment of heat-related health impacts in Brisbane, Australia: Comparison of different heatwave definitions. PLoS ONE 2010, 5, e12155. [Google Scholar] [CrossRef] [PubMed]
  53. Xu, Z.; FitzGerald, G.; Guo, Y.; Jalaludin, B.; Tong, S. Impact of heatwave on mortality under different heatwave definitions: A systematic review and meta-analysis. Environ. Int. 2016, 89, 193–203. [Google Scholar] [CrossRef] [PubMed]
  54. Chebana, F.; Martel, B.; Gosselin, P.; Giroux, J.-X.; Ouarda, T.B. A general and flexible methodology to define thresholds for heat health watch and warning systems, applied to the province of Québec (Canada). Int. J. Biometeorol. 2013, 57, 631–644. [Google Scholar] [CrossRef] [PubMed]
  55. Chiu, Y.; Chebana, F.; Abdous, B.; Bélanger, D.; Gosselin, P. Mortality and morbidity peaks modeling: An extreme value theory approach. Stat. Methods Med. Res. 2018, 27, 1498–1512. [Google Scholar] [CrossRef]
Figure 1. Excess death counts, daily mean temperatures, and temperature of reference periods for the Extreme Heat Waves between 2013 and 2017 (27 Jul –2 Aug 2015, 5 Aug–14 Aug 2016, 29 Jun–4 Jul 2017, 5 Aug–10 Aug 2017 respectively).
Figure 1. Excess death counts, daily mean temperatures, and temperature of reference periods for the Extreme Heat Waves between 2013 and 2017 (27 Jul –2 Aug 2015, 5 Aug–14 Aug 2016, 29 Jun–4 Jul 2017, 5 Aug–10 Aug 2017 respectively).
Ijerph 16 04348 g001
Figure 2. The crude death rates and CI95% of the Extreme Heat Waves between 2013 and 2017 (27 Jul–2 Aug 2015, 5 Aug–14 Aug 2016, 29 Jun–4 Jul 2017, 5 Aug–10 Aug 2017 respectively) by categories. (T: Total, M: Male, F: Female, 75+: 75 years old and above, Card: cardiovascular diseases, Resp: Respiratory diseases, Oth: Others).
Figure 2. The crude death rates and CI95% of the Extreme Heat Waves between 2013 and 2017 (27 Jul–2 Aug 2015, 5 Aug–14 Aug 2016, 29 Jun–4 Jul 2017, 5 Aug–10 Aug 2017 respectively) by categories. (T: Total, M: Male, F: Female, 75+: 75 years old and above, Card: cardiovascular diseases, Resp: Respiratory diseases, Oth: Others).
Ijerph 16 04348 g002
Table 1. Descriptive statistics of temperatures and deaths by years (for June–August).
Table 1. Descriptive statistics of temperatures and deaths by years (for June–August).
Characteristic20132014201520162017
Average of Daily Mean Temperatures (°C) (Min–Max)23.7 (18.6–26.3)23.7 (17.5–27.6)23.4 (18.5–27.9)24.2 (17.3–27.2)23.5 (17–29.9)
Number of days with daily average of mean temperatures > 95th percentiles (n)05576
Number of extreme heat waves (n)(days)001(4)1(7)2(3; 3)
Population (n) 14,160,46714,377,01814,657,43414,804,11615,029,231
Daily average number of deaths (n) (Rates/100k)140.0 (0.99)153.2 (1.07)157.7 (1.08)161.9 (1.09)160.9 (1.07)
Daily average number of deaths by sex (n) (Rates/100k)
Male 75.6 (1.06)81.5 (1.13)82.5 (1.12)86.4 (1.16)85.8 (1.14)
Female64.4 (0.91)71.7 (1.00)75.2 (1.03)75.5 (1.02)75.1 (1.00)
Daily average number of deaths by age groups (n) (Rates/100k)
0–149.6 (0.29)10.7 (0.33)10.1 (0.31)10.1 (0.31)9.7 (0.29)
15–6438.8 (0.39)41.3 (0.40)41.9 (0.40)42.9 (0.40)42.6 (0.40)
65–7428 (3.38)30.7 (3.62)31.5 (3.45)32.8 (3.49)32.8 (3.31)
≥7563.5 (12.42)70.6 (13.36)74.2 (12.91)76.1 (12.78)75.8 (12.08)
Table 2. Crude Death Rates and Risk Ratios (RR) of the Extreme Heat Waves between 2013 and 2017 (* including 3 days lag, ** excluding 3 days lag).
Table 2. Crude Death Rates and Risk Ratios (RR) of the Extreme Heat Waves between 2013 and 2017 (* including 3 days lag, ** excluding 3 days lag).
Extreme Heat Waves Start DateEnd Date * Number of Days *Average of Daily Mean Temp (°C) **PopulationNumber of DeathsDeath Rates (x100k)RR (95% CI)p Value
Heat Wave 127 Jul 152 Aug 15727.5714,657,43412031.171.11 (1.04–1.18)0.002
Reference 201329 Jul 1304 Aug 137 14,160,467943
Reference 201428 Jul 1403 Aug 147 14,377,0181057
Reference 201625 Jul 1631 Jul 167 14,804,1161198
Reference 201724 Jul 1730 Jul 177 15,029,2311125
Reference Total 2825.6458,370,83243231.06
Heat Wave 25 Aug 1614 Aug 161026.9714,804,11616581.121.06 (1.00–1.12)0.037
Reference 201302 Aug 1311 Aug 1310 14,160,4671408
Reference 201408 Aug 1417 Aug 1410 14,377,0181546
Reference 201507 Aug 1516 Aug 1510 14,657,4341603
Reference Total 3025.2943,194,91945571.05
Heat Wave 329 Jun 174 Jul 17628.7215,029,23111641.291.21 (1.14–1.30)<0.001
Reference 201327 Jun 1302 Jul 136 14,160,467839
Reference 201426 Jun 1401 Jul 146 14,377,018930
Reference 201502 Jul 1507 Jul 156 14,657,434977
Reference 201630 Jun 1605 Jul 166 14,804,116955
Reference Total 2423.7857,999,03537011.06
Heat Wave 45 Aug 1710 Aug 17627.0915,029,2319261.030.99 (0.92–1.07)0.847
Reference 201303 Aug 1308 Aug 136 14,160,467820
Reference 201402 Aug 1407 Aug 146 14,377,018911
Reference 201508 Aug 1513 Aug 156 14,657,434950
Reference Total 1825.2743,194,91926811.03
Table 3. Sex-specific Death Rates and Risk Ratios of the Extreme Heat Waves between 2013 and 2017.
Table 3. Sex-specific Death Rates and Risk Ratios of the Extreme Heat Waves between 2013 and 2017.
Heat WaveMenWomen
PopulationNumber of DeathsDeath Rate (x100k)RR (95% CI)p ValuePopulationNumber of DeathsDeath Rate (x100k)RR (95% CI)p Value
Heat Wave 17,360,4995721.11 7,296,9356311.24
Reference Total29,290,76023151.130.98 (0.90–1.08)0.71829,080,07220080.991.25 (1.15–1.37)<0.001
Heat Wave 27,424,3908631.16 7,379,7267951.08
Reference Total21,697,37823741.091.06 (0.98–1.15)0.12821,497,54121831.021.06 (0.98–1.15)0.154
Heat Wave 37,529,4915851.29 7,499,7405791.29
Reference Total29,121,76819771.131.14 (1.04–1.26)0.00428,877,26717241.001.29 (1.18–1.42)<0.001
Heat Wave 47,529,4914781.06 7,499,7404481.00
Reference Total21,697,37813961.070.99 (0.89–1.09)0.80121,497,54112851.001.00 (0.90–1.11)0.991
Table 4. Age-specific Death Rates and Risk Ratios of the Extreme Heat Waves between 2013 and 2017.
Table 4. Age-specific Death Rates and Risk Ratios of the Extreme Heat Waves between 2013 and 2017.
Heat Wave0–14≥75
PopulationNumber of DeathsDeath Rate (x100k)RR (95% CI)p ValuePopulationNumber of DeathsDeath Rate (x100k)RR (95% CI)p Value
Heat Wave 13,302,582690.301.05 (0.80–1.36)0.740337,19260025.421.16 (1.06–1.27)0.002
Reference Total13,165,6982630.29 1,345,706206721.94
Heat Wave 23,301,723980.301.08 (0.86–1.36)0.524345,01778722.811.03 (0.95–1.12)0.507
Reference Total9,842,1182710.28 975,254216422.19
Heat Wave 33,324,439490.250.75 (0.55–1.01)0.059362,62764729.741.38 (1.26–1.52)<0.001
Reference Total13,143,8412600.33 1,320,271170221.49
Heat Wave 43,324,439520.260.91 (0.67–1.24)0.556362,62744320.360.93 (0.83–1.03)0.179
Reference Total9,842,1181690.29 975,254128321.93
Table 5. Cause-specific Death Rates and Risk Ratios of the Extreme Heat Waves between 2013 and 2017.
Table 5. Cause-specific Death Rates and Risk Ratios of the Extreme Heat Waves between 2013 and 2017.
Heat WaveCardiovascularRespiratoryOthers
Number of DeathsDeath Rate (x100k)RR (95% CI)p ValueNumber of DeathsDeath Rate (x100k)RR (95% CI)p ValueNumber of DeathsDeath Rate (x100k)RR (95% CI)p Value
Heat Wave 14000.391.17 (1.05–1.31)0.0051720.170.99 (0.84–1.17)0.9186310.611.11 (1.01–1.21)0.025
Reference Total13600.33 6910.17 22720.56
Heat Wave 24920.330.98 (0.88–1.08)0.6492650.181.07 (0.93–1.23)0.3509010.611.11 (1.03–1.20)0.007
Reference Total14700.34 7230.17 23640.55
Heat Wave 34060.451.32 (1.18–1.48)<0.0011450.161.03 (0.86–1.24)0.7186130.681.20 (1.10–1.31)<0.001
Reference Total11880.34 5410.16 19720.57
Heat Wave 43050.341.07 (0.94–1.22)0.3201550.171.04 (0.87–1.25)0.6694660.520.93 (0.84–1.04)0.205
Reference Total8200.32 4280.17 14330.55

Share and Cite

MDPI and ACS Style

Can, G.; Şahin, Ü.; Sayılı, U.; Dubé, M.; Kara, B.; Acar, H.C.; İnan, B.; Aksu Sayman, Ö.; Lebel, G.; Bustinza, R.; et al. Excess Mortality in Istanbul during Extreme Heat Waves between 2013 and 2017. Int. J. Environ. Res. Public Health 2019, 16, 4348. https://doi.org/10.3390/ijerph16224348

AMA Style

Can G, Şahin Ü, Sayılı U, Dubé M, Kara B, Acar HC, İnan B, Aksu Sayman Ö, Lebel G, Bustinza R, et al. Excess Mortality in Istanbul during Extreme Heat Waves between 2013 and 2017. International Journal of Environmental Research and Public Health. 2019; 16(22):4348. https://doi.org/10.3390/ijerph16224348

Chicago/Turabian Style

Can, Günay, Ümit Şahin, Uğurcan Sayılı, Marjolaine Dubé, Beril Kara, Hazal Cansu Acar, Barış İnan, Özden Aksu Sayman, Germain Lebel, Ray Bustinza, and et al. 2019. "Excess Mortality in Istanbul during Extreme Heat Waves between 2013 and 2017" International Journal of Environmental Research and Public Health 16, no. 22: 4348. https://doi.org/10.3390/ijerph16224348

APA Style

Can, G., Şahin, Ü., Sayılı, U., Dubé, M., Kara, B., Acar, H. C., İnan, B., Aksu Sayman, Ö., Lebel, G., Bustinza, R., Küçükali, H., Güven, U., & Gosselin, P. (2019). Excess Mortality in Istanbul during Extreme Heat Waves between 2013 and 2017. International Journal of Environmental Research and Public Health, 16(22), 4348. https://doi.org/10.3390/ijerph16224348

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop