Next Article in Journal
Temporal Dynamics of Nitrous Oxide Emission and Nitrate Leaching in Renovated Grassland with Repeated Application of Manure and/or Chemical Fertilizer
Next Article in Special Issue
The Effect of Cover Crops on Soil Water Balance in Rain-Fed Conditions
Previous Article in Journal
Comparison of Simulated Tropical Cyclone Intensity and Structures Using the WRF with Hydrostatic and Nonhydrostatic Dynamical Cores
Previous Article in Special Issue
Expected Changes of Montenegrin Climate, Impact on the Establishment and Spread of the Asian Tiger Mosquito (Aedes albopictus), and Validation of the Model and Model-Based Field Sampling
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Atmosphere
Volume 9
Issue 12
10.3390/atmos9120484
atmosphere-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Toward a Weather-Based Forecasting System for Fire Blight and Downy Mildew

by
Ana Firanj Sremac
1,*,
Branislava Lalić
1,
Milena Marčić
2 and
Ljiljana Dekić
3
1
Faculty of Agriculture, University of Novi Sad, Trg Dositeja Obradovića Sq. 8, 21000 Novi Sad, Serbia
2
Forecasting and Warning Service of Serbia in plant protection, Temerinska 131, 21000 Novi Sad, Serbia
3
Republic Hydrometeorological Service of Serbia, Kneza Višeslava 66, 11000 Belgrade, Serbia
*
Author to whom correspondence should be addressed.
Atmosphere 2018, 9(12), 484; https://doi.org/10.3390/atmos9120484
Submission received: 13 October 2018 / Revised: 26 November 2018 / Accepted: 5 December 2018 / Published: 7 December 2018
(This article belongs to the Special Issue Weather and Climate Change Challenges in Agricultural and Forest Meteorology)
Browse Figures
Versions Notes

Abstract

:
The aim of this research is to present a weather-based forecasting system for apple fire blight (Erwinia amylovora) and downy mildew of grapevine (Plasmopara viticola) under Serbian agroecological conditions and test its efficacy. The weather-based forecasting system contains Numerical Weather Prediction (NWP) model outputs and a disease occurrence model. The weather forecast used is a product of the high-resolution forecast (HRES) atmospheric model by the European Centre for Medium-Range Weather Forecasts (ECMWF). For disease modelling, we selected a biometeorological system for messages on the occurrence of diseases in fruits and vines (BAHUS) because it contains both diseases with well-known and tested algorithms. Several comparisons were made: (1) forecasted variables for the fifth day are compared against measurements from the agrometeorological network at seven locations for three months (March, April, and May) in the period 2012–2018 to determine forecast efficacy; (2) BAHUS runs driven with observed and forecast meteorology were compared to test the impact of forecasted meteorological data; and (3) BAHUS runs were compared with field disease observations to estimate system efficacy in plant disease forecasts. The BAHUS runs with forecasted and observed meteorology were in good agreement. The results obtained encourage further development, with the goal of fully utilizing this weather-based forecasting system.

1. Introduction

Providing food to the growing population of the world requires agricultural production with minimized environmental, pest, and disease risks. Fire blight (Erwinia amylovora) and downy mildew (Plasmopara viticola) diseases have been the cause of massive yield losses in orchards and vineyards worldwide. Plasmopara viticola (Berk. & M.A. Curtis) Bed. and De Toni, the downy mildew of grapevine, is the obligate biotrophic oomycete that attacks the green parts of the host plant leaves, clusters, and young fruit [1]. It has two developing stages: (1) the primary sexual stage, consisting of oospores, which are a source of the inoculum for primary downy mildew infection [2,3,4]; and (2) the secondary asexual stage, when infection results from sporangia-containing asexually produced zoospores [5,6]. During the winter months, oospores are present in the top soil layers and leaf debris. Increased temperature and rain events in the spring allow their germination in a macrosporangium that releases zoospores responsible for primary infections on grape leaves and clusters [7]. After the primary infection, if the environmental conditions are favorable, then a secondary infection can start.
Erwinia amylovora (Burrill 1882) [8], an epiphytic bacterial pathogen, is the causal agent of fire blight in the Rosaceae family. One of the most disease-sensitive host phenological phases is flowering. During flowering, the cells of Erwinia amylovora multiply on the flower surface and can be washed inside the flower to the hypanthium, where infection occurs [9,10]. From the flower surface, pathogens can be spread by pollinating insects [11], but once inside, pathogens are spread internally within the host plant.
Damage caused by fire blight and downy mildew has been so severe that the chemical industry has been focused on producing highly efficient products for disease control. As environmental awareness is increasing, new systems for optimizing and minimizing chemical control are being developed, mostly through the use of warning systems based on meteorological observations and disease models of varying complexity [12,13,14]. Several models for messages of downy mildew occurrence have been developed based on agrometeorological conditions and the phenology of both the host and the pathogen [15,16,17]. Fire blight models have been developed based on the impact of temperature on bacterial growth [17,18] and downy mildew models on the impact of temperature on incubation period duration [12].
Weather data that are commonly used as inputs for plant disease prediction models include air temperature, precipitation, relative humidity, and duration of leaf wetness [13]. These data are either observed or forecasted using Numerical Weather Prediction (NWP) models. Real-time pests and disease decision-support systems commonly use real-time observations from automatic weather stations (AWSs) [14,15,16,17,18]. Real-time meteorological data and calibrated disease models can provide accurate outputs, but without the possibility of disease forecasting if they are not equipped with complex dynamic models. During the last 30 years, some attempts have been made to introduce weather forecasting in plant disease modelling [19], but the topic was not exploited enough, and systems that utilize real short-term weather forecasts created by operational running NWP models are still very rare [18,20,21].
The development of weather-based warning systems able to predict plant disease appearance is closely related to the performance of high-resolution NWP models. The high resolution of an NWP model enables forecasting of small temporal (1 to 5 days) and spatial (orchard) scale biological processes, such as those related to infection development. Since almost all disease models are diagnostic and driven by meteorological conditions, predictions of disease appearance can be made by using weather forecasts as meteorological data inputs. It is hence possible to predict the time of appearance, the duration of incubation periods, and the risk level depending on the original disease model in use. The benefits of weather-based warning systems are evident at all levels from local (farms) to global (environmental) scales.
The weather-based warning system presented in this paper consists of two models: (a) a high-resolution forecast (HRES) atmospheric model, operationally run by the European Centre for Medium-Range Weather Forecasts (ECMWF) [22]; and (b) a biometeorological system for messages on the occurrence of diseases in fruits and vines (BAHUS) [23,24]. Outputs from the HRES model were used as input data for the BAHUS model. In order to test the operational efficacy of the system, we used the fifth day of the forecast according to the World Meteorological Organisation (WMO) recommendations [25] as an optimal time window for field operations. BAHUS was chosen as a multidisease system.

2. Experiments

2.1. Weather-Based System Design

The system is designed to be prognostic and simple from the start. In order to provide forecasted weather data for input, a high-resolution 10-day forecast (HRES) atmospheric model with a resolution of 9 km is used. The advantage of this system is the 10-day time step and connection to ensemble forecasting by ECMWF [26]. The chosen NWP model, HRES, allows use of weather forecasting with different integration periods (10 days or less). For the purpose of this study, only temperature and precipitation from the 00:00 run every 3 h for the fifth day of the forecast is used. The fifth day is chosen over the first or third day forecast following recommendations of the WMO about the dissemination of model results and time that farmers need to prepare and act on disease appearance [25]. The forecast data are obtained in the form of gridded meteorological data for the grid point closest to the selected geographical location.
BAHUS requires average daily temperature and daily precipitation for downy mildew (BAHUS-P) and daily temperature, maximal daily temperature, and daily precipitation for fire blight forecasting (BAHUS-E). Daily temperature and precipitation are calculated from 8 HRES output values for the fifth day, and maximum temperature is taken as a maximum of 8 temperature values, because disease models require daily values as input.
For the purpose of downy mildew suppression, it is necessary to detect the exact time of the primary infection outbreak. In BAHUS-P, the start of the downy mildew primary infection is described by its preferred environmental and biological conditions, also known as the 3–10 rule. The 3–10 rule is a general spray rule that defines the conditions for primary infection: (1) the air temperature is above 10 °C, (2) more than 10 mm of rain has fallen within 48 h, and (3) the shoot length in the vineyard is at least 10 cm [12]. In the case of Serbian agroecological conditions, the 3–10 rule is modified by increasing the base temperature from 10 to 12 °C. The duration of incubation is determined by 3 methods in BAHUS. In this study, we chose the Milers method, which gave the best results in previous testing [23,24]. In the Milers method, the end of incubation is defined by air temperature and the Milers curve, which gives the accumulation of degree-days (DD). BAHUS-P runs are performed after the dates of phenological conditions are met and observed in the field (10 cm for the shoot length).
BAHUS-E is composed of environmental requirements for apple blossom infection as given by Steiner [27] together with Mills’s degree-day [28] and the MaryblytTM algorithm [24,29,30]. The MaryblytTM algorithm is based on an accumulation of degree-days (DD) and degree-hours (DH), which are defined as the number of degrees over the base temperature during 1 day and 1 h, respectively [28,31]. BAHUS-E runs started on the first day of flowering (Biologische Bundesanstalt, Bundessortenamt and Chemical industry (BBCH) 61, beginning of flowering: approximately 10% of flowers open [32]) and ended 30 days later. The time step of 20 days was selected because flowers fading and majority of petals fallen were not given for all locations, and the apple flower is known to last for 2–3 weeks in Serbian agroecological conditions [33].
The current system setup includes: (a) selected locations, (b) fifth day of the HRES weather forecast (-HRES), (c) observed weather data (-O), and (d) phenological stage of the plant. After the system run, BAHUS-P output was the occurrence of downy mildew primary infection, while BAHUS-E output was the risk of apple blossom fire blight infection, described as none, low, moderate, or high [23,27]. Both model outputs, driven by observed (-O) and forecast (-HRES) weather data, were compared to examine the impact of weather forecasts on disease appearance prediction. Results obtained were compared with field disease observations to estimate system efficacy in fire blight and downy mildew prediction.

2.2. HRES Data

The HRES atmospheric model is a product of the ECMWF that is created twice a day with a resolution of 9 km. The forecast is produced at 3 h intervals for the first 144 h and every 6 h until 240 h, with base times of 00:00 and 12:00 UTC. From the first run at 00:00, we used the fifth day of the forecast [22,26]. The HRES model accuracy tends to change after day 5 (Appendix A). Additionally, by using only one forecasting day, we ensured that it was comparable to the observed meteorology. HRES air temperature and precipitation data are taken as the primary meteorological set for the BAHUS model run. The model numerical point is chosen as the one geographically closest to the location of the AWS in the field. For modelling purposes, we used linear interpolation to obtain the hourly temperature from the 3 h data. The accuracy of the forecast was tested for the most host disease–sensitive times of the year, i.e., March, April, and May, in 2012–2018.
The accuracy of the forecast was evaluated using: (1) coefficient of determination (r2), (2) Root-Mean-Square Error (RMSE) and BIAS, and (3) coefficient of correlation (C); normalized standard deviation (σN) and RMS are presented on a Taylor diagram. The r2 gave the answer to the question: How well did the forecast values correspond to the observed values? The RMSE and BIAS were calculated, since r2 does not take forecast BIAS into account and forecasts with large errors can have very high r2, and to answer the question: What is the average magnitude of the forecast errors? The Taylor diagrams were added for deeper insight into the specific location forecast on the monthly level.

2.3. Agrometeorological Observations

The agrometeorological network is a part of the Forecasting and Reporting Service of Plant Protection (Prognozno-Izveštajne Službe (PIS) zaštite bilja) in Serbia. The network consists of 166 AWSs, 69 in the Vojvodina region and 97 in central Serbia, placed in the field to measure micrometeorological conditions of the plant canopy. The standard setup of the PIS AWS consists of the following sensors: air temperature, relative humidity, soil temperature, precipitation, and leaf wetness. Network AWSs are regularly maintained according to the manufacturer’s (Pesl or Campbell Scientific/Davis) instructions. Micrometeorological data from the PIS network are freely available online. For the purpose of this study, temperature and precipitation were obtained for the locations where disease appearance was observed during the 2012–2018 period. The location details are given in Table 1. Observed meteorological data were used to analyze the efficacy of the NWP in predicting meteorological conditions for plant disease appearance in the Vojvodina region.
The BC AWS was shifted to an apple orchard close to the Nera River in March 2015. The new location was 4 km from the previous location.

2.4. Disease Observation

Within the PIS network, monitoring of disease and pest appearance and development was performed daily or twice a week, depending on pathogen characteristics and favorable weather conditions for harmful organism appearance. Observations of fire blight and downy mildew were conducted in all orchards and vineyards of the PIS network, with a focus on early primary infections. Once disease symptoms were observed and meteorological conditions were suitable for disease development, adequate treatments were proposed. The presence of fire blight was confirmed at 6 locations in 3 years where observations were made.
For the comparison study, the following data were selected from the PIS biological observation database: (a) dates of observed appearance and infection potential for both diseases (Table 2 and Table 3); (b) dates when phenological conditions of host plants, for both model runs, were satisfied (Table 2 and Table 3); and (c) dates of primary (or possibly first secondary) infections of grape downy mildew (Table 2). Because the time between primary and secondary infections of downy mildew can be very short and symptoms of primary infection are difficult to observe due to a commonly low level of infection, it was hypothesized that the observed symptoms corresponded to the first secondary infection.
The monitoring of flower fire blight is very difficult because it resembles physiological drying of flowers, and the first reliable symptoms appear 1–2 weeks after the petals fall or on the tips of shoots by forming a “shepherd’s crook.” Fire blight observation data are given in Table 3.

3. Results

3.1. HRES Outputs

The efficacy of the model depends on the quality of the input data. Therefore, a comparison of the model’s temperature and precipitation forecast for the fifth day against the measured data at seven locations for March, April, and May (MAM) during the 2012–2018 period was performed. A qualitative analysis of the obtained results is based on the coefficient of determination, r2 (Figure 1) (Tables S1.1 and S1.2 in Supplementary Material 1), RMSE, and BIAS (Figure 2 and Supplementary Table S1.3) are calculated using observed and forecasted daily temperatures and precipitation. Additionally, to make site- and season-specific analyses of temperature and precipitation forecasting, the Taylor diagrams are used to graphically summarize how closely a variable (or a set of variables) matches observations (Figure 3). The similarity is quantified in terms of their correlation (C), centered RMS difference, and the amplitude of their variations (represented by normalized standard deviation, σN) [34].
In all locations, the r2 for daily average temperature was above 0.7, while in 39% of cases it was above 0.9. The lowest values of r2 were obtained for the BC and CE locations (0.7 and 0.81, respectively). Overestimation of forecasted temperature at the DI location (Figure 1) for only six days (day of year (DOY) 135–141) significantly reduced overall statistics for 2013 at this location (r2 = 0.81). Values of RMSE and BIAS calculated without those six days (DOY 135–141) were significantly lower (4.34 °C and −0.31 °C, respectively).
The calculated RMSE and BIAS values are quite uniform during the MAM period at all selected locations except DI in May 2013 and BC in 2016 and 2017 (Figure 2). RMSE variation among locations is particularly pronounced in April, while the lowest differences can be noticed in May.
High correlation of σN close to 1 and RMS difference around 0.5 °C or less for daily temperature were observed at all locations during the examined period (Figure 3). The exception was the BC location with a slightly lower correlation coefficient in March and April 2016 and 2017 and May 2017 and higher RMS around 0.7 °C (Figure 3). Seasonality was noticed with highly clustered data, with high correlation and σN in March and April. A less clustered picture in May with high σN but a much wider spectrum of correlation values and RMS higher than 0.7 °C may have been the result of spring 2015 meteorological conditions at almost all locations, which are not presented in HRES (Supplementary Material 2).
Deviations obtained for the DI location in May 2013 (Figure 1) correspond with very low temperatures measured in the orchard from DOY 133–149. High deviations in the BC location were partly related to a shift of the AWS to a nearby location with very specific local microclimate strongly affected by the Nera River and mountains on its right bank in Romania.
Noticed seasonality and variation in the calculated parameters from March to May might be related to the impact of plant development on orchard micrometeorological conditions and air temperature variation in the height of fruit tree crowns, where air temperature sensors are typically installed. Intensive plant development can cause significant variation in canopy air temperature with respect to outside of the plant canopy, while in the case of the formed canopy, this variation is negligible. Depending on the variety, intensive apple leaf development took place at the end of March and during April, while in May, the crown was commonly closed, which explains the increase in RMS difference.
The overall analysis of measured and forecasted daily precipitation in Vojvodina indicates that typically, the HRES model underestimates the amount of precipitation (except for BC in 2013 and BV in 2016) (Figure 4). Coefficient of determination, r2, of daily precipitation ranged from 0.02 (BV in 2017) to 0.42 (BC in 2013) (Table S1.2, Supplementary Material 1). RMSE and BIAS of the precipitation increase from April (average: RMSE = 4.57 mm, BIAS = −0.23 mm) to March (average: RMSE = 5.82 mm, BIAS = −0.56 mm) to May (average: RMSE = 7.98 mm, BIAS = 0.19) (Figure 5). The main cause of the observed deviation patterns was the frequency of convective precipitation, which increased near the late spring and beginning of summer, i.e., in the warmer part of the year. Differences among locations, more pronounced in May than in March, were also related to convective precipitation and its highly diverse spatial distribution [35]. Extreme values of RMSE in 2014 at numerous locations were the result of excessive precipitation (50% of the meteorological stations registered exceeded thresholds for extreme precipitation) in that year in Serbia [36] (Supplementary Material 3).
The Taylor diagrams for daily precipitation indicate significant discrepancies among seasons and locations (Figure 6). Data were not clustered, and it was difficult to determine at which location the precipitation was better forecasted, because the situation changed from month to month and year to year. The normalized standard deviation was smaller or greater than 1, C was between 0.1 and 0.9, and the RMS was within 2 mm for the whole period. It is interesting to observe that in May 2014, when the amount of precipitation in Vojvodina was well above normal, the C was between 0.75 and 0.9 for all locations and RMS was below 0.7 mm. High C, low RMS, and low σN indicate that the precipitation variations were not of the right amplitude. In this case, the timing was right but the amount of precipitation was not.

3.2. BAHUS Simulation Results

3.2.1. Downy Mildew, BAHUS-P

BAHUS-P was run using observed (-O) and forecasted (-HRES) temperature and precipitation for the location VV. The calculated and observed dates of infection are presented in Figure 7. On the incubation axes, 0 means the start of the incubation period and 3 is the end of the period. Model runs were conducted for part of the year (after the dates given in Table 2) when phenological conditions were met (10 cm shoot length). The obtained results show that BAHUS-P-HRES offered incubation period simulations that corresponded to the results obtained from BAHUS-P-O and orchard observations. The most important result was that whenever the disease was observed, BAHUS-P-HRES indicated it as well, with differences of a few days in some years. The ends of the first BAHUS-P-HRES primary infection incubations were as follows (BAHUS-P-HRES/observed): 2012, 150/153; 2013, 149/153; 2014, 128/139; 2015, 145/155; 2016, 147/141; 2017, 134/157; 2018, 136/172. Early triggering of the model with HRES data in 2012, 2014, and 2017 occurred because of the temperature difference. This difference was quantified with positive BIAS in May (Figure 2). The early triggering in 2018 occurred because of poor forecasting in terms of precipitation amount, but also the first symptoms were observed very late in the year, on day 172. This late observation also indicates that it was not primary but first secondary infection that was observed.

3.2.2. Fire Blight, BAHUS-E

BAHUS-E was tested in the six orchards for six years when disease was observed. The observations of fire blight are used only for the location selection, not for testing, because the observations were made after flowering, when bacteria have infected other plant organs.
The results from the model run with observed (-O) and forecasted (-HRES) showed good agreement between no risk = 0, low risk = 1, medium risk = 2, and high risk = 3 conditions (Figure 8a). At location GO in 2014, the air temperature was very low after flowering, and both runs showed an increased risk after 30 days. In 2015, at location BC, there was good risk response with BAHUS-E-O, but late response from BAHUS-E-HRES due to the lower forecasted temperatures at the beginning of May (Supplementary Material 4). In 2018, at locations CE, VR, and BV, BAHUS-E-O and BAHUS-E-HRES gave very similar results, showing the significance of the predicted temperatures over precipitation (Figure 8b).

4. Discussion

The described weather-based warning system design includes use of ECMWF HRES forecasts as meteorological input data for the BAHUS disease model. Particular procedures and models were selected to test the efficacy of HRES and BAHUS in forecasting the appearance of primary infection of grapevine downy mildew and risk of fire blight on apple flowers. BAHUS was selected because it includes previously developed and tested algorithms for both diseases. The comparison between results from different downy mildew and fire blight models was difficult to perform, since models are calibrated for specific locations or regions and driven by different meteorological data. Several studies have argued that a slight disease overestimation, as presented in this weather-based warning system, is preferable, since this could later induce a reduction in the number of treatments [37,38,39,40]. The importance of the meteorological data quality changes based on the model type and sensitivity. Most of the disease prediction models are temperature-sensitive and have thresholds in descriptions of humidity or precipitation [41]. Therefore, errors in input air temperature produce bigger differences in the model outputs [42]. The same study showed that BIAS of +/−1 °C in the air temperature data has a smaller impact than observed variance around the average BIAS [42].
The results from previous research on the use of NWP in disease appearance forecasting for apple scab led to the same results as those presented here, i.e., NWP output accuracy was not strictly correlated with good disease prediction [43,44].
A comparison between OBS datasets and fifth-day HRES outputs was made in order to assess the quality of the forecasted air temperature and precipitation. The procedure for evaluating forecast quality was used according to several studies and recommendations [45,46,47]. The RMSE and BIAS of the HRES air temperature at all locations and for all years was in the same limits as the results presented in the ECMWF technical report [48] and calculated for Belgrade 2017 (Appendix A) for the fifth day. A change of lead time can improve the quality of the forecast but reduce the time window for application of chemical treatment. The error analysis of air temperature showed larger RMSE and BIAS in April, when vegetation is rapidly changing. The overestimation of the HRES temperatures was not confirmed in other studies, which used much shorter lead times [49].
The results from the comparison of the sums of 24 h precipitation were not sufficient. The results from the comparison between fifth-day HRES and OBS precipitation gave very low r2, while RMSE (M: 5.8 mm, A: 4.7 mm, M: 7.2 mm) and BIAS were within the boundaries confirmed by other studies [50,51,52]. RMSE and bias were higher in May than in other months. The reason for this difference was rising frequency of convective precipitation in the warmer part of the year. In [50] the RMSEs calculated for different lead times of forecasted precipitation showed a rise above 10 mm for both winter and summer after day 5. From the NWP model point of view, it is difficult to forecast in locations close to but not on mountains, particularly when grid elements consist of both flat terrain and mountains. In that case, forecast accuracy highly depends on parameterization of subgrid-scale processes. This was the case for the BC location; all other AWS locations are on flat, relatively uniform terrain.
The errors in HRES precipitation were not evident in BAHUS-P or BAHUS-E runs, because both models were triggered by rain events in two days, and in that case, it is important that the sum of precipitation is higher than the model limit. BAHUS-P and BAHUS-E runs with observed and forecasted meteorological data showed better agreement at some locations. The results also confirm that additional downscaling of the HRES model outputs are not necessary for this proposed system. Nevertheless, improvement of the weather input data in the BAHUS model can only increase the system’s prediction capability [49]. The presented results clearly justify the use of the selected models in the proposed system. Integrated disease–weather models have been successful in practical use [39], and making them even more flexible and open will only increase their efficacy and forecasting capacity.
Following the NWP modelling strategy in relation to ensemble forecasting, adopted crop [53,54] and disease models can also drive weather-based disease warning systems to integrate ensemble forecasting. In that case, several disease models will be run with the same initial conditions and weather conditions. Systems such as BAHUS, designed to allow users the opportunity to choose among algorithms, can be the starting point for ensemble disease forecasting.

5. Conclusions

The results show the high capacity of the presented weather-based warning system to forecast disease development with selected BAHUS algorithms and HRES weather forecasting. A comparison between the average daily temperatures and the sum of precipitation calculated from observed hourly values and values given by HRES every 3 h showed the following:
-
The results for March–May in 2012–2018 at seven locations showed the high quality of the predicted temperature for the fifth day.
-
The temperature-driven algorithms in BAHUS performed very well; the predicted amount of daily precipitation had errors, since it was the fifth day of the forecast and the uncertainties associated with precipitation were high.
-
The large errors in the daily sum of precipitation did not affect BAHUS results significantly, since both disease algorithms require the sum of precipitation over the threshold value for two days.
With these results, the system can also be used to assess meteorological conditions favorable for disease appearance at locations where meteorological observations are not available. The advantage of this system is its flexibility in producing different time-scale forecasting from a 10-day HRES forecast. The disadvantage is similar to other systems; for example, the need for observed phenological data such as the date of apple flowering.
The results show great potential for the presented weather-based system. The BAHUS-P driven by HRES triggered downy mildew incubation in the period before first observed symptoms in all seven years. BAHUS-E was more sensitive to the meteorological data, since the model itself is more complex. The results from BAHUS-E were still very close to the forecasts made with observed daily air temperature and daily sum of precipitation.
The next step is to couple phenological models based on temperature sums for grapevine and apple development with the presented weather-based system to create a fully operational plant protection warning system for fire blight and downy mildew. The application of such a coupled system would improve the production of apples and grapes by lowering the amount of chemical treatments and acting at the right time. Based on the presented results, we will consider the operational aspects of the proposed system.

Supplementary Materials

The following are available online at https://www.mdpi.com/2073-4433/9/12/484/s1. 1. Table S1.1 Temperature values of r2 received from linear fits; Table S1.2 Precipitation values of r2 received from linear fits; Table S1.3 Root-mean-square error (T–RMSE, P–RMSE) and BIAS (T–BIAS, P–BIAS) for temperature and precipitation; 2. Figure S2.1 Observed daily air temperature in May during the period 2012–2018; 3. Table S3.1 Monthly sum of precipitation at the selected AWSs in the period 2012–2018.; 4. Figure S4.1 Observed and predicted temperatures in 2015 at the different locations, with a focus on BC.

Author Contributions

Conceptualization, methodology, and software, A.F.S. and B.L.; validation and formal analysis, A.F.S.; resources data curation, M.M. and Lj.D.; writing—original draft preparation, A.F.S.; writing—review and editing, B.L., M.M., and Lj.D.; visualization, A.F.S.

Funding

This paper is supported by the Serbia for Excell project, which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement no. 691998. The research work described in this paper was realized as a part of the project “Studying climate change and its influence on the environment: impacts, adaptation and mitigation” (III43007).

Acknowledgments

We acknowledge the Forecasting and Reporting Service of Plant Protection (Prognozno-Izveštajne Službe zaštite bilja) in Serbia for collecting the data in the field and providing it for the research purposes. We acknowledge Ana Mihalović from Republic Hydrometeorological Service of Serbia for providing the evidence of HRES model accuracy in Serbia, placed in Appendix A.

Conflicts of Interest

The authors declare no conflicts of interest

Appendix A

The quality of the HRES forecast based on lead time is verified by the Republic Hydrometeorological Service of Serbia for Belgrade station during its regular operational practice. The change is presented by calculating root-mean-square error (RMSE), mean average error (MAE), and mean error for 2017 for 2 m air temperature (Figure A1) and maximal air temperature (Figure A2). Both parameters show significant quality change after day 5 (D + 4 in the figures).
Atmosphere 09 00484 g0a1 550
Figure A1. Errors of 2 m HRES air temperature depending on lead time.
Figure A1. Errors of 2 m HRES air temperature depending on lead time.
Atmosphere 09 00484 g0a1
Atmosphere 09 00484 g0a2 550
Figure A2. Errors of 2 m HRES maximal air temperature depending on lead time.
Figure A2. Errors of 2 m HRES maximal air temperature depending on lead time.
Atmosphere 09 00484 g0a2

References

  1. Burruano, S. The life-cycle of Plasmopara viticola, cause of downy mildew of vine. Mycologist 2000, 14, 179–182. [Google Scholar] [CrossRef]
  2. Lafon, R.; Clerjeau, M. Downy mildew. In Compendium of Grape Diseases; Pearson, R.C., Goheen, A.C., Eds.; APS Press: St. Paul, MN, USA, 1988; pp. 11–13. [Google Scholar]
  3. Wong, F.P.; Burr, H.N.; Wilcox, W.F. Heterotallism in Plasmopara viticola. Plant Pathol. 2001, 50, 427–432. [Google Scholar] [CrossRef]
  4. Galbiati, C.; Longhin, G. Indagini sulla formazione e sulla germinazione delle oospore di Plasmopara viticola. Rivista di Patologia Vegetale 1984, 20, 66–80. [Google Scholar]
  5. Blaeser, M. Untersuchungen zur Epidemiologie des falschen Mehltaus an Weinrebe, Plasmopara viticola (Berk. et de Toni). Ph.D. Thesis, Universität Bonn Germany, Bonn, Germany, 1978; p. 127. [Google Scholar]
  6. Kennelly, M.M.; Gadoury, D.M.; Wilcox, W.F.; Magarey, P.A.; Seem, R.C. Primary infection, lesion productivity, and survival of sporangia in the grapevine downy mildew pathogen Plasmopara viticola. Phytopathology 2007, 97, 512–522. [Google Scholar] [CrossRef] [PubMed]
  7. Agrios, G.N. Plant Pathology; Academic Press: San Diego, CA, USA, 1988; p. 845. [Google Scholar]
  8. Winslow, C.-E.A.; Broadhurst, J.; Buchanan, R.E.; Krumwiede, C., Jr.; Rogers, L.A.; Smith, G.H. The families and genera of the bacteria, final report of the committee of the society of american bacteriologists on characterization and classification of bacterial types. J. Bacteriol. 1920, 5, 191–229. [Google Scholar] [PubMed]
  9. Thomson, S.V. The role of the stigma in fire blight infections. Phytopathology 1986, 76, 476–482. [Google Scholar] [CrossRef]
  10. Wilson, M.; Sigee, D.C.; Epton, H.A.S. Erwinia amylovora infection of hawthorn blossom: III. The nectary. J. Phytopathol. 1990, 128, 62–74. [Google Scholar] [CrossRef]
  11. De Wael, L.; de Greef, M.; van Laere, O. The honeybee as a possible vector of Erwinia amylovora (Burr.) Winslow et al. Acta Hortic. 1990, 273, 107–113. [Google Scholar] [CrossRef]
  12. Gessler, C.; Pertot, I.; Perazzolli, M. Plasmopara viticola: A review of knowledge on downy mildew of grapevine and effective disease management. Phytopathol. Mediterr. 2011, 50, 3–44. [Google Scholar] [CrossRef]
  13. Gleason, M.L.; Duttweiler, K.B.; Batzer, J.C.; Taylor, S.E.; Sentelhas, P.C.; Monteiro, J.E.B.A.; Gillespie, T.J. Obtaining weather data for input to crop disease-warning systems: Leaf wetness duration as a case study. Sci. Agric. 2008, 65, 76–87. [Google Scholar] [CrossRef]
  14. Melo Reis, E.M.; Sônego, O.R.; de Sales Mendes, C. Application and validation of a warning system for grapevine downy mildew control using fungicides. Summa Phytopathol. 2013, 39, 10–15. [Google Scholar] [CrossRef]
  15. Damos, P. Modular structure of web-based decision support systems for integrated pest management. A review. Agron. Sustain. Dev. 2015, 35, 1347–1372. [Google Scholar] [CrossRef] [Green Version]
  16. Pavan, W.; Fraisse, C.W.; Peres, N.A. Development of a web-based disease forecasting system for strawberries. Comput. Electron. Agric. 2011, 75, 169–175. [Google Scholar] [CrossRef]
  17. Caffi, T.; Rossi, V.; Carisse, O. Evaluation of a Dynamic Model for Primary Infections Caused by Plasmopara viticola on Grapevine in Quebec. Plant Health Prog. 2011, 12, 22. [Google Scholar] [CrossRef]
  18. Caffi, T.; Rossi, V.; Bugiani, R.; Spanna, F.; Flamini, L.; Cossu, A.; Nigro, C. A model predicting primary infections of Plasmopara viticola in different grapevine-growing areas of Italy. J. Plant Pathol. 2009, 91, 535–548. [Google Scholar] [CrossRef]
  19. Royer, M.H.; Russo, J.M.; Kelley, J.G.W. Plant Disease Prediction Using a Mesoscale Weather Forecasting Technique. Plant Dis. 1989, 73, 618–624. [Google Scholar] [CrossRef]
  20. Fernandes, J.M.C.F.; Pavan, W.; Sanhueza, R.M. SISALERT—A generic web-based plant disease forecasting system. In Proceedings of the International Conference on Information and Communication Technologies for Sustainable Agri-Production and Environment (HAICTA 2011), Skiathos, Greece, 8–11 September 2011. [Google Scholar]
  21. Johnson, K.B.; Stockwell, V.O.; Sawyer, T.L. Adaptation of Fire Blight Forecasting to Optimize the Use of Biological Controls. Plant Dis. 2004, 88, 41–48. [Google Scholar] [CrossRef]
  22. Haiden, T.; Janousek, M.; Bauer, P.; Bidlot, J.; Dahoui, M.; Ferranti, L.; Prates, F.; Richardson, D.S.; Vitart, F. Evaluation of ECMWF Forecasts, Including 2014–2015 Upgrades; ECMWF Technical Memoranda 765; ECMWF: Reading, UK, 2015; p. 53. [Google Scholar]
  23. Mihailovic, D.T.; Koci, I.; Lalic, B.; Arsenic, I.; Radlovic, D.; Balaz, J. The main features of BAHUS–biometeorological system for messages on the occurrence of diseases in fruits and vines. Environ. Model. Softw. 2001, 16, 691–696. [Google Scholar] [CrossRef]
  24. Lalić, B.; Mihailović, D.T.; Radovanović, S.; Balaž, J.; Ćirišan, A. Input data representativeness problem in plant disease forecasting models. IDŐJÁRÁS 2007, 1113, 199–208. [Google Scholar]
  25. World Meteorological Organization. Guide to Agricultural Meteorological Practices; WMO-No. 134; World Meteorological Organization: Geneva, Switzerland, 2010; p. 799. ISBN 978-92-63-10134-1. [Google Scholar]
  26. Haiden, T.; Janousek, M.; Bidlot, J.; Buizza, R.; Ferranti, L.; Prates, F.; Vitart, F. Evaluation of ECMWF Forecasts, Including the 2018 Upgrade; Series: ECMWF Technical Memoranda 831; ECMWF: Reading, UK, 2018; p. 54. [Google Scholar]
  27. Steiner, P.W. Predicting Apple Blossom Infections by Erwinia amylovora using the Maryblyt Model. Acta Hortic. 1990, 273, 139–148. [Google Scholar] [CrossRef]
  28. Mills, W.D. Fire blight development on apple in western New York. Plant Dis. Rep. 1955, 39, 206–207. [Google Scholar]
  29. Billing, E. Fire Blight in Kent, England in Relation to Weather (1955–1976). Ann. Appl. Biol. 1980, 95, 341–464. [Google Scholar] [CrossRef]
  30. Billing, E. BIS95, an Improved Approach to Fire Blight Risk Assessment. Acta Hortic. 1996, 411, 121–126. [Google Scholar] [CrossRef]
  31. McMaster, G. Growing degree-days: one equation, two interpretations. Agric. Forest Meteorol. 1997, 87, 291–300. [Google Scholar] [CrossRef] [Green Version]
  32. Meier, U. (Ed.) BBCH Monograph. Growth Stages of Mono-and Dicotyledonous Plants; Federal Biological Research Centre for Agriculture and Forestry: Rome, Italy, 2001; p. 158. [Google Scholar]
  33. Drkenda, P.; Musić, O.; Marić, S.; Jevremović, D.; Radičević, S.; Hudina, M.; Hodžić, S.; Kunz, A.; Blanke, M.M. Comparison of Climate Change Effects on Pome and Stone Fruit Phenology Between Balkan Countries and Bonn/Germany. Erwerbs-Obstbau 2018, 60, 295–304. [Google Scholar] [CrossRef]
  34. Taylor, K.E. Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res. 2001, 106, 7183–7192. [Google Scholar] [CrossRef] [Green Version]
  35. Ćurić, M.; Janc, D. Analysis of predicted and observed accumulated convective precipitation in the area with frequent split storms. Hydrol. Earth Syst. Sci. 2001, 15, 3651–3658. [Google Scholar] [CrossRef]
  36. Anđelković, G.; Jovanović, S.; Manojlović, S.; Samardžić, I.; Živković, L.; Šabić, D.; Gatarić, D.; Džinović, M. Extreme precipitation events in Serbia: Defining the threshold criteria for emergency preparedness. Atmosphere 2018, 9, 188. [Google Scholar] [CrossRef]
  37. Dalla Marta, A.; Magarey, R.D.; Orlandini, S. Modelling leaf wetness duration and downy mildew simulation on grapevine in Italy. Agric. For. Meteorol. 2005, 132, 84–95. [Google Scholar] [CrossRef]
  38. Blaise, P.; Gessler, C. Development of forecast model og grape downy mildew on a microcomputer. Acta Hortic. 1990, 63–70. [Google Scholar] [CrossRef]
  39. Caffi, T.; Rossi, V.; Bugiani, R. Evaluation of a warning system for controlling primary infections of grapevine downy mildew. Plant Dis. 2010, 94, 709–716. [Google Scholar] [CrossRef]
  40. Walker, S.; Haasbroek, P.D. Use of mathematical model with hourly weather data for early warning of downy mildew in vineyards. Presented at the Farming Systems Design 2007–International Symposium on Methodologies for Integrated Analysis of Farm Production Systems, Catania, Italy, 10–12 September 2007. [Google Scholar]
  41. Bourke, P.M.A. Use of Weather Information in the Prediction of Plant Disease Epiphytotics. Annu. Rev. Phytopathol. 1970, 8, 345–370. [Google Scholar] [CrossRef]
  42. Pfender, W.F.; Gent, D.H.; Mahaffee, W.F.; Coop, L.B.; Fox, A.D. Decision Aids for Multiple-Decision Disease Management as Affected by Weather Input Errors. Phytopathology 2011, 101, 644–653. [Google Scholar] [CrossRef] [PubMed]
  43. Lalic, B.; Francia, M.; Eitzinger, J.; Podraščanin, Z.; Arsenić, I. Effectiveness of short-term numerical weather prediction in predicting growing degree days and meteorological conditions for apple scab appearance. Meteorol. Appl. 2016, 23, 50–56. [Google Scholar] [CrossRef]
  44. Firanj Sremac, A.; Lalic, B.; Jankovic, D. The WRF-ARW application in predicting meteorological conditions for Downy mildew (Plasmopara viticola) appearance of wine grape. In Proceedings of the 16th European Meteorological Society Annual Meeting (EMS), Trieste, Italy, 12–16 September 2016. [Google Scholar] [CrossRef]
  45. Koh, T.-Y.; Wang, S.; Bhatt, B.C. A diagnostic suite to assess NWP performance. J. Gephys. Res. 2012, 117, D13109. [Google Scholar] [CrossRef]
  46. Mailier, P.J.; Jolliffe, I.T.; Stephenson, D.B. Quality of Weather Forecasts, Review and Recommendations; Royal Meteorological Society: Reading, UK, 2006; p. 89. [Google Scholar]
  47. Tri lestari, J.; Wandala, A. A study comparison of two system model performance in estimated lifted index over Indonesia. IOP Conf. Ser. J. Phys. Conf. Ser. 2018, 1025, 012113. [Google Scholar] [CrossRef]
  48. Richardson, D.S.; Hewson, T. Use and Verification of ECMWF Products in Member and Co-Operating States (2017); ECMWF Technical Memoranda, No. 818; ECMWF: Reading, UK, 2018. [Google Scholar]
  49. Lian, J.; Wu, L.; Bréon, F.-M.; Broquet, G.; Vautard, R.; Zaccheo, T.S.; Dobler, J.; Ciais, P. Evaluation of the WRF-UCM mesoscale model and ECMWF global operational forecasts over the Paris region in the prospect of tracer atmospheric transport modeling. Elem. Sci. Anthr. 2018, 6, 64. [Google Scholar] [CrossRef]
  50. Mullen, S.L.; Buizza, R. Quantitative Precipitation Forecasts over the United States by the ECMWF Ensemble Prediction System. Mon. Weather Rev. 2001, 129, 638–663. [Google Scholar] [CrossRef]
  51. Ebert, E.E.; Janowiak, J.E.; Kidd, C. Comparison of Near-Real-Time Precipitation Estimates from Satellite Observations and Numerical Models. Bull. Am. Meteorol. Soc. 2007, 88, 47–64. [Google Scholar] [CrossRef] [Green Version]
  52. Gascón, E.; Hewson, T.; Haiden, T. Improving Predictions of Precipitation Type at the Surface: Description and Verification of Two New Products from the ECMWF Ensemble. Weather Forecast. 2018, 33, 89–108. [Google Scholar] [CrossRef]
  53. Lalić, B.; Firanj Sremac, A.; Dekić, L.; Eitzinger, J.; Perišić, D. Seasonal forecasting of green water components and crop yields of winter wheat in Serbia and Austria. J. Agric. Sci. 2018, 156, 645–657. [Google Scholar] [CrossRef] [PubMed]
  54. Lalić, B.; Firanj Sremac, A.; Eitzinger, J.; Stričević, R.; Thaler, S.; Maksimović, I.; Daničić, M.; Perišić, D.; Dekić, L. Seasonal forecasting of green water components and crop yield of summer crops in Serbia and Austria. J. Agric. Sci. 2018, 156, 658–672. [Google Scholar] [CrossRef] [PubMed]
Atmosphere 09 00484 g001 550
Figure 1. Comparison between observed (TOBS) and predicted daily air temperatures in °C by high-resolution atmospheric forecast model (HRES) (THRES) for selected locations during March, April, and May (MAM) 2012–2018.
Figure 1. Comparison between observed (TOBS) and predicted daily air temperatures in °C by high-resolution atmospheric forecast model (HRES) (THRES) for selected locations during March, April, and May (MAM) 2012–2018.
Atmosphere 09 00484 g001
Atmosphere 09 00484 g002 550
Figure 2. RMSE and BIAS of daily air temperature for selected locations during MAM 2012–2018.
Figure 2. RMSE and BIAS of daily air temperature for selected locations during MAM 2012–2018.
Atmosphere 09 00484 g002
Atmosphere 09 00484 g003a 550Atmosphere 09 00484 g003b 550
Figure 3. Taylor diagram for daily temperature in (a) March, (b) April, and (c) May for selected locations during 2012–2018.
Figure 3. Taylor diagram for daily temperature in (a) March, (b) April, and (c) May for selected locations during 2012–2018.
Atmosphere 09 00484 g003aAtmosphere 09 00484 g003b
Atmosphere 09 00484 g004 550
Figure 4. Comparison between observed (POBS) daily precipitation and that forecasted by the HRES model (PHRES) in mm at selected locations during MAM 2012–2018.
Figure 4. Comparison between observed (POBS) daily precipitation and that forecasted by the HRES model (PHRES) in mm at selected locations during MAM 2012–2018.
Atmosphere 09 00484 g004
Atmosphere 09 00484 g005 550
Figure 5. Daily precipitation RMSE (mm) and BIAS (mm) for selected locations during MAM 2012–2018.
Figure 5. Daily precipitation RMSE (mm) and BIAS (mm) for selected locations during MAM 2012–2018.
Atmosphere 09 00484 g005
Atmosphere 09 00484 g006a 550Atmosphere 09 00484 g006b 550
Figure 6. Taylor diagram for daily sum of precipitation for (a) March, (b) April, and (c) May at selected locations during 2012–2018.
Figure 6. Taylor diagram for daily sum of precipitation for (a) March, (b) April, and (c) May at selected locations during 2012–2018.
Atmosphere 09 00484 g006aAtmosphere 09 00484 g006b
Atmosphere 09 00484 g007 550
Figure 7. BAHUS-P runs with (a) observed (OBS; o) and forecasted meteorology (HRES; x) on days when phenological conditions for disease development were met for primary infection (10 cm shoots; orange triangles) and disease was observed (green squares); (b) observed and predicted temperature (TOBS, THRES) and precipitation (POBS, PHRES) in the same period at VV location.
Figure 7. BAHUS-P runs with (a) observed (OBS; o) and forecasted meteorology (HRES; x) on days when phenological conditions for disease development were met for primary infection (10 cm shoots; orange triangles) and disease was observed (green squares); (b) observed and predicted temperature (TOBS, THRES) and precipitation (POBS, PHRES) in the same period at VV location.
Atmosphere 09 00484 g007
Atmosphere 09 00484 g008 550
Figure 8. BAHUS-E runs with observed (OBS; o) and forecasted (HRES; x) temperature and precipitation at the time of apple flowering (start of flowering is shown with orange vertical lines and symbols) for all locations with (a) no risk = 0, low risk = 1, medium risk = 2, and high risk = 3; and (b) observed and forecasted values of average daily temperature (Ta), maximal temperature (Tmax), and precipitation (P) for the selected time period.
Figure 8. BAHUS-E runs with observed (OBS; o) and forecasted (HRES; x) temperature and precipitation at the time of apple flowering (start of flowering is shown with orange vertical lines and symbols) for all locations with (a) no risk = 0, low risk = 1, medium risk = 2, and high risk = 3; and (b) observed and forecasted values of average daily temperature (Ta), maximal temperature (Tmax), and precipitation (P) for the selected time period.
Atmosphere 09 00484 g008
Table
Table 1. Selected automatic weather stations in the PIS agrometeorological network.
Table 1. Selected automatic weather stations in the PIS agrometeorological network.
LocationLocation IndexLatitudeLongitudeAltitude (m)Plant StandWorking Period
VršacVV45.1052021.31680128vineyard2012–2018
Bački VinogradiBV46.1306019.8603690orchard2012–2018
ČerevićCE45.2141919.6556466orchard2012–2018
GospođinciGO45.4280519.99114–orchard2013–2018
Bela CrkvaBC(Until 23.3.2015
44.914732)
44.90808		(Until 23.3.2015
21.403472)
21.41201		82orchard2013–2018
Veliki RadinciVR45.0291719.6522266orchard2013–2018
DivošDI45.0984219.47272125orchard2013–2018
Table
Table 2. Date when phenological conditions are satisfied (10 cm shoot length in the vineyard) and observed symptoms of downy mildew.
Table 2. Date when phenological conditions are satisfied (10 cm shoot length in the vineyard) and observed symptoms of downy mildew.
LocationVariety (Treated, yes/no)Phenological Conditions SatisfiedObserved Symptoms
VVŠasla (no)16.05.201201.06.2012
VVBurgundac (yes)23.05.201305.06.2013
VVŠasla (no)20.04.201419.05.2014
VVŠasla (no)02.05.201504.06.2015
VVŠasla (no)02.05.201620.05.2016
VVŠasla (no)05.05.201706.06.2017
VVŽupljanka (no)18.04.201821.06.2018
Table
Table 3. Dates of apple flowering and observations of first symptoms of fire blight. BBCH, Biologische Bundesanstalt, Bundessortenamt and Chemical.
Table 3. Dates of apple flowering and observations of first symptoms of fire blight. BBCH, Biologische Bundesanstalt, Bundessortenamt and Chemical.
LocationFlowering Date (BBCH 61, 10% of Flowers Open)Symptoms Observed
BV19.04.201829.05.2018
CE13.04.201820.05.2018
VR16.04.201810.05.2018
BC16.04.201526.05.2015
DI23.04.201505.06.2015
GO02.04.201407.05.2014

Share and Cite

MDPI and ACS Style

Firanj Sremac, A.; Lalić, B.; Marčić, M.; Dekić, L. Toward a Weather-Based Forecasting System for Fire Blight and Downy Mildew. Atmosphere 2018, 9, 484. https://doi.org/10.3390/atmos9120484

AMA Style

Firanj Sremac A, Lalić B, Marčić M, Dekić L. Toward a Weather-Based Forecasting System for Fire Blight and Downy Mildew. Atmosphere. 2018; 9(12):484. https://doi.org/10.3390/atmos9120484

Chicago/Turabian Style

Firanj Sremac, Ana, Branislava Lalić, Milena Marčić, and Ljiljana Dekić. 2018. "Toward a Weather-Based Forecasting System for Fire Blight and Downy Mildew" Atmosphere 9, no. 12: 484. https://doi.org/10.3390/atmos9120484

APA Style

Firanj Sremac, A., Lalić, B., Marčić, M., & Dekić, L. (2018). Toward a Weather-Based Forecasting System for Fire Blight and Downy Mildew. Atmosphere, 9(12), 484. https://doi.org/10.3390/atmos9120484

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop