Next Article in Journal
Advances in Genomics-Based Breeding of Barley: Molecular Tools and Genomic Databases
Next Article in Special Issue
The Influence of Environmental Factors on Seed Germination of Polygonum perfoliatum L.: Implications for Management
Previous Article in Journal
Foliar-Supplied Molybdenum Improves Phyto-Nutritional Composition of Leaves and Fruits of Loquat (Eriobotrya japonica Lindl.)
Previous Article in Special Issue
Germination Parameters of Selected Summer Weeds: Transferring of the AlertInf Model to Other Geographical Regions
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predicting Seedling Emergence of Three Canarygrass (Phalaris) Species under Semi-Arid Conditions Using Parametric and Non-Parametric Models

by
Carlos Sousa-Ortega
1,*,
Aritz Royo-Esnal
2 and
José María Urbano
1
1
Departamento de Ciencias Agroforestales, Escuela Tecnica Superior de Ingenieria Agronomica, Universidad de Sevilla, Ctra. Utrera, km. 1, 41013 Sevilla, Spain
2
Departament d’Hortofruticultura, Botànica i Jardineria, ETSEA, Agrotecnio Centre, Universitat de Lleida, 25002 Lleida, Spain
*
Author to whom correspondence should be addressed.
Agronomy 2021, 11(5), 893; https://doi.org/10.3390/agronomy11050893
Submission received: 29 March 2021 / Revised: 23 April 2021 / Accepted: 26 April 2021 / Published: 1 May 2021

Abstract

:
The Phalaris genus includes annual weed species such as short-spiked canarygrass (Phalaris brachystachys Link.), little-seed canarygrass (Phalaris minor Retz.) and hood canarygrass (Phalaris paradoxa L.), which are especially problematic in Spain; as such, there is a need to develop models to predict the timing of their emergence. Field experiments were conducted at two different locations during two (2006/07 and 2007/08) and three (from 2005/06 to 2007/08) growing seasons. In both locations, 500 seeds of each Phalaris species were sown each growing season, simulating rain-fed cereal field conditions. In addition, the models were validated with three, four and eight independent experiments for P. brachystachys, P. minor and P. paradoxa, respectively. The emergence period of the three Phalaris species lasted between 31 and 48 days after sowing (DAS), showing two main flushes. The three cardinal points for parametric and non-parametric models were established to be between −1 °C and 1 °C for base temperature, between 9.8 °C and 11.8 °C for optimal temperature and between 21.2 °C and 23.4 °C for ceiling temperature; base water potential was estimated to be between −1 and −1.1 MPa. Both parametric and non-parametric models obtained similar results and were successfully validated in 12 out of 15 independent experiments.

1. Introduction

Phalaris is a grass genus that includes annual weed species that are spread through the Mediterranean basin area [1,2], India [3,4], China [5] and the United States [6]. Similar to many problematic annual weeds, the success of this genus as a weed can be attributed to its high seed production, and hence, high rate of population growth [7].
In Spain, there are three Phalaris weed species: short-spiked canarygrass (Phalaris brachystachys Link.), little-seed canarygrass (Phalaris minor Retz.) and hood canarygrass (Phalaris paradoxa L.) [1]. These weeds are highly concerning in winter cereals crops [8,9], where yields can be reduced from 16% with densities of 50 panicles m−2, and up to 60% with densities of 900 plants m−2 [10,11,12]; however, these species can also be problematic in some summer crops, such as sunflower [9].
The difficulty of their chemical control and the high cost of effective herbicides against these weeds are probably the main reasons these weed species are problematic in winter cereal crops [13]. In addition, the selection of herbicide-resistant biotypes has made their control even more complicated. There have been 11 and 8 cases of herbicide-resistant biotypes of P. minor and P. paradoxa that involve 22 and 12 active ingredients, respectively, of three modes of actions (ACCase inhibitors, ALS inhibitors and Photosystem II inhibitors) described so far. Further, there are four more cases for P. brachystachys, belonging to eight active ingredients of two modes of action (ACCase inhibitors and ALS inhibitors) [14]. Alcantara et al. [8] demonstrated, for the Phalaris species, that the efficacy of herbicides is higher when optimizing the timing of chemical control, when it is possible to reduce the number of applied herbicides, and hence, to reduce the application cost.
Understanding the pattern of weed emergence can help in the decision support system (DSS) for weed management [15], such as timing the herbicide application. Weed emergence is mainly determined by temperature, water potential, air quality and light [15], and some of these parameters have been effectively used to predict weed emergence. Thermal time (TT)-based models are the easiest to develop for weed emergence prediction as the temperature is the only considered factor, and for this reason, they have been widely used [16]. TT is the accumulation of daily thermal degrees (DTD), which are defined by three cardinal temperatures: base (Tb), optimum (To) and ceiling temperature (Tc). Tb is defined as the temperature at which DTD accumulation starts and increases linearly until To, which is the temperature at which greater DTD accumulation happens. Above To, DTD accumulation decreases until Tc, over which no more DTD accumulation occurs [17]. Hydrothermal time (HTT)-based models are more detailed than TT, where DTD accumulation can happen only when enough soil moisture is available, that is, when the soil water potential (Ψ) is above a base water potential (Ψb) [18].
Usually, TT and HTT-based models are developed with parametric non-linear regressions. At present, 113 emergence models (66 for dicot and 47 for monocots) have been developed using this procedure [16]. The success of these kinds of models relies on ease of management and practicality; however, they have some statistical limitations, mainly due to a lack of flexibility in capturing complex features [19]. Censored observations are another statistical limitation, as knowing the exact seedling emergence moment between two sampling dates is impossible; thus, alternative non-parametric regressions could be more appropriate approaches to describe weed emergence [19] and have already been developed by other authors [19,20].
Overall, models for the emergence description of the three Phalaris species would ease their management. In fact, a model for P. minor was already published [21], but this was developed under controlled conditions and only used moisture to predict germination. Many authors underline the importance of temperature in the emergence of Phalaris species [11,22,23]; thus, a TT or HTT model would be appropriate to describe their emergence. The aim of this work was to develop new TT and HTT-based models for each Phalaris species, using the two different approaches previously described, and compare their approximations to determine which would be most appropriate for future application.

2. Materials and Methods

2.1. Plant Material

In the case of P. brachystachys and P. paradoxa, seeds were harvested in June 2005 from a commercial wheat field near Jerez, Cadiz, in southern Spain (37.15 N, 6.18 W); whereas seeds of P. minor were collected from a commercial wheat field of Cordoba (37.86 N, 4.83 W). For the three species, seeds were collected from more than 20 individuals. Once at the laboratory, seeds were cleaned and dry-stored in a refrigerator at 4 °C until the sowing date.

2.2. Experimental Design

The experiment was carried out at two different locations, Tomejil (37.40 N, 5.58 W) and in the Higher Technical School of Agricultural Engineering of the University of Seville (ETSIA) (37.35 N, 5.93 W), during two (2006/07 and 2007/08) and three (2005/06 to 2007/08) growing seasons, respectively. The soil structure at Tomejil and ETSIA was vertisol (6% sand, 34% silt and 60% clay) and loam clay soil (39% sand, 29% silt and 32% clay), respectively.
In both locations, four quadrates of 25 × 25 cm were sown with 500 seeds for each Phalaris species and each growing season. The sowing procedure varied between locations. In Tomejil, commercial field soil was used without any previous replacement or sterilization. In this location, the upper 2 cm of soil layer was extracted, mixed with the corresponding seeds and then returned to the ground so that the seeds were evenly distributed in the top 2 cm of the soil. An additional control was introduced next to each plot to avoid the natural seed bank effect. In contrast, in the ETSIA experiments, the top 5 cm of soil was replaced by a mixture of 25% Kekkilä peat (Kekkilä Oy, Vantaa, Finland), 25% sand and 50% local soil. This mixture was oven-dried at 100 °C for 22 h to sterilize from pathogens and natural seed banks. Afterward, seeds were added to this mixture and were incorporated into the corresponding plot. In both locations, the date when the seeds were mixed with the soil was considered the sowing day. This procedure was followed each year and each newly sowed plot was placed next to that from the previous season, which was left undisturbed. This procedure allowed us to monitor seedling emergence over two growing seasons and to study the emergence under a tillage simulation (sowing season), in which the TT and HTT accumulation starting from the sowing date, and under non-tillage situation (second season), in which the initial moment was chosen at the first relevant rainfall. As a result, the models could be tested in eight different situations (Table 1).
Seedling emergence was recorded weekly with destructive counts to avoid double counts, between October and May each season. A 2 mm mesh cage was placed over each plot to avoid seed or seedling losses by predators.

2.3. Estimation of TT and HTT

Three biological parameters, Tb, To and Tc, were used to estimate TT [24,25]. For the estimation of HTT, the accumulation of TT was only produced when soil moisture (soil water potential, Ψ) rose above a base water potential (Ψb). TT and HTT are the accumulated DTD and hydrothermal degrees (DHD), which were obtained via the following:
If Tb > T < To: DTD = (T − Tb)/(To − Tb)
If To > T < Tc: DTD = (To − Tb)/(Tc − To) * (Tc − T)/(To − Tb)
If T < Tb or T > Tc: DTD = 0
If Ψ < Ψb: DHD = 0
If Ψ > Ψb: DHD = DTD
where T is the daily temperature, Tb is the base temperature, To is the optimal temperature, Tc is the ceiling temperature, Ψ is the daily water potential and Ψb is the base water potential.
The three biological parameters of TT for each species were established through an iterative process until the best combinations were obtained for the most accurate models. This best combination was chosen by representing field emergence percentages against the different TT, then applying parametric and non-parametric regressions. The same procedure was used to set the biological parameters for HTT; however, in this case, we added Ψb to the combination.
Daily maximum and minimum air temperatures and daily precipitation were obtained from the closest meteorological station in each location. In order to estimate the soil Ψ, Saxton and Rawls [26] equations were followed. We also used a procedure defined by Fuentes-Yagüe and García-Legaspi [27] to estimate the soil water content needed for these equations.

2.4. Development of the Parametric Model

The Weibull three parameters in Equation (1) were tested as a biological growth model in a parametric non-linear regression.
Y = A   { 1 e [ e ( k ) x b ] }
where Y is the cumulative emergence percentage from the total percentage of seedling emergence (A), which in this case was fixed to 100% each season, x is the TT or HTT and k is the slope of the curve at b, which is the inflection point.

2.5. Development of the Non-Parametric Model

For the non-parametric regression, observed data were aggregated using the next cumulated kernel density estimation (2) [19]:
F ( x ) = i = 1 n W i   K ( t ) ( x t i h )
where K(t) is the integrated Gaussian kernel (3), ti is the TT or HTT at the middle of two consecutive inspections, Wi is the percentage of emerged seedlings counted between two consecutive inspections and h is the bandwidth or the smoother parameter.
K ( t ) = 1 2 π e t 2 / 2
The bandwidth (h) was selected using the plug-in approach. This approach was conducted with the R software, using the function “hpi.kcde” from the “ks” package [28]. Then, the “kcde” function from the same package was used to apply the Gaussian kernel distribution explained before.

2.6. Model Accuracy

The accuracy of the models was tested with the root mean square error (RMSE, 4), where greater model accuracy, and thus the best combination of biological parameters, is indicated by lower RMSE values. As a reference, RMSE was evaluated with the scale proposed by Royo-Esnal et al. [29] for percentages of emergence; their model is as follows: excellent accuracy for values under 5, very good for values between 5 and 10, good for values between 10 and 15 and not successful for values over 15.
R M S E = 1 / n i = 0 n ( x i y i ) 2
where xi represents the observed cumulative percentage emergence, yi is the predicted cumulative percentage of emergence and n is the number of observations.

2.7. Data Validation

The developed models were tested with independent data obtained from two locations (Table 2), Seville-Garden and ETSIA, during three (from 2016/17 to 2018/19 season) and one season (2018/19), respectively. In Seville-Garden (37.35 N, 5.93 W), the experiment was conducted using 35 L pots containing a mixture of 50% sand and 50% peat moss, while in the ETSIA (same location as the experiment), the experiment was performed in 25 × 25 cm2 quadrants, as in the original experiment.
Data from five independent experiments for the emergence of P. paradoxa across Spain (Table 3) were used to validate the models. In these experiments, plots were also quadrates of 25 cm × 25 cm2.
The sowing procedure for all validation experiments was the same. In each plot, the upper 2 cm of the soil layer was extracted, mixed with 1000 seeds and then returned to the ground so that the seeds were evenly distributed. Sowed seed density was between 500 and 1000; the percentage of emerged seedlings was used for validation, fixing the maximum number to 100%.
During the experiments, a temperature datalogger (digital thermometer DS18B20; Maxim, San José, CA, USA) was buried at a depth of 2 cm in each experimental location. Rainfall data were obtained from the closest meteorological station for each location.

3. Results

3.1. Weather Conditions

During the three seasons between September to May, the mean temperature was similar in both locations, ranging from 13.5 °C to 14.9 °C. September was the hottest month with a monthly mean temperature between 22.4 °C and 24.2 °C, whereas January was the coldest month (8.2–10.8 °C). On the other hand, precipitation varied in quantity and in distribution across the three seasons, ranging from 363 mm to 539 mm, with the highest recorded rainfall occurring in October, November or April, depending on the year (Table 4 and Figure 1). However, moisture was not a limiting factor that prevented seedlings from emerging in any of the three seasons.

3.2. Description of the Emergence

In the years in which seeds were sowed, the emergence period (time required for 10% to 90% emergence) of the three Phalaris species lasted between 31 and 48 days after sowing (DAS); 10% of emergence was reached between 2 and 27 December, whereas 50% and 90% were achieved between 28 December and 1 February and between 9 and 29 January, respectively. In most cases, the emergence pattern showed two flushes, the first in December and the second in February. Despite the similarity of the emergence pattern between the three species, P. minor showed a longer emergence period, whereas it was shorter in P. brachystachys (Table 5).
Seeds that did not emerge during the first season and remained in the soil began emergence earlier in the following season due to the rainfall during October. In 2006/07, 50% emergence was reached during December, but 90% emergence occurred in February; thus, the emergence period was approximately three months. On the contrary, during 2007/08, the emergence period was similar or even shorter than that in newly sowed plots, except for P. brachystachys, which showed similar behavior to the previous season (Figure 2 and Table 5).

3.3. Accuracy of the Parametric Model

The biological parameters established for Phalaris species were very similar, setting 1 °C as Tb for the three of them. Additionally, To and Tc were established at 9.9 °C and 22.4 °C, 10.0 °C and 21.2 °C and 11.8 °C and 22.8 °C for P. brachystachys, P. paradoxa and P. minor, respectively. On the other hand, the Ψb was established at −1.0 MPa for P. paradoxa and P. minor, whereas it was established at −1.1 MPa for P. brachystachys.
The parameters of the Weibull equation are defined in Table 6. Regarding these equations, Phalaris species achieved 50% emergence between 24 and 30 cumulated daily thermal degrees (DTD) for TT and between 22 and 39 cumulated daily hydrothermal degrees (DTD) for HTT.
According to the accuracy of the parametric models, TT models for each species presented average RMSE values between 10.9 and 14.5. The most accurate model was for P. minor, in which an excellent accuracy (RMSE ≤ 5) was achieved in one situation, good accuracy (5 < RMSE ≤ 10) in four more situations and sufficient accuracy (10 < RMSE ≤ 15) and unsuccessful accuracy (RMSE > 15) in one situation each (Table 7). On the other hand, P. paradoxa and P. brachystachys presented a good fit in two and three situations, sufficient in one and four situations and unsuccessful in four and one more situation, respectively (Table 7, Figure 3).
Regarding the HTT model, soil moisture did not improve the model accuracy for P. minor and P. brachystachys; however, it slightly increased the precision for P. paradoxa, reducing the RMSE value from 14.5 to 14.2 (Table 7, Figure 3).

3.4. Accuracy of the Non-Parametric Model

The best combinations of the three cardinal temperatures for the non-parametric models were different from those established with the parametric model. Tb, To and Tc were established at 0 °C, 9.8 °C and 23.4 °C for P. brachystachys, at −1 °C, 9.8 °C and 22.5°C for P. paradoxa and at 1 °C, 12.3 °C and 22.7 °C for P. minor. However, the Ψb value for the non-parametric model was the same as for the parametric models.
With the non-parametric TT model, 50% emergence for P. paradoxa was reached at 42 cumulated DTD and 90% emergence at 75; whereas 30 and 62 total accumulated DHD were needed for HTT to reach 50% and 90% emergence, respectively. For P. minor, TT and HTT models required 32 and 23 accumulated DTD/DHD, respectively, to achieve 50% emergence and 61 accumulated DTD/DHD was required to reach 90% emergence. Finally, the non-parametric models for P. brachystachys, required 40 and 67 accumulated DTD/DHD to reach 50% and 90% emergence (Figure 4).
The RMSE values for the TT-based models of the three Phalaris species were between 10.9 and 15.8. The most accurate model was obtained for P. minor, which was excellent for one situation, good for four more situations and sufficient and unsuccessful in one situation each. The accuracy of the P. brachystachys model was good in three situations, sufficient in another three and insufficient in one situation; however, the models for P. paradoxa only obtained excellent accuracy in one situation, good in another, sufficient in two more and unsuccessful in four situations (Table 8, Figure 5).
Concerning the HTT models, the soil moisture had only slightly improved the precision for the P. paradoxa model, reducing the RMSE value from 15.8 to 14.4 (Table 8, Figure 5).

3.5. Validation of the Models with Independent Data Sets

Although the validation of the parametric and non-parametric models with the independent data was similar, the consideration of the soil moisture increased the precision of the parametric and non-parametric HTT-based models for P. minor and P. paradoxa with respect to the TT-based ones; however, this did not happen for P. brachystachys (Table 9, Figure 6).
The models for P. brachystachys showed lower accuracy than the models of the other two Phalaris species for the description of the independent data. In this case, the P. brachystachys model only achieved sufficient accuracy at one location out of three and was best with the non-parametric TT model. On the contrary, P. minor and P. paradoxa models were successfully validated with the independent data. In the case of P. minor, the RMSE values were successful at three of the four studied locations (Table 9, Figure 6), with the parametric HTT model obtaining the highest accuracy. Further, HTT reduced the RMSE values for parametric and non-parametric models at all validation locations. Concerning P. paradoxa, the models were successfully validated at the eight locations, where the non-parametric HTT model was the most accurate; in this case, the soil moisture reduced the RMSE values at five out of eight locations.

4. Discussion

4.1. Emergence Pattern

Similar to the observations of Jiménez-Hidalgo [11] and Taylor et al. [30], the emergence patterns of the three Phalaris species were similar, with the first seedlings observed in November and the last ones in March or even April, depending on the season, but in all cases with two flushes during the emergence season, the first one in December and the second in February. Both flushes occurred when the mean air temperature was close to the optimum temperature, between 9 °C and 13 °C, and was separated by a period when the mean air temperature dropped below 9 °C.
When we compared the emergence of the Phalaris species to that of other problematic winter weeds in the south of Spain, we found that they were similar to that of winter wild oat (Avena sterilis subsp. ludoviciana (Durieu) Gillet and Magne), of which the emergence period also lasts from November to March [31]. On the contrary, Phalaris showed more staggered emergence periods than north African knapweed (Centaurea diluta Aiton), which emerges between late October and early January [25], or ryegrass (Lolium rigidum Gaudin), which emerges from November to February [24]. The wide emergence period of the Phalaris species could also contribute to their presence as problematic weeds in summer crops.

4.2. Threshold Parameters (Tb, To, Tc and Ψb)

The three cardinal temperatures (Tb, To and Tc) were similar between models and species. The Tb ranged from −1 °C to 1 °C, which is a similar to that used by González-Andújar et al. [20] for P. paradoxa, who established this parameter at 0.8 °C; lower than that established by Mesgaran et al. [32] and Derakhshan et al. [22] for P. minor (3.5 °C and 4.7 °C, respectively), but the latter were estimated for germination under controlled conditions in germination chambers and might vary from required field conditions for emergence.
The established To, between 9.9 °C and 12.3 °C, was also lower than that set under controlled conditions by Derakhshan et al. [22], who established them between 22.6 and 29.6 °C. The differences between the results observed in the present manuscript and those from Derakhshan et al. [22] might be explained because the authors established a To for germination, and because seeds belong to a population growing at a completely different climate (north of Iran) and temperature requirements could easily be different. However, the To estimated in the present work is in agreement with values estimated in field conditions by Jiménez-Hidaldo [11] and Taylor et al. [30], who describe an optimum range between 10–15 °C and 10–17 °C, respectively. Similar to Tb and To, Tc was also lower than that established under controlled conditions [22,33], but this parameter has not been previously estimated in field conditions for Phalaris species in any other work, and thus, no comparison is possible.
Concerning Ψb, no significant differences were found between the three Phalaris species: it was set to −1.1 MPa for P. brachystachys and −1 MPa for the other two species. Despite this, P. brachystachys was shown to be a little more tolerant to drought conditions for the emergence; this is in agreement with observations from Alcantara et al. [34], who described a higher drought tolerance by P. brachystachys than P. minor during growth stages.
Apart from soil temperature and moisture, other factors are known to affect the start of the emergence process, considering this process from dormancy release to the emergence of cotyledons on the soil surface. Light is known to promote the release of dormant seeds of Phalaris species, such as P. paradoxa [35] and P. minor [36]; however, in the present study, light could only have been a relevant variable at the sowing date, as no further soil disturbance was performed. An interesting future experiment could be established to study the effect of tillage at different dates on the emergence of these Phalaris species and the suitability of the models developed in the present manuscript to those emergence patterns.

4.3. Parametric vs. Non-Parametric Models

Although the non-parametric model could have slightly improved the accuracy of the parametric TT-based model for P. brachystachys and the HTT-based model for P. minor, in practice, both parametric and non-parametric models showed very similar accuracy (Table 3 and Table 7, Figure 3 and Figure 5), as is shown by the validation results, which show similar precision to those of the parametric model, with improved accuracy at 9 of 15 experimental locations (Table 8). These results show that non-parametric models have the same capacity to describe and predict weed emergence as classical parametric models, which is somehow in disagreement with the observations from Gonzalez-Andújar et al. [20], who concluded that non-parametric models were only useful for describing and not for predicting weed emergence. These differences could be explained by the fact that Gonzalez-Andújar et al. [20] developed a model for P. paradoxa using only data from a single location and year to predict its emergence at the same locations during the next two years. The present work used data from two locations and three seasons; thus, its conclusions are more reliable than those obtained from a single season.

4.4. Applicability of the Models

In this work, emergence models were successfully developed to describe the emergence of three Phalaris species, with both parametric and non-parametric statistical tools. Emergence models frequently present limited application due to within-species variation. Very often, models developed in one location cannot be applied to other locations with different agroclimatic conditions because models tend to fail when environmental conditions vary from where they were developed [37]. Moreover, the mother plant effect promotes differences in the dormancy and germination characteristics of the offspring, even in the same population [38]. This has been the case of L. rigidum, whose emergence model could successfully be applied in central and north-east Spain, but not in the south [24]. However, in the present work, the three models could be validated with independent data from the same (Guadalcázar, Seville, Tomejil) and from different agroclimatic locations (Burgos, Huesca, and Valladolid) within Spain. This result suggests that the applicability of these models is wider than for other weed species, and that they could be applied with fair accuracy, at least in the whole Iberian Peninsula with a Mediterranean climate. Despite this promising result, the applicability of these models for the emergence of Phalaris species in other Mediterranean climatic zones should be checked and be calibrated if necessary.
Ideally, these models should be implemented in software tools together with other already developed models. There already are several models for the emergence of winter weeds in Spain, such as those for Papaver rhoeas [37], L. rigidum [24], A. sterilis [31] or Bromus diandrus [39], among others, and an effort should be made to integrate them into a DSS tool, such as AlertInf [40,41], which is being applied in the Venetto region (Italy) with considerable success.

5. Conclusions

The parametric and non-parametric models successfully described the emergence of P. brachystachys, P. minor and P. paradoxa. In addition, both models presented similar results at independent validation data, obtaining good accuracy at one of three, three of four and eight of eight experimental locations for P. brachystachys, P. minor and P. paradoxa, respectively. Our findings demonstrate that parametric and non-parametric statistical tools can predict the emergence of seedlings under different agroclimatic conditions. These models, especially the non-parametric HTT models, are potentially applicable to actual field situations by providing guidance for DSS to implement weed management tactics.

Author Contributions

Conceptualization, J.M.U. and C.S.-O.; data curation, C.S.-O.; formal analysis, C.S.-O.; funding acquisition, J.M.U.; investigation, all authors; methodology, J.M.U.; project administration, J.M.U. and C.S.-O.; resources, J.M.U.; software, C.S.-O.; supervision J.M.U. and C.S.-O.; validation J.M.U. and C.S.-O.; visualization, all authors; writing—original draft, C.S.-O. and A.R.-E.; writing—review and editing, C.S.-O. and A.R.-E. All authors have read and agreed to the published version of the manuscript.

Funding

This research and the APC were both funded by CATEDRA ADAMA. European Regional Development Funds (FEDER) and the Spanish Ministry of Economy and Competitiveness funds (AGL2005-0405) also supported data collection between 2005 and 2008.

Acknowledgments

We would like to thank the students and the technicians who have helped in this experiment’s fieldwork in the different locations; without their help, data collection would have been more difficult.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. García Baudín, J.; Contreras de Vera, A.; Salto, T. Especies adventicias del género Phalaris, en los cereales de invierno de Andalucía Occidental. Boletín Sanid. Veg. Plagas 1982, 8, 185–191. [Google Scholar]
  2. Damanakis, M. Weed species in wheat fields of Greece-1982, 1983 survey. Zizaniologia 1983, 1, 85–90. [Google Scholar]
  3. Chhokar, R.S.; Sharma, R.K. Multiple herbicide resistance in littleseed canarygrass (Phalaris minor): A threat to wheat production in India. Weed Biol. Manag. 2008, 8, 112–123. [Google Scholar] [CrossRef]
  4. Bir, S.S.S.M. Observations on the weed flora of cultivable lands in Punjab- wheat fields in Patiala District. New Bot. 1979, 6, 79–89. [Google Scholar]
  5. Xu, G.; Shen, S.; Zhang, Y.; Clements, D.R.; Yang, S.; Li, J.; Dong, L.; Zhang, F.; Jin, G.; Gao, Y. Designing Cropping Systems to Improve the Management of the Invasive Weed Phalaris minor Retz. Agronomy 2019, 9, 809. [Google Scholar] [CrossRef] [Green Version]
  6. USDA PLANTS database. Choice Rev. Online 2006, 43. [CrossRef]
  7. Gonzalez-Andujar, J.L.; Jimenez-Hidalgo, M.; Garcia-Torres, L.; Saavedra, M. Demography and population dynamic of the arable weed Phalaris brachystachys L. (short-spiked canary grass) in winter wheat. Crop Prot. 2005, 24, 581–584. [Google Scholar] [CrossRef]
  8. Alcantara, C.; Fuentes-García, M.; Carranza, R.; Sanchez-Gamonoso, M.; Saavedra, M. Aplicaciones de Herbicidas de Postemergencia Temprana y Tardía Contra Alpisteras (Phalaris spp.) en Trigo Duro; Consejería de Agricultura, Pesca y Desarrollo Rural. Instituto de Investigación y Formación Agraria y Pesquera: Cordoba, Spain, 2017. [Google Scholar]
  9. Ortiz, R.; Contreras, J.M.; Ruiz, A.; Sanz, M.A.; Romero, M.; Gordillo, M.; Taberner, A.; Urbano, J.M.; Urbano Fuentes-Guerra, J.M. Malas hierbas preocupantes en españa. Actas la Soc. Española Malherbologia 2015, 1, 497–503. [Google Scholar]
  10. Cudney, D.W.; Hill, J.E. The response of wheat grown with three population levels of canarygrass to various herbicide treatments. Proc. West. Soc. Weed Sci. 1979, 55–56. [Google Scholar]
  11. Jimenez-Hidalgo, M. Incidencia y Evolución de los Alpistes (Phalaris ssp.) En el Trigo (Triticum sp.); Universidad de Córdoba: Andalusia, Spain, 1993. [Google Scholar]
  12. Afentouli, C.G.; Eleftherohorinos, I.G. Competition between wheat and canarygrass biotypes and their response to herbicides. Weed Sci. 1999, 47, 55–61. [Google Scholar] [CrossRef]
  13. González-Diaz, L.; Bastida, F.; Gonzalez-Andujar, J.L. Short comunication. Modelling of the population dynamics of Phalaris brachystachys Link under various herbicide control scenarios in a Mediterranean climate. Spanish J. Agric. Res. 2009, 7, 155–159. [Google Scholar] [CrossRef] [Green Version]
  14. Heap, I. International Survey of Herbicide Resistant Weeds. Available online: http://www.weedscience.org (accessed on 1 May 2020).
  15. Forcella, F.; Benech-Arnold, R.L.; Sánchez, R.; Ghersa, C.M. Modelling seedling emergence. Field Crops Res. 2000, 67, 123–139. [Google Scholar] [CrossRef]
  16. Royo-Esnal, A.; Torra, J.; Chantre, G.R. Weed Emergence Models. In Decision Support Systems for Weed Management; Springer: Berlin/Heidelberg, Germany, 2020; pp. 85–116. [Google Scholar]
  17. Bradford, K.J. Applications of hydrothermal time to quantifying and modeling seed germination and dormancy. Weed Sci. 2002, 50, 248–260. [Google Scholar] [CrossRef]
  18. Grundy, A.C. Predicting weed emergence: A review of approaches and future challenges. Weed Res. 2003, 43, 1–11. [Google Scholar] [CrossRef]
  19. Cao, R.; Francisco-Fernández, M.; Anand, A.; Bastida, F.; González-Andújar, J.L. Computing statistical indices for hydrothermal times using weed emergence data. J. Agric. Sci. 2011, 149, 701–712. [Google Scholar] [CrossRef] [Green Version]
  20. Gonzalez-Andujar, J.L.; Francisco-Fernandez, M.; Cao, R.; Reyes, M.; Urbano, J.M.; Forcella, F.; Bastida, F. A comparative study between nonlinear regression and nonparametric approaches for modelling Phalaris paradoxa seedling emergence. Weed Res. 2016, 56, 367–376. [Google Scholar] [CrossRef]
  21. Mesgaran, M.B.; Mashhadi, H.R.; Alizadeh, H.; Hunt, J.; Young, K.R.; Cousens, R.D. Importance of distribution function selection for hydrothermal time models of seed germination. Weed Res. 2013, 53, 89–101. [Google Scholar] [CrossRef]
  22. Derakhshan, A.; Gherekhloo, J.; Vidal, R.A.; De Prado, R. Quantitative Description of the Germination of Littleseed Canarygrass (Phalaris minor) in Response to Temperature. Weed Sci. 2014, 62, 250–257. [Google Scholar] [CrossRef]
  23. Singh, S.; Kirkwood, R.C.; Marshall, G. Biology and control of Phalaris minor Retz. (littleseed canarygrass) in wheat. Crop Prot. 1999, 18, 1–16. [Google Scholar] [CrossRef]
  24. Sousa-Ortega, C.; Chamber, E.; Urbano, J.M.; Izquierdo, J.; Loureiro, I.; Marí, A.I.; Cordero, F.; Vargas, M.; Saavedra, M.; Lezaun, J.A.; et al. Should emergence models for Lolium rigidum be changed throughout climatic conditions? The case of Spain. Crop Prot. 2020, 128, 105012. [Google Scholar] [CrossRef]
  25. Sousa-Ortega, C.; Royo-Esnal, A.; DiTommaso, A.; Izquierdo, J.; Loureiro, I.; Marí, A.I.; Cordero, F.; Vargas, M.; Saavedra, M.; Paramio, J.A.; et al. Modeling the Emergence of North African Knapweed (Centaurea diluta), an Increasingly Troublesome Weed in Spain. Weed Sci. 2020, 1–30. [Google Scholar] [CrossRef]
  26. Saxton, K.E.; Rawls, W.J. Soil water characteristic estimates by texture and organic matter for hydrologic solutions. Soil Sci. Soc. Am. J. 2006, 70, 1569–1578. [Google Scholar] [CrossRef] [Green Version]
  27. Fuentes-Yagüe, J.; García-Legaspi, G. Técnicas de Riego. Ministerio de Agricultura, Pesca y Alimentación; Ministerio de Agricultrua, pesca y alimentación, Ed.; Mundi-Prensa México: Ciudad Cuauhtémoc, México, 1999; ISBN 968-7462-17-5. [Google Scholar]
  28. Duong, T. Ks: Kernel density estimation and kernel discriminant analysis for multivariate data in R. J. Stat. Softw. 2007, 21, 1–16. [Google Scholar] [CrossRef] [Green Version]
  29. Royo-Esnal, A.; Torra, J.; Conesa, J.A.; Recasens, J. Characterisation of emergence of autumn and spring cohorts of Galium spp. in winter cereals. Weed Res. 2010, 50, 572–585. [Google Scholar] [CrossRef] [Green Version]
  30. Taylor, I.N.; Walker, S.R.; Adkins, S.W. Burial depth and cultivation influence emergence and persistence of Phalaris paradoxa seed in an Australian sub-tropical environment. Weed Res. 2005, 45, 33–40. [Google Scholar] [CrossRef]
  31. Sousa-Ortega, C.; Royo-Esnal, A.; Loureiro, I.; Marí, A.I.; Lezáun, J.A.; Cordero, F.; Saavedra, M.; Paramio, J.A.; Fernández, J.L.; Torra, J.; et al. Modelling Emergence of Sterile Oat (Avena sterilis spp ludoviciana) Under Semi-arid Conditions. Weed Sci. 2021, 1–29. [Google Scholar] [CrossRef]
  32. Mesgaran, M.B.; Onofri, A.; Mashhadi, H.R.; Cousens, R.D. Water availability shifts the optimal temperatures for seed germination: A modelling approach. Ecol. Modell. 2017, 351, 87–95. [Google Scholar] [CrossRef]
  33. Ahmadi, A.; Hosseini, M.; Zeidali, E. Study of ecological characteristics of canary grass (Phalaris minor). Tech. J. Eng. Appl. Sci. 2013, 3, 1835–1840. [Google Scholar]
  34. Alcantara, C.; Jimenez-Hidalgo, M.J.; Saavedra, M. Responses of Phalaris minor Rezt. and Phalaris brachystachys Link to different levels of soil water availability. Spanish J. Agric. Res. 2010, 8, 1074. [Google Scholar] [CrossRef] [Green Version]
  35. Taylor, I.N.; Peters, N.C.B.; Adkins, S.W.; Walker, S.R. Germination response of Phalaris paradoxa L. seed to different light quality. Weed Res. 2004, 44, 254–264. [Google Scholar] [CrossRef]
  36. Ohadi, S.; Rahimian Mashhadi, H.; Tavakkol-Afshari, R.; Beheshtian Mesgaran, M. Modelling the effect of light intensity and duration of exposure on seed germination of Phalaris minor and Poa annua. Weed. Res. 2010, 50, 209–217. [Google Scholar] [CrossRef]
  37. Izquierdo, J.; González-Andújar, J.L.; Bastida, F.; Lezaún, J.A.; del Arco, M.J.S. A Thermal Time Model to Predict Corn Poppy (Papaver rhoeas) Emergence in Cereal Fields. Weed Sci. 2009, 57, 660–664. [Google Scholar] [CrossRef]
  38. Baskin, C.C.; Baskin, J.M. Variation in Seed Dormancy and Germination within and between Individuals and Populations of a Species. In Seeds (Second Edition): Ecology, Biogeography, and Evolution of Dormancy and Germination; Academic Press: New York, NY, USA, 2014; pp. 277–373. ISBN 9780124166776. [Google Scholar]
  39. García, A.L.; Recasens, J.; Forcella, F.; Torra, J.; Royo-Esnal, A. Hydrothermal Emergence Model for Ripgut Brome (Bromus diandrus). Weed Sci. 2013, 61, 146–153. [Google Scholar] [CrossRef]
  40. Masin, R.; Loddo, D.; Gasparini, V.; Otto, S.; Zanin, G. Evaluation of Weed Emergence Model AlertInf for Maize in Soybean. Weed Sci. 2014, 62, 360–369. [Google Scholar] [CrossRef]
  41. Masin, R.; Loddo, D.; Benvenuti, S.; Otto, S.; Zanin, G. Modeling Weed Emergence in Italian Maize Fields. Weed Sci. 2012, 60, 254–259. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Daily mean air temperatures (lines) and daily precipitation (bars) at each experimental location.
Figure 1. Daily mean air temperatures (lines) and daily precipitation (bars) at each experimental location.
Agronomy 11 00893 g001
Figure 2. Daily emergence of the three Phalaris species at the two experimental locations. Black lines—first season emergence (sowing season); dotted lines—second season emergence (from the remaining seed bank). PHABR: Phalaris brachystachys; PAHMI: P. minor; PHAMPA: P. paradoxa.
Figure 2. Daily emergence of the three Phalaris species at the two experimental locations. Black lines—first season emergence (sowing season); dotted lines—second season emergence (from the remaining seed bank). PHABR: Phalaris brachystachys; PAHMI: P. minor; PHAMPA: P. paradoxa.
Agronomy 11 00893 g002
Figure 3. Predicted emergence for the three Phalaris species, using thermal (dotted line) and hydrothermal (black line) time-based Weibull models at each location and for the first and second seasons in each plot. Black dots—observed (mean) seedling emergence. Scarce seedling emergence did not allow fitting the models to the data in P. brachystachys and P. minor at Tomejil-S07YR08.
Figure 3. Predicted emergence for the three Phalaris species, using thermal (dotted line) and hydrothermal (black line) time-based Weibull models at each location and for the first and second seasons in each plot. Black dots—observed (mean) seedling emergence. Scarce seedling emergence did not allow fitting the models to the data in P. brachystachys and P. minor at Tomejil-S07YR08.
Agronomy 11 00893 g003
Figure 4. Representation of non-parametric thermal (a) and hydrothermal (b) time models for Phalaris brachystachys (red), Phalaris minor (blue) and Phalaris paradoxa (black).
Figure 4. Representation of non-parametric thermal (a) and hydrothermal (b) time models for Phalaris brachystachys (red), Phalaris minor (blue) and Phalaris paradoxa (black).
Agronomy 11 00893 g004
Figure 5. Predicted emergence of the three Phalaris species using thermal (dotted line) and hydrothermal (black line) non-parametric model at each location and for the first and second seasons in each plot. The black dots show observed seedling emergence. The missing values of P. brachystachys and P. minor at Tomejil-S07YR08 are due to the lack of enough seedling emergence to apply the models.
Figure 5. Predicted emergence of the three Phalaris species using thermal (dotted line) and hydrothermal (black line) non-parametric model at each location and for the first and second seasons in each plot. The black dots show observed seedling emergence. The missing values of P. brachystachys and P. minor at Tomejil-S07YR08 are due to the lack of enough seedling emergence to apply the models.
Agronomy 11 00893 g005
Figure 6. Predicted emergence of the three Phalaris species using parametric (black line) and non-parametric (dotted line) hydrothermal model at each validation location. The black dots show observed seedling emergence.
Figure 6. Predicted emergence of the three Phalaris species using parametric (black line) and non-parametric (dotted line) hydrothermal model at each validation location. The black dots show observed seedling emergence.
Agronomy 11 00893 g006
Table 1. Sowing locations and codes, sowing dates, dates of the first relevant precipitation and the total number of emergences for each Phalaris species, considering the three repetitions together. PHABR, P. brachystachys; PHAMI, P. minor; PHAPA, P. paradoxa.
Table 1. Sowing locations and codes, sowing dates, dates of the first relevant precipitation and the total number of emergences for each Phalaris species, considering the three repetitions together. PHABR, P. brachystachys; PHAMI, P. minor; PHAPA, P. paradoxa.
LocationCode 1Sowing DateFirst Relevant RainTotal Emergence
PHABRPHAMIPHAPA
ETSIAS06YR0611 November 200514 November 20051794441519
ETSIAS06YR07 13 September 20064332174
ETSIAS07YR0712 November 200616 November 2006139162914
ETSIAS07YR08 21 September 20074638178
ETSIAS08YR0815 November 200720 November 20079566895
TomejilS07YR0715 November 200616 November 20065590473
TomejilS07YR08 21 September 2007--36
TomejilS08YR0820 November 200722 November 200719058679
1 The sowing code represents the year when sowing was performed and the year when the emergence was monitored, i.e., both S06YR06 and S06YR07 are the same plot that was sown in 2005/06, but the first code represents the emergence results from the same season (2005/06), whereas the second is showing results from the second season (2006/07).
Table 2. Total emergence for the independent experiments.
Table 2. Total emergence for the independent experiments.
LocationYearNumber of Emerged Seedling
P. brachystachysP. minorP. paradoxa
Sevilla-Garden2016/171229-
2017/18-4816
2018/1917103-
ETSIA2018/19262418
Table 3. Additional P. paradoxa experiments. Location, soil texture and total emerged seedling.
Table 3. Additional P. paradoxa experiments. Location, soil texture and total emerged seedling.
LocationLatitudeLongitudeSowing Date(2019)Sand (%)Silt (%)Clay (%)Emerged Seedling
Burgos42.4402 N3.7209 WSept 18224632369
ETSIA37.3524 N5.9392 WSept 20383032242
Guadalcazar37.7558 N4.9354 WOct 17363330231
Huesca42.1277 N0.3987 WOct 17254035514
Tomejil37.4027 N5.5878 WSept 2353262158
Valladolid41.7789 N4.8752 WOct 10622216507
Table 4. Average monthly temperature and total monthly precipitation at each experimental location during the experiment.
Table 4. Average monthly temperature and total monthly precipitation at each experimental location during the experiment.
LocationMonthTemperature (°C)Precipitation (mm)
2005/062006/072007/082005/062006/072007/08
ETSIASeptember22.423.723.00.037.042.8
ETSIAOctober17.719.618.8119.2197.822.6
ETSIANovember11.514.313.125.4120.691.4
ETSIADecember9.78.99.629.043.415.0
ETSIAJanuary7.18.210.838.030.444.8
ETSIAFebruary9.111.813.452.859.668.4
ETSIAMarch13.413.414.163.612.420.0
ETSIAApril17.015.316.835.238.0165.4
Average 13.514.414.9363.2539.2470.4
TomejilSeptember 24.223.4 65.830.0
TomejilOctober 20.418.9 93.244.2
TomejilNovember 15.213.3 78.8118.2
TomejilDecember 9.410.1 37.812.8
TomejilJanuary 8.811.1 32.857.4
TomejilFebruary 12.013.6 58.065.0
TomejilMarch 12.013.0 20.625.8
TomejilApril 14.316.0 43.2177.6
Average 14.514.9 430.2531.0
Table 5. Emergence periods and dates when 50% emergence was reached in each of the three Phalaris species.
Table 5. Emergence periods and dates when 50% emergence was reached in each of the three Phalaris species.
Location.Sowing 1P. brachystachysP. paradoxaP. minor
50% 2Emergence 3
Period
50% 2Emergence 3
Period
50% 2Emergence 3
Period
ETSIAS06YR06Dec. 2844.1Dec. 2059.1Dec. 2090.2
ETSIAS06YR07Nov. 3089.9Nov. 2386.8Nov. 1188.8
ETSIAS07YR07Jan. 1731.6Jan. 0774.9Nov. 2661.6
ETSIAS07YR08Dec. 0788.6Dec. 0745.7Mar. 1846.7
ETSIAS08YR08Dec. 1842.0Dec. 0716.4Dec. 1037.8
TomejilS07YR07Feb. 0148.0Jan. 2954.0Jan. 3061.8
TomejilS07YR08--Jan. 0252.9-36.9
TomejilS08YR08Dec. 3030.7Dec. 1130.4Dec. 3036.9
1 See sowing code in Table 1. 2 50% represents when 50% emergence was reached. 3 Emergence period represents the difference in days between the dates when 10% and 90% emergence was reached.
Table 6. Parameters of Weibull Equation (1) for the thermal and hydrothermal time-based models of the three Phalaris species.
Table 6. Parameters of Weibull Equation (1) for the thermal and hydrothermal time-based models of the three Phalaris species.
ModelP. brachystachysP. paradoxaP. minor
Kbkbkb
TT8.2641872.1349855.1615811.4012544.5895671.3303
HTT9.2314232.4103346.0932731.7474694.7112481.402724
Table 7. Root mean square error (RMSE) values obtained from thermal (TT) and hydrothermal (HTT) time-based Weibull models for each Phalaris species.
Table 7. Root mean square error (RMSE) values obtained from thermal (TT) and hydrothermal (HTT) time-based Weibull models for each Phalaris species.
LocationSowingP. brachystachysP. paradoxaP. minor
TTHTTTTHTTTTHTT
ETSIAS06YR0610.28.25.65.37.39.5
ETSIAS06YR0710.011.110.78.85.46.4
ETSIAS07YR0713.013.58.212.89.58.5
ETSIAS07YR085.57.07.512.33.45.3
ETSIAS08YR0811.110.019.914.78.57.6
TomejilS07YR0724.715.924.422.230.426.4
TomejilS07YR08--22.213.0--
TomejilS08YR0813.734.817.324.511.916.5
Average 12.614.414.514.210.911.5
Scarce seedling emergence did not allow fitting the models to the data from P. brachystachys and P. minor at Tomejil-S07YR08.
Table 8. Root mean square error (RMSE) values obtained from thermal (TT) and hydrothermal (HTT) time non-parametric models for each Phalaris species.
Table 8. Root mean square error (RMSE) values obtained from thermal (TT) and hydrothermal (HTT) time non-parametric models for each Phalaris species.
LocationSowingP. brachystachysP. paradoxaP. minor
TTHTTTTHTTTTHTT
ETSIAS06YR067.98.012.33.67.39.7
ETSIAS06YR079.911.116.67.15.45.5
ETSIAS07YR0713.412.77.512.29.511.3
ETSIAS07YR086.56.54.312.43.43.6
ETSIAS08YR0811.112.129.816.58.56.6
TomejilS07YR0726.413.913.721.030.422.3
TomejilS07YR08--15.213.6--
TomejilS08YR0811.436.527.128.311.917.8
Average 12.414.415.814.410.911.0
The missing values of P. brachystachys and P. minor at Tomejil-S07YR08 are due to the lack of enough seedling emergence to apply the models.
Table 9. Root mean square error (RMSE) values obtained from thermal (TT) and hydrothermal (HTT) times of parametric and non-parametric models at each validation location. Numbers in bold show the lowest RMSE value in each location and season.
Table 9. Root mean square error (RMSE) values obtained from thermal (TT) and hydrothermal (HTT) times of parametric and non-parametric models at each validation location. Numbers in bold show the lowest RMSE value in each location and season.
WeedExperimentParametric
Model
Non-Parametric
Model
TTHTTTTHTT
P. brachystachysETSIA-2018/1915.316.315.717.1
Sevilla Garden-2016/1728.728.527.729.5
Sevilla Garden-2018/1912.512.211.312.2
Average18.919.018.219.6
P. minorETSIA-2018/199.28.89.67.6
Sevilla Garden-2016/1731.930.532.731.0
Sevilla Garden-2017/1812.711.214.312.3
Sevilla Garden-2018/197.55.87.65.9
Average15.414.116.114.2
P. paradoxaBurgos-2019/2010.66.816.94.2
ETSIA-2018/1912.311.619.711.2
ETSIA-2019/209.912.48.512.1
Guadalcazar-2019/2014.817.711.715.8
Huesca-2019/2011.19.317.08.5
Sevilla Garden-2017/1818.012.925.713.6
Tomejil-2019/2016.49.59.212.9
Valladolid-2019/2014.68.620.26.3
Average13.511.116.110.6
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Sousa-Ortega, C.; Royo-Esnal, A.; Urbano, J.M. Predicting Seedling Emergence of Three Canarygrass (Phalaris) Species under Semi-Arid Conditions Using Parametric and Non-Parametric Models. Agronomy 2021, 11, 893. https://doi.org/10.3390/agronomy11050893

AMA Style

Sousa-Ortega C, Royo-Esnal A, Urbano JM. Predicting Seedling Emergence of Three Canarygrass (Phalaris) Species under Semi-Arid Conditions Using Parametric and Non-Parametric Models. Agronomy. 2021; 11(5):893. https://doi.org/10.3390/agronomy11050893

Chicago/Turabian Style

Sousa-Ortega, Carlos, Aritz Royo-Esnal, and José María Urbano. 2021. "Predicting Seedling Emergence of Three Canarygrass (Phalaris) Species under Semi-Arid Conditions Using Parametric and Non-Parametric Models" Agronomy 11, no. 5: 893. https://doi.org/10.3390/agronomy11050893

APA Style

Sousa-Ortega, C., Royo-Esnal, A., & Urbano, J. M. (2021). Predicting Seedling Emergence of Three Canarygrass (Phalaris) Species under Semi-Arid Conditions Using Parametric and Non-Parametric Models. Agronomy, 11(5), 893. https://doi.org/10.3390/agronomy11050893

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop