Challenges for Simulating Growth and Phenology of Silage Maize in a Nordic Climate with APSIM

Round 1

Reviewer 1 Report

In this manuscript, the authors try to calibrate and validate the biomass and phenology in 6 different maize silage cultivars using the APSIM next generation. Although, as I mentioned in the manuscript I see value in this study and exploring the performance of the model in high latitudes, I think the methodology needs to be improved. Specifically I was hoping the authors would split their two years of data for one year calibration and one year validation. There are other parameters also in the model like RUE which could have been involved in the initial calibration instead of conducting a separate exercise in the discussion section.  

Comments for author File: Comments.pdf

Author Response

Dear reviewer, thank you for your work and constructive suggestions and comments. Please find attached our replies.

We acknowledge that our choices of (i) not performing a complete calibration and validation procedure and (ii) not including RUE as a calibration parameter are arguable, and hope that the arguments detailed in the attached document will make sense to you.

Best regards,

Julien, on behalf of all the authors

Please find here below our replies. Apart from the points that you raised, we also performed some minor changes in the manuscript:

Line 52: “Predict”, which infers a model is involved, has been replaced with “estimate”.

Table 4: A definition of the ADMY acronym has been added in the caption.

Table 5: The caption has been modified: “Optimal values of the phenology-related cultivars parameters”.

“Interception efficiency” has been clarified to “light interception efficiency” throughout the manuscript.

Replies to comments

Line 24: Please define the acronyms before using them

Accepted: The “BBCH” acronym has been removed from the abstract and further defined in the material and methods (line 138, “Phenological stage of each treatment was recorded between emergence and anthesis using the “Biologische Bundesanstalt, Bundessortenamt und CHemische Industrie” (BBCH) scale [42]…”).

Line 71: What would be more interesting to know about this study and similar studies in your literature review is what were their findings in regards to your research question in this study? For example, how did their model do in terms of predicting phenology in higher latitudes compared to reporting on change in risk of N leaching.

The paper of the study that used the FASSET model cited in our manuscript (reference 25) gave no information about their results of calibration for phenology-related parameters, and MAISPROQ does not simulate phenology at all. To the best of our knowledge, there is no other paper available that describes the performances of a crop model for maize simulations at high latitude: as our research question has not been previously addressed, our aim here is to describe what work has been done in the surrounding subject area.

Line 89: delete “phenomenon”

Accepted: “Phenomenon” has been replaced with “factors”. Changes have been made consistently throughout the manuscript.

Line 109: Although I think there is value in this study and trying to understand how does the APSIM model do in high latitudes, I don't know why the process of accumulating GDD for modelling phenology would be an insufficient approach for high latitudes.

Accepted: The paper does not only address the question of modelling phenology but it also models biomass production (‘growth’). For phenology, the issue is that, in addition to GDD, other factors such as e.g. vernalisation and daylength could affect the development of maize. At high latitudes, the effect of daylength could be particularly important. Changes have been made in the text to make this clearer (line 20, “APSIM Next Generation was used to simulate the phenological development and biomass production of silage maize…”, line 114, “…day length, low temperatures or the solar zenith angle may affect the model’s capability to correctly simulate growth and phenology…”).

Line 193: I was really expecting to see RUE (Radiation use efficiency) also being calibrated given the latitude of your sites and cultivars you used. Is there any reason you didn't calibrate RUE?

Although RUE is a very important parameter for controlling the biomass production, we did not include it into the calibration procedure. The first reason for this is that in APSIM, the biomass production is mainly driven by RUE and leaf area index (LAI), and as we do not have any field measurements of LAI to control the simulated LAI, part of the error in biomass estimation could be the result of an underestimation of LAI, not of the RUE. The second reason is that it would have been necessary to almost double RUE values to match simulations of biomass with field measurements. Such RUE values would have fallen into an unrealistic range, so we preferred to flag this as a research question rather than proposing an unrealistic value for the RUE.

Line 194: Please add the range to the Table2 for each parameter.

Accepted: The range of values used for the calibration of the phenology parameters have been specified. However, we decided to display them in a new table (Table 3) to make them easier to read. The “Phyllochron” variable has been further specified into the number of leaf tips appeared and the corresponding leaf rate appearance (see table 2 and 3 in the new manuscript).

Line 198: Does this mean that you validated/evaluated your model performance on the same year/site that you calibrated your model? If yes why didn't you use one year for calibration and one year for validation?

We initially considered to use a calibration/validation approach, but eventually decided to use the whole dataset for calibration. The main reason is that we wanted to perform a calibration that would include as much as variability in initial conditions as possible so that the calibrated parameters could be used or further studies and sites. Another reason is that, as eventually the calibration results were not satisfactory yet, we did not see the point for keeping a separated dataset for validating the calibrated parameters. This has been clarified in the new version of the manuscript (line 188, “As the aim of this work is to be further assessed with other studies, the calibration was performed on the complete available dataset, with no validation step.”).

Line 201: It'd be interesting to see heatmaps of RMSE for combination of parameters maybe in the supplementary materials.

Declined: We agree that showing heatmaps would be an interesting display of the performances of calibration, however we are concerned that doing this for all combinations of parameters (par. 1 vs par. 2, par. 1 vs par. 3, etc.) would be uneasy and result in too much of graphical information.

Line 209: I would like to also encourage you to calculate the NRMSE (RMSE/average observed), which gives more context.

Accepted: NRMSE has been added in the table 4 of the new manuscript and in the main text (line 238, “… and a normalised root mean square error NRMSE = 0.23…” and line 270, “… the accuracy is low (RMSE = 5.2 t.ha^-1 and NRMSE = 0.41)…”) The equation of calculation of NRMSE has also been added in the main text (line 213)

Line 214: Please move this section to materials and methods.

Declined: We consider that presenting the weather data in the result section is a common practice in crop modelling studies, so we would prefer to keep this section here. However, if this is very important for the reviewer and the editor, we are willing to move this section to material and methods.

Line 217: I'm confused by what you mean by global? Could you please elaborate? To me it implies that it's not site specific? is it?

Accepted: “Global” radiation refers to the fact that all radiation are considered here (direct and diffused radiation). Changes have been made in the text to clarify this point (line 221, “Global (i.e., direct and diffuse) radiation data did not exhibit…”).

Figure 3: These are pretty good plots but please make a higher resolution of them.

Accepted: Figures with higher resolutions have been created. Apart from them, vector (pdf) version of the figures have also been created and have been uploaded on MDPI’s webpage. Note that the data displayed for Färjestaden were incorrect due to an error in the script used to create the figure, which has been corrected in the new version of the figure 3. This error did not affect the other computations that were performed in separate R scripts.

Figure 4: Please change this plot to have the observed in the X axis rather than Y.

Declined: Although we acknowledge that this is an arguable point, we would prefer to keep the current version of the figure (observations on the Y-axis), as recommended by Piñeiro et al: https://www.sciencedirect.com/science/article/pii/S0304380008002305. However, if this point is important to the reviewer and the editor, we are willing to perform the changes.

Line 235: 8.8 days?

The unit of RMSE related to phenology is expressed as BBCH unit. This has been clarified in the main text (line 240, “… a root mean square error RMSE = 8.8 phenological units…”).

Figure 5: The same here, please remove the last few days with zero values.

Accepted: New figures have been created where the last days with 0 values have been removed.

Table 4 (original manuscript): Does this mean that all the combination of PCx and PCy for example for the first row resulted in RMSE of 9.0?

Phyllochron x and y include several values to account for changes in required degree days to have a new leaf tip appearing as a function of the number of leaves already appeared. The combination of values therefore describes the changes over time of the phyllochron. The RMSE value of the table describes the performance of the model for the values of all the parameters described in the table. The text has been modified in the caption of the table (“As PC x and PC y change depending on the number of leaf tips already appeared, values listed in this table indicate the dynamic relationship between both variables”).

Line 288: Add the uncertainty around your observed data to your plots.

Declined: We created a new version of the figure that included uncertainty lines, but it resulted in a loss of clarity of the figure (see below).

Line 358: This could be moved to materials and methods where you described the APSIM model.

Declined: We think that including this equation in the discussion section makes the manuscript easier to read, as it is related to Eq. 5 (which has become Eq. 4 in the new version of the manuscript), and also because it is used for discussing the potential reasons why APSIM underestimates the biomass. However, if this point is important to the reviewer and editor, we are willing to include it into the material and method section.

Line 383: You could also check to see if the model estimated some sort of water/nitrogen stress resulting in this underestimation.

Potential stresses affecting the biomass production have been controlled, and it appeared that there was no effect of low nitrogen, CO₂, vapor pressure deficit or water levels on RUE. However, a stress related to temperature was reported for all sites, resulting in a simulated yield loss ranging between 15 and 4%. This point has been assessed in the manuscript (line 346).

Figure 7: You may want to consider plotting this maybe in a different format. It's not as easy to disgusting between the dash lines. 1:1 plot with different symbols for different scenarios maybe? Remove the zeros at the end of time series please. Also did it do any different/better in 2013?

Accepted: A new version of the figure, easier to read, and with zeros removed at the end of the time series, has been included. The results obtained for the same site and cultivar for year 2013 exhibited a similar trend.

Reviewer 2 Report

The paper is well written and shows the limitations for using APSIM for high latitude sides.  It is very important to show such shortcomings of models to avoid extensive usage of models without testing, and also for highlighting were model improvements are needed.

I have only minor comments which are provided in the pdf file

Comments for author File: Comments.pdf

Author Response

Dear reviewer, thank you for your work and interesting suggestions and comments. Please find attached our replies.

Best regards,

Julien, on behalf of all the co authors.

Dear reviewers, thank you for your work and valuable comments. Please find here below our replies. Apart from the points that you raised, we also performed some minor changes in the manuscript:

--------------------------------------------------------------------------------------------------------------------------

Line 52: “Predict”, which infers a model is involved, has been replaced with “estimate”.

Table 4: A definition of the ADMY acronym has been added in the caption.

Table 5: The caption has been modified: “Optimal values of the phenology-related cultivars parameters”.

“Interception efficiency” has been clarified to “light interception efficiency” throughout the manuscript.

Replies to comments

Line 54: but APSIM doesn't simulate maize quality.

Indeed, APSIM does not simulate the quality of silage maize. This work is actually the first step toward the development of a new APSIM module that would simulate the quality of silage maize.

Line 149: was there no warming up period used in the simulations?

Using a warming up period approach could have been interesting to provide relevant values for the initial conditions of simulations, particularly for the soil carbon fractions. However, we were limited to do so as we did not have more than 2 years of history of the fields. Nevertheless, our assumptions regarding the filling of the soil water profile were supported by literature (Eckersten, H.; Herrmann, A.; Kornher, A.; Halling, M.; Sindhøj, E.; Lewan, E. Predicting silage maize yield 448 and quality in Sweden as influenced by climate change and variability. Acta Agric. Scand. Sect. B - Soil 449 Plant Sci. 2012, 62, 151–165.) and soil samplings.

Line 183: this has been mentioned before. Please combine.

Accepted: The first statement (line 149 of original manuscript, “A full soil water profile was assumed at sowing due to the snow melt in spring.”) has been removed according to your suggestion.

Line 194: ranges investigated need to be provided, with justification.

Accepted: The ranges have been added in the table 3 of the new version of the manuscript. A justification for the choice of these values has also been specified in the text (line 198 of the new manuscript, “… potential values of the parameters were defined within a realistic range (see Table 3) based on the existing values provided for the early cultivars of the APSIM catalogue …”)

Line 196: how were these selected? I assume it was not a full factorial? Monte Carlo?

Accepted: As the number of combinations remained fairly “low”, we decided to use a full factorial approach and run APSIM for every combination of parameters. This has been clarified in the manuscript (line 199, “…and combined to create a full factorial simulation…”).

Line 198: more detail needed re the set up of the simulations, was this done separately for the 2 years or continuous.

Accepted: Simulations were set up separately for the two years. This has been clarified in the new version of the manuscript (line 176, “The model was parameterized for each location and each season according to the measured environmental and agronomic data (Table 1)”).

Line 204: was photosynthesis measured? or are you meaning biomass?

Accepted: Photosynthesis actually refers here to biomass production. The term has been changed in the manuscript (line 209, “… the model to simulate the phenology and biomass production of…”).

Line 211: equations 2 to 4 could be combined, no need to separate these.

Accepted: Equations have been combined into a single one.

Line 218: delete the s.

Accepted: The correction has been applied.

Line 270: could this also be due to the partitioning between above and below ground biomass?

The partitioning of dry matter between above and below ground biomass could partly explain the underestimation of the above ground dry matter by APSIM, but this influence should be limited in regard to the gap between observations and simulations of aboveground dry matter yield.

Line 387: need to use different lines very hard to distinguish the tow black dashed lines.

Accepted: The figure has been re designed.

Round 2

Reviewer 1 Report

Authors in this revision tried to address most of comments made on the manuscript and this has resulted in a major improvement in the quality of the work. However, I'm still not convinced that presenting just the result of a model calibration without any validation can reflect the true potential and/or drawback of the APSIM model.

Author Response

Dear reviewer,
Please find enclosed our reply to your concern regarding the absence of validation in our previous manuscript.

Author Response File: Author Response.pdf