Comment on Krüger, L. Decreasing Trends of Chinstrap Penguin Breeding Colonies in a Region of Major and Ongoing Rapid Environmental Changes Suggest Population Level Vulnerability. Diversity 2023, 15, 327

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

 

 

I have completed my review for ‘Comment on Krüger (2023): Decreasing Trends of Chinstrap Penguin Breeding Colonies in a Region of Major and Ongoing Rapid Environmental Changes Suggest Population Level Vulnerability. Diversity 2023, 15, 327’ is currently under consideration for publication in Diversity. The authors critically reassessed the data analyses and modeling approach used by Krüger. They offered alternatives to improve future population trend analyses, given that statistical modeling approaches (mixed models) are more complex than expected. This comment is very insightful and offers valuable information for the population trend analysis.

 

 

Author Response

Comments and Suggestions for Authors: 

I have completed my review for ‘Comment on Krüger (2023): Decreasing Trends of Chinstrap Penguin Breeding Colonies in a Region of Major and Ongoing Rapid Environmental Changes Suggest Population Level Vulnerability. Diversity 2023, 15, 327’ currently under consideration for publication in Diversity. The authors critically reassessed the data analyses and modeling approach used by Krüger. They offered alternatives to improve future population trend analyses, given that statistical modeling approaches (mixed models) are more complex than expected. This comment is very insightful and offers valuable information for the population trend analysis.

Authors response: No changes required.

Reviewer 2 Report

Comments and Suggestions for Authors

Is there not a specific MDPI template that needs to be followed for journal submissions to MDPI journals? 

Also, I don't believe that the authors have used the correct MDPI citation format either. 

I think the authors need to be clear in the abstract what challenges they do address and which ones they do not. 

Please can the abstract have a conclusion that identifies the way forward or acknowledges the beneficial information in the original paper and how this reply adds to our knowledge of penguin population dynamics?

I think the layout of the manuscript is a little unorthodox. Does the paper not fit the traditional structure of introduction, methods, results, discussion, conclusion? I.e., can you not comment on the limitations of the original manuscript's sections? 

You need a reference for long term data are lacking for most populations, and I recommend a species example or two (with references). 

Scientific names need to be in italics.

Line 56-57: I think you need to explain why you have these concerns early on, so the reader see your justification for the response being published.

Line 90: Whilst you give references here, you should explain why this is risky.

Line 104: Why challenging? Please explain. Don't just rely on the citation. 

Lines 128-133: Needs further explanation. You describe these potential areas but you don't explain what you would do about them or what they look like in the original paper.

Section 4: You compare the original models with your choice of new model. Please explain why your models are an improvement.

Line 197: Don't all mixed effects models come with random factors included?

Line 210-212: I think you need to explain this better. What do you mean by propagate?

Line 215-216: Why does it?

Line 270-271: The context is useful here. So I think you need to be clearer in the start of this paragraph at what you are setting out to do. That data selection and different model parameters can lead to uncertainty in population estimates. Are you saying that the original paper may still have value?

Your conclusion is discursive and it should really be a summary of what you have found out and what your important take home message is. Your discursive elements, and the new citations, should be placed in the main body of your work and thoroughly evaluated.

How do these data help with conservation action? And what is your estimate of the chinstrap penguin population based on your re-analysis or re-assessment of these data?

I think you need to add to your discussion or conclusion that given that the original paper you are responding to would have been peer-reviewed why are you confident in your assessment of the paper, and that your interpretation is correct, compared to those that reviewed the manuscript originally?

I also think that your introduction needs to acknowledge some of the value or usefulness of the original paper and put into context the specific areas that could be improved by your new attempt.

Comments on the Quality of English Language

The overall quality of written English is good. Some longer sentences could be made more succinct and clear. Otherwise, minor edits required. Please follow correct MDPI citation format.

Author Response

Comments and Suggestions for Authors: 

Comment 1: Is there not a specific MDPI template that needs to be followed for journal submissions to MDPI journals? I think the layout of the manuscript is a little unorthodox. Does the paper not fit the traditional structure of introduction, methods, results, discussion, conclusion? I.e., can you not comment on the limitations of the original manuscript's sections? Also, I don't believe that the authors have used the correct MDPI citation format either.

Authors' response: Our manuscript is a Comment rather than a research paper. We believe the manuscript follows a logical and coherent structure even though it does not include traditional sections like Methods and Results. In our view it would be difficult to structure the Comment in that format. Other published Comments (see for example Diversity 2022, 14, 200. https://doi.org/10.3390/d14030200) also do not follow “traditional” sections.

 

Comment 2: I think the authors need to be clear in the abstract what challenges they do address and which ones they do not. Please can the abstract have a conclusion that identifies the way forward or acknowledges the beneficial information in the original paper and how this reply adds to our knowledge of penguin population dynamics?

Authors' response: The exact shortcomings are difficult to summarise concisely in the Abstract, but we made the following change to be clearer. Instead of writing “We discuss these shortcomings…” we changed the text to “We discuss oversights in several key steps of that paper’s analysis (including data processing, exploratory data analysis, model fitting, model evaluation and prediction) to help others….”

We deleted the reference to “While we do not address all challenges”. This refers to not including uncertainty of counts in the analysis, a relatively minor point we refer to later in the paper.

The last request is that the abstract have a conclusion that either (1) identifies the way forward or (2) acknowledges the beneficial information in the original paper or (3) explains how this reply adds to our knowledge of penguin population dynamics. We address this more broadly in the last sentence of the Abstract. Our methodological explanations (the criticisms and solutions offered in the paper) are not only relevant for penguin population dynamics, but also for other animal populations: “This case study highlights (1) the profound influence that seemingly minor differences in modelling procedures (both unintentional errors and other decisions) can have on predictions of population trends, and (2) the substantial inherent uncertainty in population trend predictions derived from sparse, heterogenous data.”

 

Comment 3: You need a reference for long term data are lacking for most populations, and I recommend a species example or two (with references). 

Authors' response: We added 2 references. Paleczny et al. 2015 assessed the population trend of the world’s monitored seabirds (1950–2010) by compiling a global database of seabird population size records – they estimated that it represented approximately 19% of the global seabird population. White (2019) deals with the problem of short time-series in the context of IUCN assessments. We do not elaborate with examples as the Comment is supposed to focus specifically on the paper it comments on.

 

Comment 4: Line 56-57: I think you need to explain why you have these concerns early on, so the reader see your justification for the response being published.

Authors' response: The comment refers to this sentence in the Introduction: “However, we caution that Krüger (2023)’s statistical analyses (intended to form the foundation for drawing valid, evidence-based inferences from sparse data) contain fundamental errors, and that these oversights generate unreliable population trend predictions that invalidate that paper's findings.

The next two sentences explain that “we revisit the main aim of that paper” and (in line 61) we stated “We identify and discuss shortcomings and unintentional errors in several key steps of Krüger (2023)’s analysis, including data processing, exploratory data analysis, model fitting, model evaluation and prediction (Table 1)”. Therefore, we believe our concerns are already given “early on” and summarized in Table 1 so that readers can get an immediate overview of our main concerns. No changes were made.  

 

Comment 5: Line 90: Whilst you give references here, you should explain why this is risky.

Authors' response: The comment refers to this sentence: “It can be risky, in general, to diagnose multi-year trends based on a comparison of counts in two years, especially when counts are uncertain (e.g., Hill et al. 2019, Supplementary text 2)”.

At the end of the sentence, we refer readers to the Supplementary material of our contribution, where we explain and illustrate with an example why this is risky. No change made.  

 

Comment 6: Line 104: Why challenging? Please explain. Don't just rely on the citation. 

Authors' response: The comment refers to this sentence: “Fitting Poisson GLMMs is a useful and common approach to model counts of animals, but the correct application of these models can be challenging (e.g., Zuur et al. 2009, Chapter 13).”

We amended to specify that “the correct application of these models can be challenging <<due to the inclusion of random effects>>”.

 

Comment 7: Lines 128-133: Needs further explanation. You describe these potential areas but you don't explain what you would do about them or what they look like in the original paper.

Authors' response: The comment refers to this section: “The second and more serious issue is lack of model fit. The Krüger (2023) model yields extremely uncertain and biased estimates of the predicted abundance against observed values (Figure 2, Figure 3). This simple model checking procedure shows that model predictions cannot support downstream inferences about long-term changes in chinstrap penguin abundance. The lack of model fit arises because the model does not allow the sites’ population trajectories to vary (see below) (Supplementary code 2).”

We refer to Figure 2 and Figure 3, which shows the lack of model fit of the Krüger (2023) model applied to simulated data (Figure 2) and according to the original paper (Figure 3). We explain, in detail, what should be done in the next section “Modelling penguin population trends with GLMMs: a reanalysis”. No changes were made.

 

Comment 8: Section 4: You compare the original models with your choice of new model. Please explain why your models are an improvement.

Authors' response: We explained in Section 3 (“Modelling penguin population trends with GLMMs: a statistical critique”) why the original model Krüger (2023) is incorrect. Section 4 explains, in detail, how the correct model is constructed. Figure 2 and Figure 3 clearly shows that predictions based on the original model (Krüger 2023) do not agree with the observed data, in contrast to our model’s predictions. No changes were made.

 

Comment 9: Line 197: Don't all mixed effects models come with random factors included?

Authors' response: The comment refers to this sentence: “With mixed models, we must decide whether to include information about the random effects in the predictions”.

No, mixed models are fitted with random effects, but there are several choices to make about the random effects when it comes to predicting from mixed models (the emphasis is on the “predictions” part). For example, by including or excluding the random effects we can calculate a global grand mean, conditional effects for existing sites, or conditional effects for a new sites. This is one of the important misunderstandings that we are trying to address in this section (“Predicting penguin population trends with GLMMs”).

 

Comment 10: Line 210-212: I think you need to explain this better. What do you mean by propagate?

Authors' response: Propagation of uncertainty is widely appreciated in statistics – for example, the phrase "you should not do statistics on statistics" generally refers to the idea that applying statistical analyses to data that have already been aggregated, summarized, or statistically manipulated can lead to misleading or incorrect results, because the error (uncertainty) is not propagated by the summary statistics.

We amended the wording to specify specifically that we are propagating model and prediction uncertainty <<through to the estimates of population change>>.

 

Comment 11: Line 215-216: Why does it?

Authors' response: The comment refers to this sentence: “The posterior mean discards the uncertainty in the posterior distribution, and this leads to overconfident predictions”. We made no change. As the comment above explained, aggregated or summarized estimates do not capture the uncertainty that surrounds the estimates. When uncertainty is ignored, conclusions will be overconfident. 

 

Comment 12: Line 270-271: The context is useful here. So I think you need to be clearer in the start of this paragraph at what you are setting out to do. That data selection and different model parameters can lead to uncertainty in population estimates. Are you saying that the original paper may still have value?

Authors' response: The comment refers to this sentence: “We did not attempt to replicate Krüger (2023)’s approach of estimating population change.” We amended the sentence to: We did not attempt to replicate Krüger (2023)’s approach of estimating population change <<as that approach did not propagate model and prediction uncertainty>>.”  We also moved this sentence to the previous section where, indeed, the sentence can be interpreted in the correct context. We appreciate that the context may not have been clear in the original text. The previous section dealt with “Predicting penguin population trends with GLMMs” and this sentence explains why it is not advised to follow the approach implemented in Krüger (2023). We also moved the last sentence of the paragraph to the start of the paragraph, as that seems to help summarize the content of the paragraph (at the start of the paragraph, rather than at the end).

 

Comment 13: Your conclusion is discursive and it should really be a summary of what you have found out and what your important take home message is. Your discursive elements, and the new citations, should be placed in the main body of your work and thoroughly evaluated.

Authors' response: We made several changes to the Conclusion to improve the focus. The Conclusion section has two paragraphs. The first paragraph reviews what was achieved and how this differs from Krüger (2023). The second paragraph looks forward: it comments on how this data (and complementary data) can aid conservation action (as requested by the reviewer’s Comment 14). The conclusion now reads as follows:

Historical data on chinstrap penguin breeding population sizes are sparse and sometimes highly uncertain, making it hard to estimate true population trajectories. Krüger’s study [8] attempted to summarize the decline of chinstrap penguin populations – an important topic in the context of conservation management in the Southern Ocean – and the author’s ultimate conclusion about population vulnerability may even be perfectly correct. Unfortunately, a series of unintentional analytic errors undermine the validity of the findings.. We show through reanalysis that improved statistical modelling can yield better predictions of chinstrap penguin population trends, at least within the range of observed data. Mixed model analyses are intricate, but good statistical protocols can help expose pitfalls and prevent incorrect model-based inference [23]. Ultimately, appropriate statistical models are required for evidence-based conclusions, and the assumptions and fit of every model must be checked (e.g., by comparing the model predictions against the observed data) before conclusions can be drawn [24].

 

Prediction uncertainty increases substantially as we move further from the observed data, even when models are correctly specified. Extrapolation and interpolation of chinstrap penguin population trends are difficult to avoid in the absence of systematic surveys, and it is important to incorporate prediction uncertainty when estimating population change. While historical population trends of chinstrap penguins will remain difficult to estimate, we are more optimistic about obtaining better inferences of contemporary trends. This optimism is due to recent increases in sampling (count data available in MAPPPD) and the potential for more accurate and precise penguin colony counts in the future (e.g., through remotely piloted aircraft;[25]). Beyond monitoring trends, there is a real need to understand the drivers of population change in chinstrap penguins. Though labour intensive, individual-based capture-recapture data [26] and integrated population model analysis [27] can identify the demographic parameters (e.g., reproduction, survival, dispersal) and external factors (e.g., environmental and fisheries related variables) that drive population change. Collecting more data may be a crucial step toward a deeper understanding of the magnitude and underlying causes of population changes in chinstrap penguins. However, robust data analysis will be essential to draw meaningful conclusions that can enhance conservation and management effectiveness in the Antarctic Peninsula.

 

Comment 14: How do these data help with conservation action? And what is your estimate of the chinstrap penguin population based on your re-analysis or re-assessment of these data?

Authors' response: We address this comment in the final paragraph of the Conclusion. We also make the point that ‘data alone’ does not help conservation action. We need appropriate and effective (i.e., ‘correct’) analysis of the data to enhance conservation and management effectiveness. We gave our estimates of chinstrap penguin population declines in the section titled “How sparse is too sparse?” For example, we stated that: “Our reanalysis found that there was a 59% (dataset 1), 43% (dataset 2) or 88% (dataset 3) probability that the aggregate abundance of the chinstrap penguin colonies included in each dataset decreased by at least 30% between 1990 and 2019. For dataset 3, the 90% posterior credible interval for the change in abundance from 1990 to 2019 was a decrease of 26 to 49% (Figure 5), but no clear trend between colony declines and latitude was observed (Figure 4).”

 

Comment 15: I think you need to add to your discussion or conclusion that given that the original paper you are responding to would have been peer-reviewed why are you confident in your assessment of the paper, and that your interpretation is correct, compared to those that reviewed the manuscript originally?

Authors' response: We don’t think we should discuss the quality of the previous peer review in our Comment. But it is widely recognised that in many cases “…peer review has done little to identify failures of scientific rigor (e.g., improper statistics…” (see Proctor et al. 2023; https://doi.org/10.1128/mbio.03183-22). The original paper provided reproducible data and analysis scripts, which is to be applauded. This allowed us to assess the results from that paper directly (for example, simply by plotting the model predictions against observed data at the site level).

We also conducted our own simulation study using the original analysis code and our own analysis. Simulation studies are powerful to assess model performance, so we are confident in our results.

The reviewers of the original paper (Krüger 2023) apparently did not consider the methods (the reviewer reports available for Krüger (2023) and these do not suggest any considerations about the methods).

 

Comment 16: I also think that your introduction needs to acknowledge some of the value or usefulness of the original paper and put into context the specific areas that could be improved by your new attempt.

Authors' response:  The Introduction acknowledge some of the value or usefulness of the original paper. We stated that “Assessment of chinstrap population trends is an important research topic that can help inform policy decisions or conservation management plans within the Southern Ocean”. We also state the specific areas that should be improved (i.e., the analysis): “However, we caution that Krüger (2023)’s statistical analyses… contain fundamental errors” and “We identify and discuss shortcomings and unintentional errors in several key steps of Krüger (2023)’s analysis, including data processing, exploratory data analysis, model fitting, model evaluation and prediction (Table 1)”. No change made.

 

Comments on the Quality of English Language

The overall quality of written English is good. Some longer sentences could be made more succinct and clear. Otherwise, minor edits required. Please follow correct MDPI citation format.

Authors' response: We formatted the manuscript according to the journal’s citation format.

 

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

An improved version of the paper has been supplied. The article is still not submitted on the mdpi template but otherwise the edits clarify the approach and explain the reasons for this response article. I am happy for this paper to be published.

Comments on the Quality of English Language

Please check sentence structure on occasion for clarity and conciseness.