Prioritizing Stream Protection, Restoration and Management Actions Using Landscape Modeling and Spatial Analysis

Round 1

Reviewer 1 Report

Nice work, but you failed to add a social science component. If the social science piece is missing and watershed landowners don't buy into the physical, chemical and biological analysis you have performed - nothing gets done!

Author Response

Reviewer #1

Nice work, but you failed to add a social science component. If the social science piece is missing and watershed landowners don't buy into the physical, chemical and biological analysis you have performed - nothing gets done!

Thank you for the comment.   We partially included a social science component by accounting for environmental justice in the prioritization process (see Section 2.4.2). However, we agree that the broader social science considerations of costs, benefits, and acceptance are beyond the scope of this project.   We have added text to the Discussion section acknowledging that “Consideration of the broader social benefits of restoration and management were beyond the scope of this study but are an important area for future investigation.”

Reviewer 2 Report

Comments for author File: Comments.pdf

Author Response

Reviewer #2

In this paper, the authors use geospatial, census, and other tabular data in a random forest classification to determine the status of stream reaches in California, thereby providing useful information to be used for prioritizing limited resources for reclamation, management, and protection. I found this paper to be very well written and researched, however I have a few comments below related to the methods and the formatting of the paper.

Overall:

Use the same color legend across all figures for the “recommended actions.” There appears to be 3 different colorations of the recommended actions: Figure 4, Figure 6, and all others.

The color scheme of the recommended actions within the figures has been standardized across all figures.

By Line:

Line 44: suggest rewording, “watersheds benefits” sounds a bit weird, perhaps try “Watershed protection benefits….”

We have made this change

Line 61: Need to define Hydrologic Unit Codes, and how they work, for international readers.

We have revised the text to better the define the term HUC – “... produce results at the scale of distinct hydrologic units defined by topography and hydrology (often termed HUC-12 in the United States)”

Line 132: change to “an appropriate unit of analysis” instead of “and”

We have made this change

Figure 1: text is lo-res and jaggy.

We have increased the resolution of this figure and revised the figure legend to improve clarity

Line 185: I would explain Random Forests a bit more, you’ve reduced an entire process that is critical to this paper down to a single sentence. I would follow up with pros/cons of RF in the discussion (e.g. black box nature of RF)

We appreciate this suggestion.  We have added the following explanation, “A random forest model is machine learning algorithm that build a collection of decision trees to predict an output based on a series of explanatory input variables in order to achieve output consensus across these trees while avoiding model overfitting. Random forest models are commonly used to predict attributes across broad spatial scales using relationships derived from existing data. When developed with sufficient data density they are an efficient way to develop predictions for unmonitored locations across a land-scape. The downside of random forest modeling is that their accuracy may be lower than methods such as boosted regression analysis and they are frequently accused of being “black box” methods because they produce predictions with minimal model configuration. In this instance, random forest is an appropriate approach for determining stream condition across the entire state of California and is reliable given that the models are based on data from a statewide monitoring program with broad spatial coverage.”

Line 208: On line 191, you use 1/3 of the dataset for training, while here you are doing 75:25 split. Any reason why you are using different splits for different methods?

We believe the reviewer misread the paragraph.  The numbers listed on line 191 were the number of stream reaches used for the CRAM and CSCI/ASCI analysis, respectively.  They do not represent the split between training and testing datasets.  The 75:25 split described on line 216 is correct.

Line 234: spell out reference to #22 when used in this context, “site data from Fong et al. [22].”

We have made this change

Line 257: databases used in the study should have numbered citations

Where they are available, we have provided citations for databases used in the analysis.  There is no formal citation for the SMC database listed on line 268, as it is compiled from a regional stream condition assessment program.  We have provided the url for the publicly available database, which includes all relevant metadata and documentation.

Line 299: place period inside quotes

We have made this change

Table 3: It would be best if the table (especially such a short one) were all on one page.

We agree with this suggestion and request that the table be consolidated on a single page during the final typesetting process.

Line 398: I think a weighted average based upon the percentage of the stream reach within each census track would be best here. I can foresee instances, especially on a statewide or national scale, where you may only have a small fraction of the total stream reach represented b/c the largest track was used, though it may only be a small fraction of the total.

We evaluated the use of a weighted average, and it did not fundamentally change the results.  This is because the prioritization based on the distribution of pollution burden scores independent of the drainage density.  We feel this is appropriate because it allows prioritization of areas that are subject to high pollution burden, irrespective of the number of streams located in those areas.

Figure 2: I would split the figure and table into separate entities. I would also add an inset or graticles with coordinates to show readers where this area is.

We agree with this suggestion and have deleted the table and instead provided a description of the watershed characteristics in the text.  An inset of the study area in context with the State of California has also been added.

Line 444-447: % impervious, urban, and road density are all very similar (or at least strongly correlated) metrics that are probably redundant.

We used recursive feature elimination in the RF modeling to deal with multiple collinearity.  Therefore, although there are correlated variables in the dataset, it should not affect model performance.

Table 5: I would suggest shortening the white space after “Stressor Description,” as the Stressor column goes past the printable portion of the page, and gets partially cut off when printed.

We have made this change and will work with the editors to improve the final formatting during the typsetting process.

Line 535: insert comma after “areas”

We have made this change

Figure 5: The X coordinates on the submaps are illegible. I would suggest lowering the number of graticles on the X axis, this problem is evident in the Supplementary material as well.

The longitude has been revised to be legible in the figures within the body of the document and Appendix 3.

Line 728: insert comma after “areas”

We have made this change

Line 778: side note, livestock stocking information is made available every 5 years from the USDA Census of Agriculture, groundwater data should be readily available from the state, as I would assume that California, like many other states have monitoring wells. I am not suggesting that you use this information, but it may be pertinent to mention these possible data sources, should someone want to try to reproduce/refine this process in other states, or nationally.

We appreciate this comment and have added a notation to investigate other sources of stressor data.

Line 781: I assume you mean “extraction” instead of “abstraction”

Yes, thank you.  We have made this change

Line 916: I think MDPI typically uses initials in author contributions, but double check that.

Thank you for pointing this out. I was under the same impression, but the online template suggested that full names be used.  We will verify this with the editorial staff prior to publication.

Line 939: a few of the references have DOIs, please add the rest (where applicable) Supplementary material:

We have added DOIs where they are available

Most of the figures are low-resolution, I would suggest re-exporting them to a higher point density to see if that helps

We have exported higher density figures in jpg format.  We can supply higher resolution TIFF or PNG files for final production

Appendix 2: San Diego River Watershed Mgmt Area Water Quality Improvement Plan is not hyperlinked

Thank you for calling this to our attention.  The hyperlink has been restored.

Appendix 3: I would recommended reducing the number of graticles on the X axis so that there is no overlapping text on the coordinate

The longitude has been revised to be legible in the Appendix 3 figures.

Reviewer 3 Report

Congratulations on a fine paper.  The need for prioritization tools to identify where scant resources should be spent is critical.  It also gets away from a hammer of a strategy looking for a nail of a place to implement it.  

I have a few comments and some minor suggestions:

The writing is very pithy, and is more understandable to a modeling expert than a practitioner in the field. That said, the supporting materials should satisfy the needs of a practitioner.  If the authors want to make the manuscript more accessible to practitioners, there will need to be some rewriting to adjust for that, or perhaps just keep the manuscript the way it is and have it focused more towards a modeling crowd to understand the benefits of the approach.  That is up to the authors.

Be clear for the reader that the indices are all relative, so that areas with no impact may have impact, but are in excellent condition comparatively to the others.

Where does connectivity get addressed, other than culverts - from dams and levee impacts for instance.  Is this in the landscape context evaluation?

Be more explicit about the ecoregions used (line 209) - state they were EPA freshwater ecoregions developed for water quality assessment, so the reader does not confuse them with Bailey or Abell ecoregions.

Actions at the reach level may not address impacts from sources at the catchment level (line 302).  You get at this by discussing table 2, where some actions are at the stream reach level, and some at the catchment level.  Might ad a sentence or two here connecting the catchment impacts to stream reaches and why even though the stream reaches are the unit that is defined by the models as needing a certain type of action, those actions will take place at appropriate scales to address sources of impact at appropriate scales.  

Where would things like environmental flows fit? It might serve the reader if Table 2 was explained as a very broad set of categories of action needs, not the actions themselves.  For instance, channel and floodplain restoration as a response to dam density does not describe the actual action, such as environmental flow management, dam removal, levee removal, etc.  Also, are levees incorporated into the analyses for connectivity at all?  I found table 2 to be quite unfulfilling in terms of actions.  For instance, soil erodibility on ag lands is addressed through revegetation buffers.  I am at a loss for why something like ag BMPs is not included as a larger catch all set of actions for that as well.  Actions such as irrigation water use efficiency are not included - perhaps because water use is not an attribute that was part of the analysis? This table does not seem prepared by folks that are very active in taking conservation actions for catchments and streams, and leaves a lot out.  I am not sure how to deal with this.  One major change suggestion I have is to change the header Specific Action to something like Action Category, because some of those are not very specific solutions to address the problem (e.g. channel and floodplain restoration to address dam density), while culvert retrofit is a very specific solution to road density.

When using the attributes of Table 3, why divide by 10, why not normalize from 0 - 1?

In table 5, can you remind the reader the difference between catchment and watershed so they do not need to go back into the text to get the distinctions?

Line 710 ...condition and stress data (ARE), not (IS).  Data are plural.

Do you suggest for practitioners to use all 4 models given the different strengths and weaknesses, or to chose one that they seem most comfortable with and understand those strengths and weaknesses upfront?  I would be a bit clearer for potential users about the approach to take.

In discussion about short comings of StreamCat, for areas where attributes not included in it (although there are over 600), can you provide recommendations on how to deal with the isue?

Given the high proportion of areas that are identified for protection, can you suggest how to prioritize among them?  You suggest monitoring, but that can be extensive and costly.  Nowhere in the text do you discuss future sources of threats, such as renewable energy expansion (including dams, solar, wind and associated infrastructure such as roads), ag expansion, water use increases, etc.  I suggest addressing this issue in the discussion as a short coming in the prioritization, and a next step for prioritizing among protection opportunities.

Given the discussion about incorporating finer-scale and local data in analyses, it might serve the reader to understand that the indices reflect the potential for a good outcome, which might decrease in quality with a finer-scale analyses.  The ratings will never get better, they can only get worse with more data exploration within any given situation - e.g. something rated as excellent might not be excellent after a finer scale evaluation, but those places not rated as excellent are not going to be rated higher, that is the nature of using these types of data, and a strength of the approach - it defines greatest potential.

Author Response

Reviewer #3

Congratulations on a fine paper.  The need for prioritization tools to identify where scant resources should be spent is critical.  It also gets away from a hammer of a strategy looking for a nail of a place to implement it.  

Thank you!  We share and appreciate the reviewer’s perspective

I have a few comments and some minor suggestions:

The writing is very pithy and is more understandable to a modeling expert than a practitioner in the field. That said, the supporting materials should satisfy the needs of a practitioner.  If the authors want to make the manuscript more accessible to practitioners, there will need to be some rewriting to adjust for that, or perhaps just keep the manuscript the way it is and have it focused more towards a modeling crowd to understand the benefits of the approach.  That is up to the authors.

Thank you for this perspective.  Our primary target audience is modelers/analysts who may be interested in developing analogue approaches in other locations or for other applications.  However, it is also important that the writing be accessible to managers and practitioners.  As part of the project, we coordinated with a statewide workgroup of agency practitioners who reviewed drafts of the manuscript and the online data products.  Their feedback regarding language and clarity is reflected in the draft manuscript.  However, it is always helpful to have a fresh perspective and we have gone back through the document again with an eye towards improving clarity and accessibility.

Be clear for the reader that the indices are all relative, so that areas with no impact may have impact, but are in excellent condition comparatively to the others.

We have added the following statement to the Materials and Methods section to clarify this point, “All indices assess condition relative to reference conditions, so a “good” score indicates proximity to reference condition vs. complete absence of stress”.

Where does connectivity get addressed, other than culverts - from dams and levee impacts for instance.  Is this in the landscape context evaluation?

Connectivity is only indirectly accounted for in the analysis through some of the StreamCat variables.  We recognize that connectivity issues are critical to watershed function, and we attempted to incorporate them into our analysis.  However, we were unable to develop an approach that could provide a consistently appropriate approach and be applied statewide in a fairly automated manner.  Consequently, connectivity issues are not directly addressed in our approach.  We acknowledge this shortcoming the Conclusions section by stating that, “In reality, holistic watershed management requires consideration of the processes and interactions across the watershed such as sediment transport and continuity for wildlife movement [41,42]. Advances in spatial modeling can be applied to future analysis to further prioritize sites for protection, restoration, or management based on their importance for maintaining or restoring corridors or linkages or aquatic or riparian organisms [43].”

Be more explicit about the ecoregions used (line 209) - state they were EPA freshwater ecoregions developed for water quality assessment, so the reader does not confuse them with Bailey or Abell ecoregions.

We stratified the analysis based on State of California ecoregions as stated in the text and cited in Citation #20.  The state of CA ecoregions are a derivative of the Omernik ecoregions, but have been adjusted to align with watershed boundaries within the state of California.  These ecoregions are the basis of stratification in the statewide monitoring and assessment programs and were used in our analysis to maintain consistency with the underlying datasets.

Actions at the reach level may not address impacts from sources at the catchment level (line 302).  You get at this by discussing table 2, where some actions are at the stream reach level, and some at the catchment level.  Might add a sentence or two here connecting the catchment impacts to stream reaches and why even though the stream reaches are the unit that is defined by the models as needing a certain type of action, those actions will take place at appropriate scales to address sources of impact at appropriate scales.  

Thank you for this suggestion.  We have added the following text to the Materials and Methods section that discusses management recommendations, “Management recommendations at the stream vs. catchments scale depend on the specific stressor, but reach scale recommendations consider the setting of the reach within the catchment to help ensure continuity in management actions”

Where would things like environmental flows fit? It might serve the reader if Table 2 was explained as a very broad set of categories of action needs, not the actions themselves.  For instance, channel and floodplain restoration as a response to dam density does not describe the actual action, such as environmental flow management, dam removal, levee removal, etc.  Also, are levees incorporated into the analyses for connectivity at all?  I found table 2 to be quite unfulfilling in terms of actions.  For instance, soil erodibility on ag lands is addressed through revegetation buffers.  I am at a loss for why something like ag BMPs is not included as a larger catch all set of actions for that as well.  Actions such as irrigation water use efficiency are not included - perhaps because water use is not an attribute that was part of the analysis? This table does not seem prepared by folks that are very active in taking conservation actions for catchments and streams and leaves a lot out.  I am not sure how to deal with this.  One major change suggestion I have is to change the header Specific Action to something like Action Category, because some of those are not very specific solutions to address the problem (e.g., channel and floodplain restoration to address dam density), while culvert retrofit is a very specific solution to road density.

You raise a good point.  Many of the stressor would affect runoff patterns, but the table does not directly address those issues, but provides recommendations that indirectly help mitigate the stressor.   We have revised the table to provide more specificity in the actions, as suggested, and to rename the column headings to clarify that the recommended actions are example action types that could be considered.

When using the attributes of Table 3, why divide by 10, why not normalize from 0 - 1?

There was not specific reason.  We could have normalized on a 0-1 scale. However, since we use the results in a relative sense to help prioritize locations, the results of the analysis would have been the same.

In table 5, can you remind the reader the difference between catchment and watershed so they do not need to go back into the text to get the distinctions?

We have language to the Table 5 legend reminding the reader that catchments typically encompass larger areas that capture and consolidate runoff vs. smaller watersheds which are defined by local topography.

Line 710 ...condition and stress data (ARE), not (IS).  Data are plural.

We have made this change

Do you suggest for practitioners to use all 4 models given the different strengths and weaknesses, or to chose one that they seem most comfortable with and understand those strengths and weaknesses upfront?  I would be a bit clearer for potential users about the approach to take.

We are not completely clear on what the reviewer is asking.  All the components of the watershed prioritization analysis are meant to be used in an integrated manner.  The four condition models are used together to assign an overall condition score.  The stress and prioritization models are then used to develop management recommendations.  

In discussion about short comings of StreamCat, for areas where attributes not included in it (although there are over 600), can you provide recommendations on how to deal with the issue?

We have added text to the Discussion section stating that once the initial screening analysis using this approach is completed, it is important for users to account for finer scale stress data before making final management decisions. This data may be available from datasets not represented in StreamCat, but from other regional or local sources.

Given the high proportion of areas that are identified for protection, can you suggest how to prioritize among them?  You suggest monitoring, but that can be extensive and costly.  Nowhere in the text do you discuss future sources of threats, such as renewable energy expansion (including dams, solar, wind and associated infrastructure such as roads), ag expansion, water use increases, etc.  I suggest addressing this issue in the discussion as a short coming in the prioritization, and a next step for prioritizing among protection opportunities.

This is an excellent point.  We have added the following text to the Discussion section, “However, it is important to keep in mind that all stream reaches are likely are already being affected by shifts in temperature and rainfall patterns associated with climate change. Moreover, future land use and resource management changes may pose additional threats. Therefore, every site has some degree of vulnerability and should be monitored for degradation from climate change, land use and resource management changes or a combination. Existing statewide monitoring programs could be used to periodically re-evaluate these “high quality” sites to determine their level of risk and whether their condition is declining.”

Given the discussion about incorporating finer-scale and local data in analyses, it might serve the reader to understand that the indices reflect the potential for a good outcome, which might decrease in quality with a finer-scale analyses.  The ratings will never get better, they can only get worse with more data exploration within any given situation - e.g., something rated as excellent might not be excellent after a finer scale evaluation, but those places not rated as excellent are not going to be rated higher, that is the nature of using these types of data, and a strength of the approach - it defines greatest potential.

We agree and have added the following statement to the Conclusion section, “Moreover, the outcomes likely represent a “best case scenario” of condition.  Holistic watershed management requires consideration of the processes and interactions across the watershed such as sediment transport and continuity for wildlife movement [41,42]. Such finer scale analysis often reveals additional stressors that affect condition, but also inform more directed management actions.”

Thank you for the thoughtful and insightful comments and suggestions.