Multispectral, Aerial Disease Detection for Myrtle Rust (Austropuccinia psidii) on a Lemon Myrtle Plantation

Round 1

Reviewer 1 Report

The goal of the manuscript is to detect myrtle rust on lemon myrtle trees using random forest classifier. The classification is performed on multispectral data acquired from a drone.

The topic is relevant for the community and the paper is well written and well structured. The literature review is comprehensive, the discussion is thorough and underlined by several references.  

However, I am still skeptical about the general experimental setup. As in my first review about the previous version of the manuscript, I still think that the goal stated by the authors “to explore whether it would be possible to spectrally discriminate healthy and infected lemon myrtle tree canopies on a plantation” is not reached. Since the class “shadow” is used as a separate class, no distinction can be made between treated and untreated plants within the shadow area, which is half of the presented dataset. From an application point of view, it is unrealistic to wait for sunny weather, and some areas may always lie in shadow regions. From a theoretical point of view, conclusions about relevant wavelengths and indices for this disease may be incomplete and biased. Therefore, I recommend an experiment without the shadow class before publication.

I highly recommend to use a proper mathematical writing style, that means variables should be written as actual variables and not as text, since it can be confused with multiplying several variables (e.g., y with subscript text NDVI).

Author Response

Dear reviewer,

thank you very much for your critical and helpful comments. We hope that our new revision (Reviewer1_response.docx) is addressing all of your concerns.

All the best,

René

----

Response to reviewer 1 (round 2)

Comment:

However, I am still skeptical about the general experimental setup. As in my first review about the previous version of the manuscript, I still think that the goal stated by the authors “to explore whether it would be possible to spectrally discriminate healthy and infected lemon myrtle tree canopies on a plantation” is not reached. Since the class “shadow” is used as a separate class, no distinction can be made between treated and untreated plants within the shadow area, which is half of the presented dataset. From an application point of view, it is unrealistic to wait for sunny weather, and some areas may always lie in shadow regions. From a theoretical point of view, conclusions about relevant wavelengths and indices for this disease may be incomplete and biased. Therefore, I recommend an experiment without the shadow class before publication.

I highly recommend to use a proper mathematical writing style, that means variables should be written as actual variables and not as text, since it can be confused with multiplying several variables (e.g., y with subscript text NDVI).

Response:

Thank you for your helpful comments. We added additional classification results where we (a) excluded the shadow class (S4) and (b) where we mixed shadow pixels with their respective treatments (S3). For both classifications (a, b) we added relevant classification features (S5 and S6).  We hope this addresses your concern.

 

Your comments led to an improved version of our manuscript which now is covering the aspect of “shadow processing” to some extent. However, we included the new results as supplementary data as we think that our original classification approach captures the most realistic sampling condition, a situation where shadows are part of the landscape. You are correct that a classification without shadows would be another valid, and practical, approach. Eventually, including or excluding shadows is a decision that must be made by the end-user. Both methods are widely accepted in the remote sensing community at the moment.

 

It is true that shadows are, in our case, half of the data. We added a new discussion paragraph where we provide some information about the processing of shadows. However, for our classification, a bias is unlikely as the classes were sampled in a balanced way (see error matrices). Conclusions about the disease are incomplete as our study, and our previous study, are the first remote sensing approaches for myrtle rust. As always, more studies must follow. 

 

Regarding a proper mathematical writing style, we used the MS Word equation tool for all equations in our manuscript.

 

Detailed edits:

Line 33: We added “shadow” to our keywords.

Line172-180: We added a section about our new classification analysis.

Line 276-279: We added results of our new classification analysis.

Line 281-282: We corrected text that was leftover from the very first submission.

Line 309-315: We added results of our new feature selection.

Line 362-364: We refer to the supplementary data now.

Line 411-426: We added a new paragraph on shadow-processing in our discussion section.

Line 450-453: We are listing the new supplementary data now.

Line 597-604: We added references that were used in our new discussion paragraph.

S3-S6: We provide new results of classification and feature selection as tables. 

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors adequately replied to all my questions and suggestions, and the changes that they have made improved considerably the quality of the work.

As far I am concerned I believe that the paper is now suited for publication.

Author Response

Dear reviewer,

thank you very much for taking time to improve our work. This is very much appreciated.

All the best from Germany,

René

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The Article, Multispectral, aerial disease detection for myrtle rust …. Is an interesting well written contribution fitting well in the scope of drones.

I found the paper well written in language, the methods described adequately and sufficient and the results presented and discussed well.

You will find some detailed suggestions and comments below.

However, I have a major critic or suggestion needing substantial changes of the manuscript if considered. To me it is not clear why the hyperspectral leaf data should be down-sampled to the drone camera resolution and compared this way. To me it makes little sense, neither scientifically nor from a practical point of view for the following reasons.

1.      You compare an active hyperspectral measurement on leaf sub-area (leaf clip, one pixel) with a passive imaging sensor approach, the principles are fundamentally different

2.      Although you average the selected areas of interest from the drone image, the reflectance signal contains whole leaves of variable directions towards the incoming light, shades, twigs, compared to the leaf clip which has a 3mm2 area of a leaf with standard light conditions, without interleaf areas or other features except leaf surface

3.      A micasense would never be used on leaf level because you need a minimum distance to the target (lens focus etc.)

4.      The down sampling of cause reduces the power of the learning algorithm compared to the full spectra

To me it seems more appropriate to:

Compare both sensors using a transfer model (from leaf spectra to canopy spectra), respecting leaf angle distribution sun angle etc. resulting in a similar view angle and abject level (Canopy), A multispectral drone based sensor and a hyperspectral ground based sensor was compared on plot (canopy level) for a sugar beet genotype trial by Joaland et al. (2018). https://www.mdpi.com/2072-4292/10/5/787

On the other hand you can still compare the ground based handheld method to UAV based spatial monitoring, using both methods as best as possible and discuss other parts like labour intensity, technology establishment etc. and leave the decision to the user or practitioner to select the appropriate method depending on needs…

 

Other comments:

 

L63: what do you mean by high capacity?  Large areas ?

L85: lemon-scented essential oils

L90 organically produced product (the leaf is always organic, even if its sprayed)

99-101 see main critic , in the next line you even say the a carefull selection is important, for such studies you take the best spectral resolution you can get and adjust wavebands of a camera not other way around, adjust argumentation

137 video A is supplement S2, right, be aware that such cloud  would add substantial noise to your dataset, please add discussion how this could affect the dataset and what could be done toimprove

145 2.78 cm is just to fine 2.8 cm is enough digits, even 3 cm is enough, respecting the differences in fligt altitude possible

150 company and country missing

191 are likely to confound… actually you have the data her to say something quantitatively, please do so

 

203-206 ff I’m questioning the usefulness of this scenario, see main critic above

 

226: this should not be an assumption, after Chlorophyll break down carotinoids and anthocyanins make the color, that’s a fact

 

326 its not a fair comparison, its two complete different systems, in terms of sensor, sensor posion and object of interest, see main critic

379 do you mean arable land, I think its better to talk of an orchard in the context of this paper

383 you are not discriminating single trees but infected from not infected parts of the canopy (trees)

389 trees are partly shaded (not overcast by shadows)

395 what you do is purely down sampling, simulation is a bit exagerated

397 ff see critic above

450 this sentence has nothing to do with your paper, and is far fetched, what about myrtle rust?

 

481 also present studies should do so! See main critic.

I think the paper is a good example of using multispectral drone based cameras to support plant pathology managment in field situations, but the downsampling part is somehow misleading.

I strongly encourage to improve the manuscript and it can be a valuable contribution.

Author Response

Dear reviewer #1,

thank you very much for your critical review. We assume you spent some time to help improving our manuscript and hope we could address all of your concerns. If you would like to discuss any comment you made that we did not address adequately, I would be glad if we could discuss your points as I am keen to learn ([email protected]). Please find a detailed response to your comments attached (2019Heimetal_drones-396097_ResponseToReviewer1). 

Kind regards,

René

Author Response File: Author Response.docx

Reviewer 2 Report

The goal of the manuscript is to detect myrtle rust on lemon myrtle trees using random forest classifier. The classification is performed on multispectral data acquired on a leaf-level as well as on a canopy-level. 

The topic is relevant and the paper is well written and well structured. The literature review is comprehensive.

Nevertheless, the comparison of leaf-level classification and canopy-level classification is not valid, in my eyes. The authors compare two classification results, which are based on different data sets and different sensors (field spectrometer vs. UAV image data). Therefore, a statement whether one experimental setup is more suitable to detect this kind of disease is not possible. The reason why the leaf-level based classifier produce worse results than the canopy-level classify can be various, where one could be that the number of training samples is only 5% of the amount used in comparison to the canopy-based classifier. Or, that the distribution of training samples covers a larger variety from both sides of the trees and in different height levels.  

Beside this, for the leaf-level based classification, the relevance for practice is questionable as shadows are used as a separate class and no distinction can be made between treated and untreated plants within the shadow area. Therefore, conclusions about relevant wavelengths and indices for this disease are incomplete and biased.

In sum, the knowledge gain for the community is small considering the above points. The overall idea of the paper is relevant, but a fair experimental setup needs to be used to draw reliable conclusions. Moreover, a more thorough discussion must be conducted, especially if differences in accuracy of ~20% occur between leaf-level and canopy-level.

Author Response

Dear reviewer #2,

thank you very much for your clear words. We removed all leaf-level data passages throuout the text and hope that our manuscript has improved in clarity. Additionally we ran some new classifications based on comments of reviewer #1 and revised the entire manuscript again. Please find a more detailed response in the attached document (2019Heimetal_drones-396097_ResponseToReviewer2).

Kind regards,

René

Author Response File: Author Response.docx

Reviewer 3 Report

General Remarks

I believe this manuscript is appropriate to be published within the Drones journal since it contains significant applications of drones UAS. The paper addresses the use of unmanned aerial systems (UAS) and a multispectral camera to discriminate fungicide treated and untreated lemon myrtle trees. I believe the paper is very well written and may be an asset for future monitoring and understanding of the disease. The paper is well structured, and the results and discussion are clear. With this mind, I believe the paper is suited for publication after a few minor revisions.

Figure 1: the authors refer in the text the shadow as SHD, but in the figure appears “B”. Please rectify.

Figure 5: please improve the quality.

Table 3 and 4: in these contingency tables the authors use the accuracy metrics OA, PA and UA. Are these skill scores? Can you write the equations on the section “Accuracy assessment”. I suggest the use of common skill scores like the Heidke skill score (HSS), the Peirce skill score (PSS), the Clayton skill score (CSS), the Gilbert skill score (GSS), and so on. These would improve the paper on the statistical focus, which is the most important tool used to summarise your results.

Author Response

Dear reviewer #3,

thank your for suggesting your skill scores. We provide a detailed response in the attached document (2019Heimetal_drones-396097_ResponseToReviewer3). We hope we could address your concerns and imroveour manuscript.

Kind regards,

René

Author Response File: Author Response.docx