Artificial Intelligence Applied to a First Screening of Naevoid Melanoma: A New Use of Fast Random Forest Algorithm in Dermatopathology
, Alessandro Massaro 2,3,†, Anna Colagrande 1, Irma Trilli 4, Giuseppe Ingravallo 1, Nadia Casatta 5, Carmelo Lupo 5, Andrea Ronchi 6, Renato Franco 6, Eugenio Maiorano 1 and Angelo Vacca 7
Round 1
Reviewer 1 Report
I enjoyed reviewing this article entitled "Artificial Intelligence applied to a first screening of Nevoid Melanoma: a new use of Fast Random Forest Algorithm in Dermatopathology ". Indeed AI represents the future in clinical and diagnostic routines. The topic is intriguing, and the study is well performed and described in the present manuscript. The Fast Random Forest algorithm is applied in pre-screening and image post-processing, helping pathology laboratories for faster screening of skin lesions and initiating immunohistochemical or molecular investigations.
Limitations of this method are mainly due to the need for a certain amount of data, as correctly stated by the authors.
Author Response
Dear Reviewer n'1,
thank you very much for this wonderful comments.
A warm greeting,
the authors.
Reviewer 2 Report
This work is focused on the results of applying artificial intelligence (AI) in the diagnosis of Nevoid melanoma.
Firstly, this work is curious as Nevoid melanoma is very infrequent and the diagnosis is difficult, thus I believe it will be interesting. Secondly, the used of artificial intelligence is novel and attractive. Thirdly, melanoma, is of course, the main concern of dermatologist, as is the unique mortal tumor in that area of medicine.
Introduction: Would it be possible to add a brief description of what Fast Ramdon Forest is? Just for introduce the reader.
Material and methods: they are clear and accessible even for no pathologist, the figures are descriptive and the writing well-organized.
Discussion: I believe it is brief in comparison to the introduction, for example. Maybe you could add something about:
-The importance of these infrequent but frequently misdiagnosis melanomas.
-The possible translation to clinical practice, how is the future? How the authors see this tool of AI and others? Are they real in the following years?
Author Response
Reviewer n'2:
This work is focused on the results of applying artificial intelligence (AI) in the diagnosis of Nevoid melanoma.
Firstly, this work is curious as Nevoid melanoma is very infrequent and the diagnosis is difficult, thus I believe it will be interesting. Secondly, the used of artificial intelligence is novel and attractive. Thirdly, melanoma, is of course, the main concern of dermatologist, as is the unique mortal tumor in that area of medicine.
Introduction: Would it be possible to add a brief description of what Fast Ramdon Forest is? Just for introduce the reader.
Answer n'1: Dear Reviewer n'2, Thank you very much for your comments. According to your suggestions we have added at the begin of Section 2.2 the following sentence explaining better the Fast Randon Forest algorithm:
“FRF is a powerful machine learning supervised algorithm which optimizes the Random Forest (RF) algorithm in performance about computational speed and classification accuracy: it defines step by step the best decision tree split condition thus avoiding the unnecessary computations.”
Reviewer n'2:
Material and methods: they are clear and accessible even for no pathologist, the figures are descriptive and the writing well-organized.
Discussion: I believe it is brief in comparison to the introduction, for example. Maybe you could add something about:
-The importance of these infrequent but frequently misdiagnosis melanomas.
-The possible translation to clinical practice, how is the future? How the authors see this tool of AI and others? Are they real in the following years?
Answer n'2: Dear Reviewer n'2, thanks again for these important and useful suggestions. We have added some important informations about our work in the Discussion section and we hope that it will be fine.
A warm greeting and thank you very much.
The authors
Reviewer 3 Report
The article presented for review focuses on using artificial intelligence to evaluate histopathological specimens in melanoma. It is an original approach and arouses great interest as a potential diagnostic and predictive tool in assessing the severity of the disease and the possibility of rapid therapeutic or surgical intervention.
The article is very well written and contains original data. However, it lacks a limitation of the study. It would be good to add a short discussion on the limitations or problems with the selection of appropriate algorithms, including other methods of staining slides or genetic testing.
Author Response
Reviewer n'3: The article presented for review focuses on using artificial intelligence to evaluate histopathological specimens in melanoma. It is an original approach and arouses great interest as a potential diagnostic and predictive tool in assessing the severity of the disease and the possibility of rapid therapeutic or surgical intervention.
Dear Reviewer n'3, Thank you very much for your comments. According to your suggestions, we have improved the work. Below and in the text (highlighted in yellow) are detailed our answers. Other your suggestions will be well accepted.
Below and in the revised we have added the following comment about this point (end of Section 4):
“The very good performance of the algorithm (see the ‘Precision’ parameter Fig. 3) indicates that the algorithm automatically defines the number of instances required for the best computation. A limit is certainly the first setting of the parameter of the algorithm (definition of the parameters such as Gaussian/Sobel/Hessian filter parameters Bilateral/ Lipschitz/Gabor/Derivative/Structured filtering conditions, etc. [13] ). Moreover, a high sensitivity of the error response is checked by slowly varying the algorithm’s parameters. Also other neural network algorithms could be used for image processing such as Long Short Term Memory (LSTM). This last algorithm requires to establish further setting of the parameters calibrated for the specific images which could increase the error calculus. The choice of the FRF is then mainly due to the optimization of the dermapathology platform to decrease the computational time and increasing simultaneously the response accuracy (main properties of the FRF algorithm)”.
Reviewer 4 Report
In this paper, the author proposes a melanoma detection method based on the Fast Random Forest (FRF) algorithm and provides ideas for integrating the method into clinical treatment processes, which is innovative. However, we believe that there are still some areas for improvement in the paper:
1. The literature review section can be expanded. In the paper, the literature review section is relatively brief and does not fully describe the methods used or some of the progress in the research field. We suggest expanding the literature review section to help readers better understand the research background and current status.
2. The actual meaning of each indicator in the experimental verification should be explained more specifically. In the paper, the author uses some indicators to evaluate the proposed method and also conducts an analysis in the experimental verification. However, we believe that it is still possible to explain the actual meaning of each indicator more specifically to help readers better understand the experimental results.
3. It is also suggested to introduce more indicators to analyze the method and present a more complete display of its performance. In Section 2.2 of the paper, it is mentioned that recall and precision are commonly used to evaluate the performance of FRF, but only precision is presented later. We believe that more indicators can be introduced to analyze the method and present a more complete display of its performance. In addition, in Section 2.2, the paper presents a comparison of other methods at the algorithm level, but does not explain how these differences lead to superior performance. We suggest further comparison and discussion of performance to help readers better understand the differences and advantages and disadvantages of different methods.
Author Response
Dear Reviewer,
Thank you very much for your comments. According to your suggestions, we have improved the work. Below and in the text (highlighted in yellow) are detailed our answers. Other your suggestions will be well accepted.
In this paper, the author proposes a melanoma detection method based on the Fast Random Forest (FRF) algorithm and provides ideas for integrating the method into clinical treatment processes, which is innovative. However, we believe that there are still some areas for improvement in the paper:
- The literature review section can be expanded. In the paper, the literature review section is relatively brief and does not fully describe the methods used or some of the progress in the research field. We suggest expanding the literature review section to help readers better understand the research background and current status.
Thank you very much. Done.
- The actual meaning of each indicator in the experimental verification should be explained more specifically. In the paper, the author uses some indicators to evaluate the proposed method and also conducts an analysis in the experimental verification. However, we believe that it is still possible to explain the actual meaning of each indicator more specifically to help readers better understand the experimental results.
At the end of section 4 (Discussion) we have added the following comments/explanations about the adopted indicator
“The indicators adopted for the analysis of the experimental results of Table 4 are:
- Total image area (mm2): this indicator allows to have a reference about the related percentage of anomalous clusters of a specific area dimension according to the image scale (this percentage will increase for a wider image);
- Number of red pixels: are the possibly anomalous pixels contained in the image and represent a primary quantification of the ‘risk distribution’;
- Equivalent area (mm2): the number of red pixels are merged to estimate an equivalent area useful to define the final percentage value, by losing the information associated with the spatial distribution;
- Equivalent area (%): final indicator defining the threshold risk for the pre-screening analysis, the threshold could change after the changing of the FRF algorithm parameters (the change of the parameters could increase or decrease the number of red pixels thus changing the threshold value which is set to 12 % according to the clinical point of view). ”
- It is also suggested to introduce more indicators to analyze the method and present a more complete display of its performance. In Section 2.2 of the paper, it is mentioned that recall and precision are commonly used to evaluate the performance of FRF, but only precision is presented later. We believe that more indicators can be introduced to analyze the method and present a more complete display of its performance.
The FRF tool provides different performance parameters. In the new section ‘Appendix A’ we discuss the main performance indicators which about our opinion are enough to prove the FRF efficiency.
Appendix A
In Table A1 are listed the main performance parameters of the adopted FRF algorithm. In Figs. A1, A2 and A3 are plotted the precision, the recall and the Fmeasure parameter of the NM8 case, respectively. All the graphs prove the efficiency of the FRF algorithm.
Table A1. Main performance parameters of the FRF algorithm (TP: true positive; TN: true negative; FP: false positive; FN: false negative).
|
Performance parameter |
Function |
|
Precision |
TP/(TP+FN) |
|
Recall |
TP/(TP+FP) |
|
FMeasure |
2TP/(2TP+FP+FN) |
Figure A1. Precision versus epoch number (sample NM8).
Figure A2. Recall parameter versus epoch number (sample NM8).
Figure A3. FMeasure parameter versus epoch number (sample NM8).
- In addition, in Section 2.2, the paper presents a comparison of other methods at the algorithm level, but does not explain how these differences lead to superior performance. We suggest further comparison and discussion of performance to help readers better understand the differences and advantages and disadvantages of different methods.
Thank you very much for this important observation. We answered to this point by adding the following discussion (see Section 4 after Fig. 4):
“The very good performance of the algorithm (see the ‘Precision’ parameter Fig. 3) indicates that the algorithm automatically defines the number of instances required for the best computation. A limit is certainly the first setting of the parameter of the algorithm (definition of the parameters such as Gaussian/Sobel/Hessian filter parameters Bilateral/ Lipschitz/Gabor/Derivative/Structured filtering conditions, etc. [13]). Moreover, a high sensitivity of the error response is checked by slowly varying the algorithm’s parameters. Also other neural network algorithms could be used for image processing such as Long Short Term Memory (LSTM). This last algorithm requires to establish further setting of the parameters calibrated for the specific images which could increase the error calculus. The choice of the FRF is then mainly due to the optimization of the dermapathology platform to decrease the computational time and increasing simultaneously the response accuracy (main properties of the FRF algorithm). “
Reviewer 5 Report
The authors present a machine learning analysis utilizing artificial intelligence to differentiate between regular regions and possible nevoid melanoma (NM) regions. They employed the Fast Random Forest (FRF) algorithm on dermatopathology images to enhance the understanding of the clinical diagnosis of nevoid melanomas at an early stage, aiming to minimize the risks associated with misdiagnosis.
This manuscript seems to be an extension of their previous work by Cazzato et al. (2022). However, it is not explicitly clarified whether the FRF trained with the current dataset specifically targets nevoid melanoma.
Specific points
1. Fig.2, The authors mentioned a 12% FRF threshold value. However, it appears that this threshold was determined based on only the four images presented in Figure 2.
2. The authors represented four images as probabilistic of NM and processed them with FRF. The authors should clarify whether any of them have been proven to be MN by immunohistochemistry markers. If so, they should include that data.
3. Fig.2, The authors should include image(s) where NM was not detected to demonstrate that the FRF is well-trained and specifically effective in detecting NM, but not in general malignant melanoma (MM).
4. Line 217 to 233, The ‘sequential phases’ explained in the discussion should go to the materials and methods section.
Minor-Points
Line 25, “…..properly recognised, can easily lead to death.” change to “………..timely diagnosed, can even lead to death.”
Line 110 – spell correction “Theoretically”
Author Response
Reviewer n'5:
The authors present a machine learning analysis utilizing artificial intelligence to differentiate between regular regions and possible nevoid melanoma (NM) regions. They employed the Fast Random Forest (FRF) algorithm on dermatopathology images to enhance the understanding of the clinical diagnosis of nevoid melanomas at an early stage, aiming to minimize the risks associated with misdiagnosis.
This manuscript seems to be an extension of their previous work by Cazzato et al. (2022). However, it is not explicitly clarified whether the FRF trained with the current dataset specifically targets nevoid melanoma.
Answer n'1: Dear Reviewer n'5, thank you very much for your useful and kind suggestions to improve the quality of our manuscript. We selectively tested and trained the Artificial Intelligence (AI) algorithm only on images of Nevoid Melanoma (NM) and not Malignant Melanoma in general, as done in previous work. Furthermore, we enriched our paper by reporting the performance data of the AI algorithm (Fast Random Forest) with a cohort of 18 benign nevi, referred to in dermatopathology as Unna nevi (i.e. verrucous papillary nevi that can closely mimic NM). We have appropriately added this information in the Results section.
Reviewer n'5:1. Fig.2, The authors mentioned a 12% FRF threshold value. However, it appears that this threshold was determined based on only the four images presented in Figure 2.
Answer n'2: Dear Reviewer n'5, thank you very much for this comment.He is absolutely right in that the way we had written the sentences was rather equivocal and one could easily have thought it was only 4 images tested, but there are a total of 18 NM cases used of which we have added a summary table regarding the demographic, histological and immunohistochemical characteristics of the various cases analysed.
Reviewer n'5: 2. The authors represented four images as probabilistic of NM and processed them with FRF. The authors should clarify whether any of them have been proven to be MN by immunohistochemistry markers. If so, they should include that data.
Answer n'3: Thank you for this comment. We have added a detailed table with all the clinical, demographic, histological and immunohistochemical information of the analysed patients.
Reviewer n'5:3. Fig.2, The authors should include image(s) where NM was not detected to demonstrate that the FRF is well-trained and specifically effective in detecting NM, but not in general malignant melanoma (MM).
Answer n'4: Thank you very much. So, We have included in the 'results' section images of benign nevus (Nevus of Unna), which is an entity that rightfully enters into differential diagnosis with NM. We have also emphasised more that the entire study was conducted on histological images of NM and not of Malignant Melanoma (MM) in general.
Reviewer n'5:4. Line 217 to 233, The ‘sequential phases’ explained in the discussion should go to the materials and methods section.
Answer n'5: Thank you very much. Done.
Reviewer n'5:
Minor-Points
Line 25, “…..properly recognised, can easily lead to death.” change to “………..timely diagnosed, can even lead to death.”
Line 110 – spell correction “Theoretically”
Answer n'6: Done. Thank you for all.
Round 2
Reviewer 4 Report
The authors have addressed the comments. I recommend its publication as the current form.
Reviewer 5 Report
The authors have answered most of my comments.
