Inferior Alveolar Canal Automatic Detection with Deep Learning CNNs on CBCTs: Development of a Novel Model and Release of Open-Source Dataset and Algorithm
, Arrigo Pellacani 1, Federico Bolelli 2, Marco Cipriano 2, Luca Lumetti 2, Sara Negrello 3, Stefano Allegretti 2, Paolo Minafra 4, Federico Pollastri 2, Riccardo Nocini 5, Giacomo Colletti 6, Luigi Chiarini 6, Costantino Grana 2 and Alexandre Anesi 6
Round 1
Reviewer 1 Report
A brief summary (one short paragraph) outlining the aim of the paper, its main contributions and strengths.
A report about an experiment with CNN and some data, a good experimental report for the beginners who are interested in machine learning. But if consider it as a research paper, the only contribution is its datasets.
General concept comments
Line 36, misuse of “thanks to”.
Line 42, “for the first time datasets were released”, how do you learn English?
Line 60, why “.”?
Lots of problems about use of language.
Line 75, who cares about the threshold, what about other methods?
Line 150-170, lots of nonsense. You are doing a research!
Line 345, a blurry picture.
Line 325-340, a naive training strategy, lack of novelty.
Line 403, another blurry picture.
Is the manuscript clear, relevant for the field and presented in a well-structured manner?
Although it’s quite relevant for the field, it’s not clear enough and well-structured enough because of lacking some necessary parts, for example, introduction about related work, comparison with other similar experimental results and don’t just compare with yourself.
Are the cited references mostly recent publications (within the last 5 years) and relevant? Does it include an excessive number of self-citations?
Most of the cited references are more than ten years ago, but they’re quite relevant. No self-citations found.
Is the manuscript scientifically sound and is the experimental design appropriate to test the hypothesis?
The experiment is not sound and the design is not appropriate because it use specific datasets to train a specific network and no comparisons with related work.
Are the manuscript’s results reproducible based on the details given in the methods section?
No.
Are the figures/tables/images/schemes appropriate? Do they properly show the data? Are they easy to interpret and understand? Is the data interpreted appropriately and consistently throughout the manuscript? Please include details regarding the statistical analysis or data acquired from specific databases.
No, the figures are all vague and the size of datasets is small.
Are the conclusions consistent with the evidence and arguments presented? Please evaluate the ethics statements and data availability statements to ensure they are adequate.
Ethics statements seems good, but the data is not available now.
Novelty: Is the question original and well-defined? Do the results provide an advancement of the current knowledge?
Not original.
Scope: Does the work fit the journal scope*?
Quite fit.
Significance: Are the results interpreted appropriately? Are they significant? Are all conclusions justified and supported by the results? Are hypotheses carefully identified as such?
Not appropriately.
Quality: Is the article written in an appropriate way? Are the data and analyses presented appropriately? Are the highest standards for presentation of the results used?
Not appropriate.
Scientific Soundness: Is the study correctly designed and technically sound? Are the analyses performed with the highest technical standards? Is the data robust enough to draw conclusions? Are the methods, tools, software, and reagents described with sufficient details to allow another researcher to reproduce the results? Is the raw data available and correct (where applicable)?
Not sound.
Interest to the Readers: Are the conclusions interesting for the readership of the journal? Will the paper attract a wide readership, or be of interest only to a limited number of people? (Please see the Aims and Scope of the journal.)
Not quite interesting.
Overall Merit: Is there an overall benefit to publishing this work? Does the work advance the current knowledge? Do the authors address an important long-standing question with smart experiments? Do the authors present a negative result of a valid scientific hypothesis?
No benefit at all.
English Level: Limited. It's all machine translation.
Author Response
Dear Reviewer,
We thank you for your precious comments. We modified and improved the paper according to your suggestions.
Moreover, as both reviewers suggested to improve the English, we obtained a revision of the English made by a professional translator.
We stay at disposal for any inconvenience.
Reviewer 2 Report
The study is interesting. However, some expression is unclear and the focus is not clear. The structure is inadequate. Below are some comments to further improve the quality of the manuscript.
Ë— The abstract needs to be changed and revised to be more quantitative. Authors can attract readers' attention by listing numerical results in this section. Also, the authors can state the major findings of the study.
Ë— There are too many keywords, those that reflect the essence of the work should be chosen.
Ë— The sections "The inferior alveolar canal: clinical insights", "Cone beam computed tomography and conventional radiology", "CBCT image processing", and" Automatic segmentation of the Inferior Alveolar Canal" should be combined with the title ‘’Literature Review’’.
Also, Lines 194-202 should be moved to the introduction.
The introduction and literature review structure of this paper should be as follows:
1. Introduction
2. Literature Review
2.1. The inferior alveolar canal: clinical insights
2.2.Cone beam computed tomography and conventional radiology
2.3. CBCT image processing
2.4.Automatic segmentation of the Inferior Alveolar Canal
3. Material and Methods…………………….
Ë—The introduction section (Lines 48-55) is too general. Which gap in the literature does this study intend to fill, and what contribution does it make to the literature? Please highlight more in the introduction section.
Ë— The main objectives of this study and the novelty point should be discussed clearly and in detail. Make the novelty clearer and in the context of existing literature.
Ë— How did you optimize your CNN model?
Ë— What were the number of neurons, activation function, optimizer, learning rate, batch size, and epochs?
Ë— Please give descriptive statistics of the data set.
Author Response
Dear Reviewer,
We thank you for your precious comments. We modified and improved the paper according to your suggestions.
Moreover, as both reviewers suggested to improve the English, we obtained a revision of the English made by a professional translator.
We stay at disposal for any inconvenience.
Author Response File: 
Author Response.docx
Reviewer 3 Report
Comments for author File: 
Comments.pdf
Author Response
Dear Reviewer,
We have carefully read the reviews received to the paper “Inferior Alveolar Canal automatic detection with deep learning CNNs on CBCTs: development of a novel model and release of open-source dataset and algorithm”.
We modified the paper according to your precious suggestions and we also want to thank for the constructive and positive observations that helped us to improve the paper.
We stay at disposal for any inconvenience.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
After analyzing the CBCT data, the authors collected and labeled a new densely labeled 3D dataset, on which the CNN model was trained to automatically recognize IAC, and achieved a Dice score of 0.79 and an IoU score of 0.64 on the basis of IAC guaranteed 3D data. And the model running time is also in line with expectations, real-time performance is good, but of course the model size can be further optimized to improve the running speed.
However, there are a few points below that I hope the author can improve further.
1. It should be noted that the internal structure of the CNN model is not described in detail in the text. A flowchart of the entire system is also not given, which can be difficult to understand. I hope the author will improve this aspect.
2. The paper only presents and analyzes the final result, and although the authors say that the code will be open sourced, the lack of a detailed description of the model parameters in the paper may make it more difficult to reproduce the work.
3. The paper "3. Materials and Methods" designs three different sets of experiments, but does not give the relevant process of model training, nor does it explain how to judge whether the model training results are appropriate. For example, how many rounds of different experimental models were trained, what is the loss value, etc.
4. In 4.3.2, please explain more about "In this way it was possible to expand the sparse 2D annotationsinto 3Dannotations". Is it possible to combine a specific example with a diagram?
5. Less comparison with other methods of the same type, with regard to model efficiency, only "... Annotation volumes take an average of 87.3 seconds to obtain..."Humans don't spend as much time consuming as other modeling methods. In addition to the experimental comparison of model methods that did not compare the effects of many different schemes, I would like to emphasize again the advantages of this paper method over other existing methods.
Overall, the work done in this article is very meaningful and the results are satisfactory. I hope the author will further improve the above.
Author Response
Dear Reviewer,
We have carefully read the reviews received to the paper “Inferior Alveolar Canal automatic detection with deep learning CNNs on CBCTs: development of a novel model and release of open-source dataset and algorithm”.
We modified the paper according to your precious suggestions and we also want to thank for the constructive and positive observations that helped us to improve the paper.
We stay at disposal for any inconvenience.
Author Response File: Author Response.docx
Reviewer 2 Report
First of all, I appreciate the opportunity to review the paper Inferior Alveolar Canal automatic detection with deep learning CNNs on CBCTs: development of a novel model and release of open-source dataset and algorithm. I have read the paper in detail and I cannot recommend it to be published in its current state. There are numerous reasons for this opinion:
The abstract is not well written, ( Please see https://www.mdpi.com/journal/applsci/instructions)
The introduction is very poor. It must be made clear what the novelty of the current study is and what its significance is for the readers of the journal
Applying an existing model to a limited example does not justify publication (How did you optimize your CNN model? What was the number of neurons, activation function, optimizer, learning rate, batch size, and epochs?)
No descriptive statistics are given for the dataset.
In the conclusion section: Contributions? Future research directions? Limitations?
Overall, the work unfortunately does not meet the standards of a scientific paper.
Author Response
Dear Reviewer,
We have carefully read the reviews received to the paper “Inferior Alveolar Canal automatic detection with deep learning CNNs on CBCTs: development of a novel model and release of open-source dataset and algorithm”.
We modified the paper according to the suggestions received. Attached you will find the reply report.
We stay at disposal for any inconvenience.
Author Response File: Author Response.docx
Round 3
Reviewer 1 Report
Novelty
The innovation of this paper is the labeling of a new dataset and the optimization of the model. The structure of the CNN is taken from other papers, and the way the dataset is divided is also from other papers. So the novelty is limited. If there are other innovations, I hope the author can emphasize them in the paper and reduce the description of non-original parts.
Scope
It’s indeed a kind of current advance in Dentistry.
Scientific Soundness
The paper only lists the results of other work compared with this paper and does not consider the difference in data sets, so it is not possible to verify the soundness of the method in this paper. There is a lack of experiments with different models in the same dataset and experiments with the same model in different datasets.
Quality
The image is in the form of a screenshot, not a vector image, which makes it look blurry. The tables are too large for the description of the model structure, which should be placed in the appendix, while the data for the experimental comparison are not tabulated, but only briefly described in the body. In the comparison experiments with other methods, the experiments of other methods are not conducted, but simply copy the results from other papers, which leads to the lack of factual evidence for the analysis of the superiority of the model.
English Level: Is the English language appropriate and understandable?
line 33: focuse on -> focus on
line 60: in this field -> in this field,
line 85: dataset -> datasets
line 90: this study was to develop a CNN able to -> this study is to develop a CNN which is able to
line 91: results achieved this far -> results achieved so far
line 107: known as -> known as the
line 124: can lead to temporary damage of the nerve -> can lead to a temporary damage to the nerve
line 126: this is a relevant data -> this is the relevant data
line 141: accurate -> accurately
line 152: in such cases -> in such cases,
line 156: a definitely -> definitely
line 198: CCNNs -> CNNs
line 201: CNNSs -> CNNs
line 234: are publicly accessible making impossible -> is publicly accessible making it impossible
Author Response
Dear Reviewer,
We have carefully read the reviews received to the paper “Inferior Alveolar Canal automatic detection with deep learning CNNs on CBCTs: development of a novel model and release of open-source dataset and algorithm”.
We modified the paper according to your precious suggestions and we also want to thank for the constructive and positive observations that helped us to improve the paper.
We stay at disposal for any inconvenience.
Author Response File: Author Response.docx
Reviewer 2 Report
The authors have greatly improved the article. Now the article is acceptable.
Author Response
We thank the reviewer for the kind comments.
Author Response File: Author Response.docx
