The Chart Diagnostic System Improves the Diagnostic Accuracy of Cervical Lymph Node Metastasis in Oral Squamous Cell Carcinoma

Round 1

Reviewer 1 Report (Previous Reviewer 1)

A potential title suggession:

The Chart Diagnostic System improves the diagnostic accuracy of cervical lymph node metastasis in oral squamous cell carcinoma. 

1. Past concern:

"A 57 patients cohort was used in deriving inferences about the use of current standard practices (Tables 3). The P- values of analysis are often borderline, and the power of statistics was not discussed in the manuscript. Some of the obtained p-values are substantially different than a few other previous works, e.g., Johannes Laimer et al. 2020. Surprisingly authors never discussed their approach and its performance in some of the previously published cohorts adopting multimodal imaging. Please provide supporting discussion on how general or representative the chosen 57-patient cohort is. Considering the impact of this study and inferences, this discussion is essential."

Authors' update:

Line 146: "Sample size adequacy was evaluated with β=0.80 and α=0.05 by post hoc test."

New comment:

I am concerned with sample numbers considering all potential heterogeneity in the sample series. The authors' response in Line 146 needs further elaboration. Please share the results, assumptions, and parameters for this beta value estimate (a supporting table may be good).

2.Past concern:

Authors had discussed US results in the light of past results and suggested observed long/short ratio is 2.1 in 33 histopathologically confirmed cases. Authors never addressed in the case of <=2 as L/S ratio what would have been the sensitivity and specificity in the n=1587 cohort. Inference with US analysis is unclear (What are recommended cut points, if any?) 

Authors' update:

The ROC curves cutoff values, AUC, sensitivity and specificity of long diameter, short diameter and long/short ratio were 9.14 vs 5.20 vs 1.55, 0.761 vs 0.826 vs 0.626, 66.7% vs 66.7% vs 48.5% and 41.4% vs 43.1% vs 65.8%, respectively (data not shown). 

And  In this study we included 72 OSCC patients to evaluate the association between imaging findings and cervical lymph node metastases and 58 patients to evaluate with biopsy pathology findings. To evaluate this cohort study's validity, we used a post hoc test with Effect size=0.8 and α=0.05 to assess the power. The results showed that Power=0.999, which was considered to be a reasonable sample size.

New comment: Data needed to be shared in proper tabulated form. What is the rationale for the 0.8 effect size? An appropriate justification is a must.

3. Past concern:

Due to the chosen cohort, the Figure 1 AUC-ROC curve trajectory was substantially choppy. Do authors feel confident in suggesting SUV-max as 0.804?

Authors' response:

Our cut-off value is 2.80, which is close to the 2.50 reported by Caylakl et al. However, the specificity is not high at 65.3%, so there is a possibility of over-surgery.

New comment:

The response is acceptable. It is not essential, but authors may also need to show the performance of SUV min results.

4. New minor comment:

Line 283: 5-8≧ should be ≧5-8

The rest of the responses from the authors are satisfactory and concerns are appropriately addressed.

Author Response

Thank you for your kind review and suggestions. We are not experts in statistical analysis. We have recalculated the power using G*Power 3.1 software. However, we are not sure if the correction is correct. If we are wrong, could you please point it out again?

Author Response File: Author Response.docx

Reviewer 2 Report (Previous Reviewer 3)

It is still not clear if in the radical neck dissection (which involves the I-V levels) the spinal nerve, the internal jugular vein and the sternocleidomastoid muscle are always sacrificed.

Twenty-nine cN0 patients that did not undergo neck dissection as their primary surgical treatment have been improperly considered as pN0.

A review of the preoperative images and surgical specimens could help evaluate the DOI and make it possible to update to the 8th edition of the TNM making all the work more useful.

Author Response

We have reviewed and revised your suggestion. Please review again for peer review.

Author Response File: Author Response.docx

Reviewer 3 Report (New Reviewer)

I have reviewed the manuscript and please consider the following comments. In my opinion, the work cannot be published in its current form. - Please double check all the abbreviations (e.g., there is a CT that is undefined in the abstract) - Please double check the text for grammatical errors and typos (e.g., there is a “patent” in the abstract) - It is not clear to me the number of patients involved (57 or 73?) - I’m not sure the authors applied statistical analysis properly. There is no mention about the normality of numerical variables. Moreover, the authors reported a sentence about the statistical power evaluation but they did non provide any information about the formula they used and the effect size. I’m serious doubts about the multivariable analysis that included a lot of variables for the low sample size (please use the correct term “multivariable” and not “multivariate” analysis; they are different). - No mention about the ROC curve in the statistical analysis section. - I would suggest an extensive revision by a statistical expert. Best regards

Author Response

Thank you for your kind review.

Two patient groups were included in this study: 72 and 57 patients.

The 72 oral cancer patients were compared in relation to imaging findings of cervical lymph nodes and metastases.

In 57 OSCC patients, biopsy specimens were evaluated for histopathologic factors (budding score, lymphatic invasion, vascular invasion, nerve invasion, and histologic invasion factors) and the presence of cervical lymph node metastases.

We have modified the abstract line26-29.

We have checked and corrected the abbreviations and corrected "multivariable" to "multivariate".

We have added a description of the ROC curve on line 137-138.

We have added a table that supplements the results of the post hoc tests for the power evaluation.

We have modified line 141-142, line212-215 and added a supplement table 1 to line440.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report (Previous Reviewer 1)

1. The requested data should be presented.

2. If needed a statistician specialist should be consulted. 

Author Response

Thank you for your kind review.

We have added a table that supplements the results of the post hoc tests for the power evaluation.

We have modified line 141-142, line212-215 and added a supplement table 1 to line440.

Author Response File: Author Response.docx

Reviewer 2 Report (Previous Reviewer 3)

The paper is now suitable for publication

Author Response

Thank you for your kind review.

Reviewer 3 Report (New Reviewer)

The authors did not address my previous comments. Please consider the previous round of revision and address all issues, especially those related to statistical analysis.

Author Response

To Reviewer:

We would like to express our sincere appreciation for the important comments.

Please accept my sincere apologies for the poor initial response.

#1 Abbreviations confirmed: We apologize for the insufficient presentation, and we have corrected the text according to the reviewer’s recommendation.

Line 18: ultrasonography (US)

Line 19: Computed Tomography (CT), Yamamoto-kohama (YK)

Lines 26-27: standardized uptake value (SUV), Positron Emission Tomography (PET)

Lines 49-51: Ultrasonography (US), Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET)

Lines 107-108: standardized uptake value (SUV)

Line 121: Yamamoto-Kohama (YK)

#2 Grammatical errors and typos: We apologize for this error, and we have again checked and corrected the grammar and typos.

Line 28: “patent” to “patients”

Line 160: “rang” to “range”

 We do English editing at edigage and we attach a certificate.certificateをください。

#3 Patients group: We apologize for the insufficient explanation. We studied two patient groups in this study - 72 and 57 patients. In a group of 72 oral cancer patients, the association between cervical lymph node metastases and imaging findings was investigated. 57 of the 72 patients whose biopsy specimens were available for evaluation were examined for the association between cervical lymph node metastases and histopathologic factors.

#4 Statistical analysis: We appreciate your important suggestions. Following your instructions, we have added a colleague with knowledge of statistics and made additional corrections to the statistics statement.

①All values of imaging feature were not normally distributed. We added this sentence in lines 164-165 and corrected the resulting values to the median and interquartile range in Table 2 and added legend in line 179. In pathological factors, only budding score of lymph node metastasis cases showed normal distribution. We added this sentence in lines 199-200 and line 211 in legend of table 3. The distribution of each numerical value was shown in the boxplots in figure 1 and 2.

②We have added a description about the results of the post hoc tests for the power evaluation in lines 144-145. We also added supplemental table 1 in line 457.

③We examined the factors included in the multivariable analysis by forward stepwise selection method and modified the multivariable analysis. We added methods in lines 143-144. We added the results of forward stepwise selection method in lines 192-194 and lines 221-223.

We have corrected "multivariate" to "multivariable" as you pointed out. Line 36, 140, 142, 143, 194, 223, 224, 280, 283 and table 2, 3.

#5 About ROC curve: We apologize for the insufficient explanation. We have added a description of the ROC curve in lines 139-140.

We look forward to hearing from you regarding our submission. We would be glad to respond to any further questions and comments that you may have.

Round 3

Reviewer 3 Report (New Reviewer)

None

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The study titled “The Chart Diagnostic System for Presurgical Lymph Node Metastasis Combining Imaging and Pathological Features of Oral Squamous Cell Carcinoma” addresses extremely crucial concerns related to surgical oncology practices (at least in Japan). The authors have very aptly introduced the critical concern regarding the need for accurate determination of cervical lymph node involvement in Oral squamous cell carcinoma patients. The authors discussed how their chart-based multimodal preoperative detection of lymph node involvement is more accurate than the current standard of care methods like Ultra Sonography, FDG-PET SUV max, etc. 

Although very significant work, I recognize a language check and writing flow betterment is imminent; in other words, professional input may substantially benefit the readability of this manuscript.

Major Concerns: 

1. The title is missing a critical subject matter. A specific and simple title may benefit the readership quotient of this manuscript.

2. A 57 patients cohort was used in deriving inferences about the use of current standard practices (Tables 3). The P- values of analysis are often borderline, and the power of statistics was not discussed in the manuscript. Some of the obtained p-values are substantially different than a few other previous works, e.g., Johannes Laimer et al. 2020. Surprisingly authors never discussed their approach and its performance in some of the previously published cohorts adopting multimodal imaging. Please provide supporting discussion on how general or representative the chosen 57-patient cohort is. Considering the impact of this study and inferences, this discussion is essential.

3. Authors had discussed US results in the light of past results and suggested observed long/short ratio is 2.1 in 33 histopathologically confirmed cases. Authors never addressed in the case of <=2 as L/S ratio what would have been the sensitivity and specificity in the n=1587 cohort. Inference with US analysis is unclear (What are recommended cut points, if any?) 

4. Due to the chosen cohort, the Figure 1 AUC-ROC curve trajectory was substantially choppy. Do authors feel confident in suggesting SUV-max as 0.804?

5. I am very excited to see the chart-based approach and its performance. It needs further clarification on how do the authors arrived at this unique flow or sequence of tests.

Minor Issues:

1. For clarity, the p=values should be written up to 3 decimal places.

2. Bibliography format may need validation by a single reference manager. I am noting at least two styles are adopted in this manuscript. A reference manager validated and journal Onco recommended style is desirable.

Author Response

Thank you for your precise peer review and valuable advice.

We have considered the points you raised and made revisions to the content of the manuscript. We look forward to hearing from you.

Major Concerns: 

1. The title is missing a critical subject matter. A specific and simple title may benefit the readership quotient of this manuscript.

We have changed the title in order to indicate the content more simply.

2. A 57 patients cohort was used in deriving inferences about the use of current standard practices (Tables 3). The P- values of analysis are often borderline, and the power of statistics was not discussed in the manuscript. Some of the obtained p-values are substantially different than a few other previous works, e.g., Johannes Laimer et al. 2020. Surprisingly authors never discussed their approach and its performance in some of the previously published cohorts adopting multimodal imaging. Please provide supporting discussion on how general or representative the chosen 57-patient cohort is. Considering the impact of this study and inferences, this discussion is essential.

We have added statistical methods to Line135-136 for evaluating the power. We have added the results and discussion in Line 222-226. As a result, we consider this cohort study of 57 patients to be acceptable.

3. Authors had discussed US results in the light of past results and suggested observed long/short ratio is 2.1 in 33 histopathologically confirmed cases. Authors never addressed in the case of <=2 as L/S ratio what would have been the sensitivity and specificity in the n=1587 cohort. Inference with US analysis is unclear (What are recommended cut points, if any?) 

ROC cureve results in US findings were added to Line 156-159. The cut-off value of L/S retio was 1.55, which is not high (AUC=0.626), therefore, L/S retio is not as important as the others for diagnosis.

4. Due to the chosen cohort, the Figure 1 AUC-ROC curve trajectory was substantially choppy. Do authors feel confident in suggesting SUV-max as 0.804?

Our cut off value is 2.80, which is close to the 2.50 reported by Caylakl et al. However, the specificity is not high at 65.3%, so there is a possibility of over surgery.

5. I am very excited to see the chart-based approach and its performance. It needs further clarification on how do the authors arrived at this unique flow or sequence of tests.

 We have added background to L207-L212 where we began our research.

Minor Issues:

1. For clarity, the p=values should be written up to 3 decimal places.

We have corrected the problem as you indicated. There is a calculation error and some values have been changed, but the conclusion remains unchanged.

2. Bibliography format may need validation by a single reference manager. I am noting at least two styles are adopted in this manuscript. A reference manager validated and journal Onco recommended style is desirable.

We have corrected references as you indicated.

Author Response File: Author Response.docx

Reviewer 2 Report

The authors have presented a sound study in cancer diagnostics by using a predictive pattern. Overall presentation and arrangement of components is good and I find no issues. But, I will suggest the authors to check for the following amendments in the manuscript:

The abbreviations must be elaborated on their first use. Please check for all possible abbreviations throughout the manuscript.

In my opinion, the abstract can be improved for clarity and preciseness. It must be attractive enough to grab the attention of the audience. Make it more concise.

Incorrect spellings and format settings have been highlighted. Superscripts and subscripts should be checked.

Table legends and their footnotes should be validated for *, ** and ***.

After these revisions, the manuscript will be ready for acceptance.

Comments for author File: Comments.pdf

Author Response

Thank you for your precise peer review and valuable advice.

We have considered the points you raised and made revisions to the content of the manuscript. We look forward to hearing from you.

The authors have presented a sound study in cancer diagnostics by using a predictive pattern. Overall presentation and arrangement of components is good and I find no issues. But, I will suggest the authors to check for the following amendments in the manuscript:

The abbreviations must be elaborated on their first use. Please check for all possible abbreviations throughout the manuscript.

In my opinion, the abstract can be improved for clarity and preciseness. It must be attractive enough to grab the attention of the audience. Make it more concise.

Incorrect spellings and format settings have been highlighted. Superscripts and subscripts should be checked.

Table legends and their footnotes should be validated for *, ** and ***.

After these revisions, the manuscript will be ready for acceptance.

We have revised the sections you pointed out.

Author Response File: Author Response.docx

Reviewer 3 Report

This paper analyzes a debated problem that is the preoperative diagnosis of the involvement of the cervical lymph nodes in oral cancers.

The study includes preoperative evaluation with CT, US and FDG-PET / CT together with the biopsy of T.

However, there are some critical issues right from the start of the study.

The reference to the TNM classification is to the 7th version which has now been supplanted by the 8th already in 2016 with the introduction of new significant parameters capable of modifying the classification such as the DOI (depth of invasion) in T and extranodal extension (ENE) in N.

All patients with T> 3cm received preoperative chemo-radiotherapy. Beyond the considerations on this practice, this treatment obviously modifies the lymphatic drainage and subsequent evaluations. Furthermore, it is not reported how long after the end of preoperative chemo-radiotherapy the surgery is performed and whether the evaluation with CT, US and FDG-PET / CT is repeated again.

In case of clinical diagnosis of lymph node involvement, the reported treatment is a radical neck dissection which involves the removal of the I-V levels together with the spinal nerve, the internal jugular vein and the sternocleidomastoid muscle) and unless a bad interpretation of the terminology could result excessive given the current treatment indications in case of cN1 of level I. If cN0 patients need reconstructive surgery, they undergo a neck dissection defined as "hyperhyoid" (line 81) but which I believe can be referred to as Supraomohyoid Neck Dissection or more correctly selective neck dissection of levels I-III.

The biopsy on T was performed only in 52 of the 73 patients analyzed as also reported in the authors' comments on the study. Interesting data seem to arise from the evaluation of the tumor budding score and from the YK classification while there are no references to the DOI now necessary for a correct classification of the TNM.

There is no clear assessment of the results of the postoperative histological examination, with the number of positive lymph nodes that may be present, their topography and the presence of micrometastases or extranodal diffusion.

The imaging analysis is still valid and although it does not take into account MR, which is considered by many authors to be the gold standard in the evaluation of suprahyoid soft tissues, it brings interesting results (there is an error in the line 213 short / long ratio should be long / short ratio).

The resulting diagnostic algorithm, although interesting, cannot be validated by the reported data.

Author Response

Thank you for your precise peer review and valuable advice.

We have considered the points you raised and made revisions to the content of the manuscript. We look forward to hearing from you.

In this study, we examined the imaging findings of metastatic lymph nodes without distinguishing between extranodal invasion and micrometastasis. This is because the clinical problem is micrometastasis, and it was necessary to create a system to predict metastasis based on the imaging findings of lymph nodes with micrometastasis.

The reason why DOI was not examined in this study was added to the limitation. Since the patients in this study were diagnosed and treated preoperatively based on the 7th edition, we were not able to examine DOI, which is an important factor related to metastasis.

The radical surgery was performed within 1 month after preoperative treatment. We added it to Line 77.

We have corrected hyperhyoid to supraomohyoid and long/short ratio in line230 as you suggested.

Author Response File: Author Response.docx