Predicting Leukoplakia and Oral Squamous Cell Carcinoma Using Interpretable Machine Learning: A Retrospective Analysis
Abstract
:1. Introduction
- Assess predictive models that analyse trends in the underlying features contributing to common white lesions, such as leukoplakia and OSCC;
- Extract explanations for the decisions of the best performing machine learning models.
2. Material and Methods
2.1. Dataset Description
2.2. Data Preprocessing
2.3. Machine Learning Models Used
2.3.1. K Nearest Neighbours (KNN)
2.3.2. Logistic Regression
2.3.3. Naive Bayes
2.3.4. Support Vector Machines (SVM)
2.3.5. Random Forest
2.4. Evaluation Metrics
3. Results
3.1. Model Performances across the Board
3.1.1. Model Performance for Leukoplakia without Dysplasia
3.1.2. Model Performance for Leukoplakia with Dysplasia
3.1.3. Model Performance for Oral Squamous Cell Carcinoma
3.1.4. Overall Performance across All Three Categories
3.2. Receiver Operating Characteristics of the Models
3.3. Confusion Matrices
3.4. Shapley Additive Explanations (SHAP) for Model Interpretability
3.4.1. Interpretability of Leukoplakia without Dysplasia
3.4.2. Interpretability of Leukoplakia with Dysplasia
3.4.3. Interpretability of Oral Squamous Cell Carcinoma
4. Discussion
5. Conclusions
- The Random Forest model achieved the highest performance with an overall accuracy of 93%, showing superior class-specific precision, recall, and F1 scores for both OSCC and various types of leukoplakia.
- SHAP (SHapley Additive exPlanations) analysis identified the top predictors influencing the model’s decisions. For leukoplakia with dysplasia, these included buccal mucosa localisation, an age over 60 years, and lesion size. For leukoplakia without dysplasia, the key predictors were gingival and tongue localisation, along with lesion size. For OSCC, gingival, floor-of-mouth, and buccal mucosa localisations were the most influential. The model notably indicated that lesions on the floor of the mouth were highly unlikely to be dysplastic, instead showing one of the highest probabilities for being OSCC.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- World Health Organisation. Cancer. 2020. Available online: https://www.who.int/news-room/fact-sheets/detail/cancer (accessed on 9 August 2024).
- Dhanuthai, K.; Rojanawatsirivej, S.; Thosaporn, W.; Kintarak, S.; Subarnbhesaj, A.; Darling, M.; Kryshtalskyj, E.; Chiang, C.P.; Shin, H.I.; Choi, S.Y.; et al. Oral cancer: A multicenter study. Med. Oral Patol. Oral Cir. Buccal 2018, 23, e23–e29. [Google Scholar] [CrossRef] [PubMed]
- Di Spirito, F.; Di Palo, M.P.; Folliero, V.; Cannata, D.; Franci, G.; Martina, S.; Amato, M. Oral bacteria, virus and fungi in saliva and tissue samples from adult subjects with Oral squamous cell carcinoma: An umbrella review. Cancers 2023, 15, 5540. [Google Scholar] [CrossRef] [PubMed]
- Warnakulasuriya, S. Global epidemiology of oral and oropharyngeal cancer. Oral Oncol. 2009, 45, 309–316. [Google Scholar] [CrossRef] [PubMed]
- Neville, B.W.; Damm, D.D.; Allen, C.M.; Bouquot, J.E. Oral and Maxillofacial Pathology; WB Saunders: Philadelphia, PA, USA, 2002. [Google Scholar]
- Farook, T.H.; Haq, T.M.; Ramees, L.; Dudley, J. Predicting masticatory muscle activity and deviations in mouth opening from non-invasive temporomandibular joint complex functional analyses. J. Oral Rehabil. 2024, 51, 1770–1777. [Google Scholar] [CrossRef]
- Montavon, G.; Samek, W.; Müller, K.-R. Methods for interpreting and understanding deep neural networks. Digit. Signal Process. 2018, 73, 1–15. [Google Scholar] [CrossRef]
- Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
- Farook, T.H.; Dudley, J. Automation and deep (machine) learning in temporomandibular joint disorder radiomics. A systematic review. J. Oral Rehabil. 2023, 50, 501–521. [Google Scholar] [CrossRef]
- Lipton, Z.C. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 2018, 16, 31–57. [Google Scholar] [CrossRef]
- Mahmood, H.; Shaban, M.; Indave, B.I.; Santos-Silva, A.R.; Rajpoot, N.; Khurram, S.A. Use of artificial intelligence in diagnosis of head and neck precancerous and cancerous lesions: A systematic review. Oral Oncol. 2020, 110, 104885. [Google Scholar] [CrossRef]
- Adeoye, J.; Koohi-Moghadam, M.; Lo, A.W.I.; Tsang, R.K.-Y.; Chow, V.L.Y.; Zheng, L.-W.; Choi, S.-W.; Thomson, P.; Su, Y.-X. Deep learning predicts the malignant-transformation-free survival of oral potentially malignant disorders. Cancers 2021, 13, 6054. [Google Scholar] [CrossRef]
- Kutlu, H.; Avcı, E. A novel method for classifying liver and brain tumors using convolutional neural networks, discrete wavelet transform and long short-term memory networks. Sensors 2019, 19, 1992. [Google Scholar] [CrossRef] [PubMed]
- Farook, T.H.; Haq, T.M.; Ramees, L.; Dudley, J. Predictive modelling of freeway space utilising clinical history, normalised muscle activity, dental occlusion, and mandibular movement analysis. Sci. Rep. 2024, 14, 16423. [Google Scholar] [CrossRef] [PubMed]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 618–626. [Google Scholar]
- Lundberg, S.M.; Lee, S.-I. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017; Curran Associates Inc.: Red Hook, NY, USA, 2017; Volume 30, p. 30. [Google Scholar]
- Norgeot, B.; Quer, G.; Beaulieu-Jones, B.K.; Torkamani, A.; Dias, R.; Gianfrancesco, M.; Arnaout, R.; Kohane, I.S.; Saria, S.; Topol, E.; et al. Minimum information about clinical artificial intelligence modeling: The MI-CLAIM checklist. Nat. Med. 2020, 26, 1320–1324. [Google Scholar] [CrossRef] [PubMed]
- Ribeiro-de-Assis, M.C.F.; Soares, J.P.; de Lima, L.M.; de Barros, L.A.P.; Grão-Velloso, T.R.; Krohling, R.A.; Camisasca, D.R. NDB-UFES: An oral cancer and leukoplakia dataset composed of histopathological images and patient data. Data Brief 2023, 48, 109128. [Google Scholar] [CrossRef]
- James, G.; Witten, D.; Hastie, T.; Tibshirani, R. An Introduction to Statistical Learning; Springer: New York, NY, USA, 2013; Volume 112. [Google Scholar]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
- Farook, T.H.; Jamayet, N.B.; Abdullah, J.Y.; Alam, M.K. Machine learning and intelligent diagnostics in dental and orofacial pain management: A systematic review. Pain Res. Manag. 2021, 2021, 6659133. [Google Scholar] [CrossRef]
- Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
- Cover, T.; Hart, P. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 1967, 13, 21–27. [Google Scholar] [CrossRef]
- Duda, R.O.; Hart, P.E. Pattern Classification and Scene Analysis; Wiley: New York, NY, USA, 1973; Volume 3. [Google Scholar]
- Hosmer, D.W., Jr.; Lemeshow, S.; Sturdivant, R.X. Applied Logistic Regression; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar]
- Cox, D.R. The regression analysis of binary sequences. J. R. Stat. Soc. Ser. B Stat. Methodol. 1958, 20, 215–232. [Google Scholar] [CrossRef]
- Bishop, C.M.; Nasrabadi, N.M. Pattern Recognition and Machine Learning; Springer: New York, NY, USA, 2006; Volume 4. [Google Scholar]
- Holzinger, A.; Biemann, C.; Pattichis, C.S.; Kell, D.B. What do we need to build explainable AI systems for the medical domain? arXiv 2017, arXiv:1712.09923. [Google Scholar]
- Keshava, A.; Gugwad, S.; Baad, R.; Patel, R. Gingival squamous cell carcinoma mimicking as a desquamative lesion. J. Indian Soc. Periodontol. 2016, 20, 75–78. [Google Scholar] [CrossRef] [PubMed]
- Farah, C.S.; Fox, S.A. Dysplastic oral leukoplakia is molecularly distinct from leukoplakia without dysplasia. Oral Dis. 2019, 25, 1715–1723. [Google Scholar] [CrossRef] [PubMed]
- Razzaki, S.; Baker, A.; Perov, Y.; Middleton, K.; Baxter, J.; Mullarkey, D.; Sangar, D.; Taliercio, M.; Butt, M.; Majeed, A.; et al. A comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis. arXiv 2018, arXiv:1806.10698. [Google Scholar]
- Kirton, J.A.; Thompson, W.; Pearce, M.; Brown, J.M. Ability of the wider dental team to triage patients with acute conditions: A qualitative study. Br. Dent. J. 2020, 228, 103–107. [Google Scholar] [CrossRef] [PubMed]
- Liakoni, E.; Edwards, K.C.; St Helen, G.; Nardone, N.; Dempsey, D.A.; Tyndale, R.F.; Benowitz, N.L. Effects of nicotine metabolic rate on withdrawal symptoms and response to cigarette smoking after abstinence. Clin. Pharmacol. Ther. 2019, 105, 641–651. [Google Scholar] [CrossRef]
- Loh, T.; Rubin, A.G.; Jiang, S.I.B. Management of mucosal basal cell carcinoma of the lip: An update and comprehensive review of the literature. Dermatol. Surg. 2016, 42, 1313–1319. [Google Scholar] [CrossRef]
Coded Feature Names | Description |
---|---|
localization_Tongue | Tongue localisation |
localization_Lip | Lip localisation |
localization_Floor of mouth | Floor-of-mouth localisation |
localization_Buccal mucosa | Buccal mucosa localisation |
localization_Palate | Palate localisation |
localization_Gingiva | Gingival localisation |
larger_size | Larger lesions |
tobacco_use_Yes | Current habit of tobacco use |
tobacco_use_Former | History of tobacco use |
tobacco_use_No | No history of tobacco use |
tobacco_use_Not informed | Undisclosed habit of tobacco use |
alcohol_consumption_No | No alcohol consumption |
alcohol_consumption_Former | History of alcohol consumption |
alcohol_consumption_Yes | Current habit of alcohol consumption |
alcohol_consumption_Not informed | Undisclosed habit of alcohol consumption |
sun_exposure_No | No abnormal sun exposure |
sun_exposure_Yes | Abnormal sun exposure |
sun_exposure_Not informed | Sun exposure (not informed) |
gender_M | Male sex |
gender_F | Female sex |
age_group_2 | Age older than 60 years |
age_group_1 | Age between 41 and 60 years |
age_group_0 | Age younger than 40 years |
Class Names | Sample Counts |
---|---|
Leukoplakia without dysplasia | 57 |
Leukoplakia with dysplasia | 89 |
Oral squamous cell carcinoma | 91 |
Models | Precision | Recall | F1 Score | ROC |
---|---|---|---|---|
KNN | 0.75 | 0.67 | 0.71 | 0.86 |
Logistics Regression | 0.76 | 0.72 | 0.74 | 0.92 |
Naive Bayes | 0.43 | 1 | 0.6 | 0.86 |
SVM | 0.86 | 0.67 | 0.75 | 0.76 |
Random Forest | 1 | 0.78 | 0.88 | 0.99 |
Models | Precision | Recall | F1 Score | ROC |
---|---|---|---|---|
KNN | 0.75 | 0.86 | 0.8 | 0.94 |
Logistics Regression | 0.7 | 0.67 | 0.68 | 0.85 |
Naive Bayes | 0.9 | 0.43 | 0.58 | 0.81 |
SVM | 1 | 0.81 | 0.89 | 0.94 |
Random Forest | 0.91 | 1 | 0.95 | 0.99 |
Models | Precision | Recall | F1 Score | ROC |
---|---|---|---|---|
KNN | 0.85 | 0.81 | 0.83 | 0.95 |
Logistics Regression | 0.74 | 0.81 | 0.77 | 0.93 |
Naive Bayes | 0.88 | 0.33 | 0.48 | 0.75 |
SVM | 0.69 | 0.95 | 0.8 | 0.93 |
Random Forest | 0.91 | 1 | 0.95 | 1 |
Models | Overall Accuracy | Kappa Score |
---|---|---|
KNN | 0.78 | 0.67 |
Logistics Regression | 0.73 | 0.6 |
Naive Bayes | 0.57 | 0.37 |
SVM | 0.82 | 0.72 |
Random Forest | 0.93 | 0.9 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alam, S.S.; Ahmed, S.; Farook, T.H.; Dudley, J. Predicting Leukoplakia and Oral Squamous Cell Carcinoma Using Interpretable Machine Learning: A Retrospective Analysis. Oral 2024, 4, 386-404. https://doi.org/10.3390/oral4030032
Alam SS, Ahmed S, Farook TH, Dudley J. Predicting Leukoplakia and Oral Squamous Cell Carcinoma Using Interpretable Machine Learning: A Retrospective Analysis. Oral. 2024; 4(3):386-404. https://doi.org/10.3390/oral4030032
Chicago/Turabian StyleAlam, Salem Shamsul, Saif Ahmed, Taseef Hasan Farook, and James Dudley. 2024. "Predicting Leukoplakia and Oral Squamous Cell Carcinoma Using Interpretable Machine Learning: A Retrospective Analysis" Oral 4, no. 3: 386-404. https://doi.org/10.3390/oral4030032
APA StyleAlam, S. S., Ahmed, S., Farook, T. H., & Dudley, J. (2024). Predicting Leukoplakia and Oral Squamous Cell Carcinoma Using Interpretable Machine Learning: A Retrospective Analysis. Oral, 4(3), 386-404. https://doi.org/10.3390/oral4030032