Benign and Malignant Oral Lesion Image Classification Using Fine-Tuned Transfer Learning Techniques
Abstract
:1. Introduction
- We added extra patch tokens and distillation tokens to the DeIT method for performing the classification effectively and detecting the lesions by providing 98.73% correct information for malignant lesions.
- We modified the structure of VGG19 and MobileNet by changing the fully connected (FC) layer, and both models give 100% accuracy.
- We established a standard comparison table with similar works. Among them, our models outperform on the basis of accuracy.
- Null Hypothesis (: there are no significant differences between the performances of fine-tuned transfer learning models;
- Alternative Hypothesis (): the performances of fine-tuned transfer learning models have statistically significant differences.
2. Literature Review
- The efficacy of fine-tuned TL methods: The purpose of this work is to determine how successfully fine-tuned transfer learning models can be adapted to the job of categorizing oral lesions based on picture data.
- Classification accuracy: This work focuses on the classification process’s correctness, especially in discriminating between benign and malignant oral lesions.
- Comparison to existing works: This study compares the performance of fine-tuned transfer learning approaches to classic or conventional methods for oral lesion categorization.
3. Proposed Methodology
3.1. Dataset Description
3.2. Dataset Splitting and Enhancement
3.3. Transfer Learning Model Design
3.3.1. VGG19 Model
3.3.2. MobileNet Model
3.3.3. DeIT Model
- Interpolation: used to estimate the value of unknown points using known nearby points.
- Convolution: a mathematical operation that combines two functions to create a third function.
- Fourier transform: a mathematical tool representing a signal or image as a sum of sine and cosine waves of different frequencies.
- Resampling: the process of changing the resolution of an image by altering the number of pixels.
- Normalization: the process of adjusting the data values to a common scale.
- Histogram equalization: a contrast adjustment technique used to improve the contrast of an image by redistributing the pixel intensities.
- Edge detection: the process of locating sharp changes in an image.
- Affine transformation: a mathematical tool to transform an image by modifying its shape, size, and orientation.
3.4. Experimental Preparation and Assessment
4. Result Analysis and Discussion
4.1. Cross-Validation
4.2. Discussions
- VGG19 has some significant drawbacks. These include its high memory requirements, inefficiency for installations with limited resources, and high computational complexity. It is made just for picture data, frequently has to be adjusted, and is ineffective in capturing spatial hierarchies. Furthermore, the quality and quantity of training data have a significant impact on its performance, which raises issues with fairness when biased data are present.
- Despite its effectiveness, MobileNet is not without its restrictions. It is not well adapted for tasks like natural language processing or textual data. It may struggle with fine detail detection, be inaccurate in complicated picture recognition, and perform best in generic image categorization. Its small size could make it less capable of handling high resolutions or bigger photos. Finally, understanding its decision-making process might be difficult.
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Life expectancy, W. Bangladesh: Oral Cancer. 2020. Available online: https://www.worldlifeexpectancy.com/bangladesh-oral-cancer (accessed on 10 August 2023).
- Chaturvedi, P.; Singh, A.; Chien, C.Y.; Warnakulasuriya, S. Tobacco related oral cancer. BMJ 2019, 365, l2142. [Google Scholar] [CrossRef] [PubMed]
- WCRF. Mouth and Oral Cancer Statistics. 2022. Available online: https://www.wcrf.org/cancer-trends/mouth-and-oral-cancer-statistics/ (accessed on 15 August 2023).
- Permi, H.S.; Agrawal, T.; Sajitha, K.; Bhat, S.P.; Prasad, H.K.; Srinivas, T. A histomorphological study of benign lesions of oral cavity in a tertiary care hospital. J. Evol. Med. Dent. Sci. 2015, 4, 16495–16499. [Google Scholar] [CrossRef]
- Reid, D.E. Benign Lesions. 2022. Available online: https://www.tricitiesderm.com/general-dermatology/benign-lesions/ (accessed on 15 August 2023).
- Givony, S. Oral squamous cell carcinoma (OSCC) an overview. J. Med. Sci 2020, 8, 67–74. [Google Scholar]
- Prachi, S.; Jitender, S.; Rahul, C.; Jitendra, K.; Priyanka, M.; Disha, S. Impact of oral contraceptives on periodontal health. Afr. Health Sci. 2019, 19, 1795–1800. [Google Scholar] [CrossRef] [PubMed]
- McKinney, R.; Olmo, H. Physical and chemical lesions of the oral mucosa. In StatPearls [Internet]; StatPearls Publishing: Treasure Island, FL, USA, 2023. [Google Scholar]
- Folmsbee, J.; Liu, X.; Brandwein-Weber, M.; Doyle, S. Active deep learning: Improved training efficiency of convolutional neural networks for tissue classification in oral cavity cancer. In Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA, 4–7 April 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 770–773. [Google Scholar]
- Gupta, R.K.; Kaur, M.; Manhas, J. Tissue level based deep learning framework for early detection of dysplasia in oral squamous epithelium. J. Multimed. Inf. Syst. 2019, 6, 81–86. [Google Scholar] [CrossRef]
- Jeyaraj, P.R.; Samuel Nadar, E.R. Computer-assisted medical image classification for early diagnosis of oral cancer employing deep learning algorithm. J. Cancer Res. Clin. Oncol. 2019, 145, 829–837. [Google Scholar] [CrossRef]
- Xu, S.; Liu, C.; Zong, Y.; Chen, S.; Lu, Y.; Yang, L.; Ng, E.Y.; Wang, Y.; Wang, Y.; Liu, Y.; et al. An early diagnosis of oral cancer based on three-dimensional convolutional neural networks. IEEE Access 2019, 7, 158603–158611. [Google Scholar] [CrossRef]
- Rana, A.; Yauney, G.; Wong, L.C.; Gupta, O.; Muftu, A.; Shah, P. Automated segmentation of gingival diseases from oral images. In Proceedings of the 2017 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT), Bethesda, MD, USA, 6–8 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 144–147. [Google Scholar]
- Warin, K.; Limprasert, W.; Suebnukarn, S.; Jinaporntham, S.; Jantana, P. Automatic classification and detection of oral cancer in photographic images using deep learning algorithms. J. Oral Pathol. Med. 2021, 50, 911–918. [Google Scholar] [CrossRef]
- Prajapati, S.A.; Nagaraj, R.; Mitra, S. Classification of dental diseases using CNN and transfer learning. In Proceedings of the 2017 5th International Symposium on Computational and Business Intelligence (ISCBI), Dubai, United Arab Emirates, 11–14 August 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 70–74. [Google Scholar]
- Welikala, R.A.; Remagnino, P.; Lim, J.H.; Chan, C.S.; Rajendran, S.; Kallarakkal, T.G.; Zain, R.B.; Jayasinghe, R.D.; Rimal, J.; Kerr, A.R.; et al. Automated detection and classification of oral lesions using deep learning for early detection of oral cancer. IEEE Access 2020, 8, 132677–132693. [Google Scholar] [CrossRef]
- Prabhakar, S.K.; Rajaguru, H. Performance analysis of linear layer neural networks for oral cancer classification. In Proceedings of the 2017 6th ICT International Student Project Conference (ICT-ISPC), Johor, Malaysia, 23–24 May 2017; IEEE: Piscataway, NJ, USA, 2017. [Google Scholar]
- Nanditha, B.; Geetha Kiran, A.; Dinesh, M.; Murali, S.; Chandrasheka, H. Texture analysis of color oral images for lesion detection. In Proceedings of the 2020 International Conference on Computational Performance Evaluation (ComPE), Shillong, India, 2–4 July 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 067–072. [Google Scholar]
- Thomas, B.; Kumar, V.; Saini, S. Texture analysis based segmentation and classification of oral cancer lesions in color images using ANN. In Proceedings of the 2013 IEEE International Conference on Signal Processing, Computing and Control (ISPCC), Solan, India, 26–28 September 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1–5. [Google Scholar]
- Nanditha, B.R.; Geetha, A.; Chandrashekar, H.S.; Dinesh, M.S.; Murali, S. An Ensemble Deep Neural Network Approach for Oral Cancer Screening. Int. J. Online Biomed. Eng. 2021, 17, 121–134. [Google Scholar] [CrossRef]
- Shamim, M.Z.M.; Sadatullah, S.; Shiblee, M.; Usman, M.; Ali, S.K. Automated detection of oral pre-cancerous tongue lesions using deep learning for early diagnosis of oral cavity cancer. Comput. J. 2022, 65, 91–104. [Google Scholar] [CrossRef]
- Wu, M.P.; Hsu, G.; Varvares, M.A.; Crowson, M.G. Predicting Progression of Oral Lesions to Malignancy Using Machine Learning. Laryngoscope 2022, 133, 1156–1162. [Google Scholar] [CrossRef] [PubMed]
- Bansal, K.; Bathla, R.; Kumar, Y. Deep transfer learning techniques with hybrid optimization in early prediction and diagnosis of different types of oral cancer. Soft Comput. 2022, 26, 11153–11184. [Google Scholar] [CrossRef]
- Marzouk, R.; Alabdulkreem, E.; Dhahbi, S.; Nour, M.K.; Duhayyim, M.A.; Othman, M.; Hamza, M.A.; Motwakel, A.; Yaseen, I.; Rizwanullah, M. Deep Transfer Learning Driven Oral Cancer Detection and Classification Model. Comput. Mater. Contin. 2022, 73, 3905–3920. [Google Scholar] [CrossRef]
- Rahman, A.U.; Alqahtani, A.; Aldhafferi, N.; Nasir, M.U.; Khan, M.F.; Khan, M.A.; Mosavi, A. Histopathologic oral cancer prediction using oral squamous cell carcinoma biopsy empowered with transfer learning. Sensors 2022, 22, 3833. [Google Scholar] [CrossRef] [PubMed]
- Chandrashekar, H.S.; Geetha Kiran, A.; Murali, S.; Dinesh, M.S.; Nanditha, B.R. Oral Images Dataset, Mendeley Data, V2. 2021. Available online: https://data.mendeley.com/datasets/mhjyrn35p4/2 (accessed on 10 July 2023).
- Islam, M.M.; Hossain, M.B.; Akhtar, M.N.; Moni, M.A.; Hasan, K.F. CNN based on transfer learning models using data augmentation and transformation for detection of concrete crack. Algorithms 2022, 15, 287. [Google Scholar] [CrossRef]
- Islam, M.M.; Uddin, M.R.; Ferdous, M.J.; Akter, S.; Akhtar, M.N. BdSLW-11: Dataset of Bangladeshi sign language words for recognizing 11 daily useful BdSL words. Data Brief 2022, 45, 108747. [Google Scholar] [CrossRef]
- Hossain, M.B.; Iqbal, S.H.S.; Islam, M.M.; Akhtar, M.N.; Sarker, I.H. Transfer learning with fine-tuned deep CNN ResNet50 model for classifying COVID-19 from chest X-ray images. Inform. Med. Unlocked 2022, 30, 100916. [Google Scholar] [CrossRef]
- Iman, M.; Arabnia, H.R.; Rasheed, K. A Review of Deep Transfer Learning and Recent Advancements. Technologies 2023, 11, 40. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2015, arXiv:cs.CV/1409.1556. [Google Scholar]
- Islam, M.M.; Uddin, M.R.; AKhtar, M.N.; Alam, K.R. Recognizing multiclass Static Sign Language words for deaf and dumb people of Bangladesh based on transfer learning techniques. Inform. Med. Unlocked 2022, 33, 101077. [Google Scholar] [CrossRef]
- Wang, S.; Zheng, R.; Wang, L.; Zhang, M. A MobileNet Based Model for Tongue Shape Classification. In Proceedings of the Cognitive Systems and Information Processing: 7th International Conference, ICCSIP 2022, Fuzhou, China, 17–18 December 2022; Revised Selected Papers. Springer: Berlin/Heidelberg, Germany, 2023; pp. 605–616. [Google Scholar]
- Matsoukas, C.; Haslum, J.F.; Sorkhei, M.; Söderberg, M.; Smith, K. What makes transfer learning work for medical images: Feature reuse & other factors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 9225–9234. [Google Scholar]
- Zabin, M.; Choi, H.J.; Islam, M.M.; Uddin, J. Impact of Tuning Parameters in Deep Convolutional Neural Network Using a Crack Image Dataset. In Proceedings of the 15th KIPS International Conference on Ubiquitous Information Technologies and Applications (CUTE 2021), Jeju, Repubilc of Korea, 15–17 December 2021. [Google Scholar]
Reference | Year | Model | Accuracy |
---|---|---|---|
Welikala et al. [16] | 2020 | ResNet-101 | 87.07% |
Prabhakar and Rajaguru [17] | 2017 | TNM | 90.88% |
Nanditha et al. [18] | 2020 | BPNN | 95% |
Thomas et al. [19] | 2013 | ANN11 | 97.92% |
Nanditha [20] | 2021 | VGG16 | 96.2% |
Shamim et al. [21] | 2019 | VGG19 | 96.7% |
Wu et al. [22] | 2019 | AlexNet | 91% |
Proposed | 2023 | VGG19 and MobileNet | 100% |
Parameter | Parameter Value |
---|---|
Batch size | 32 |
Weight matrix | imagenet |
Activation function | softmax |
Criterion | sparse_categorical_crossentropy |
Optimizer | Adam |
Parameter | Parameter Value |
---|---|
Batch size | 32 |
Weight matrix | Imagenet |
Activation function (dense: 1024) | ReLU |
Activation function (dense: 512) | ReLU |
Activation function (dense: 2) | softmax |
Criterion | categorical_crossentropy |
Optimizer | Adam |
Learning Rate | 0.02 |
Parameter | Parameter Value |
---|---|
Batch size | 128 |
Weight matrix | Linear |
Activation function | ReLU |
Regularization | Dropout |
Criterion | LabelSmoothingCrossEntropy |
Optimizer | Adam |
Learning rate | 0.001 |
Model | P (%) | R (%) | F1 (%) | MCC (%) | Accuracy (%) |
---|---|---|---|---|---|
DeIT | 98.73 | 94.24 | 96.38 | 93.49 | 96.40 |
VGG19 | 100 | 100 | 100 | 100 | 100 |
MobileNet | 100 | 100 | 100 | 100 | 100 |
Model | P (%) | R (%) | F1 (%) | MCC (%) | Accuracy (%) |
---|---|---|---|---|---|
DeIT | 98.75 | 94.5 | 96.54 | 93.58 | 96.70 |
VGG19 | 100 | 100 | 100 | 100 | 100 |
MobileNet | 100 | 100 | 100 | 100 | 100 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Islam, M.M.; Alam, K.M.R.; Uddin, J.; Ashraf, I.; Samad, M.A. Benign and Malignant Oral Lesion Image Classification Using Fine-Tuned Transfer Learning Techniques. Diagnostics 2023, 13, 3360. https://doi.org/10.3390/diagnostics13213360
Islam MM, Alam KMR, Uddin J, Ashraf I, Samad MA. Benign and Malignant Oral Lesion Image Classification Using Fine-Tuned Transfer Learning Techniques. Diagnostics. 2023; 13(21):3360. https://doi.org/10.3390/diagnostics13213360
Chicago/Turabian StyleIslam, Md. Monirul, K. M. Rafiqul Alam, Jia Uddin, Imran Ashraf, and Md Abdus Samad. 2023. "Benign and Malignant Oral Lesion Image Classification Using Fine-Tuned Transfer Learning Techniques" Diagnostics 13, no. 21: 3360. https://doi.org/10.3390/diagnostics13213360
APA StyleIslam, M. M., Alam, K. M. R., Uddin, J., Ashraf, I., & Samad, M. A. (2023). Benign and Malignant Oral Lesion Image Classification Using Fine-Tuned Transfer Learning Techniques. Diagnostics, 13(21), 3360. https://doi.org/10.3390/diagnostics13213360