Evaluating Brain Tumor Detection with Deep Learning Convolutional Neural Networks Across Multiple MRI Modalities
Abstract
:1. Introduction
2. Materials and Methods
2.1. Dataset Collection and Preparation
2.2. Image Preprocessing
- p is the original pixel intensity value;
- μ is the mean of all pixel values in the image;
- σ is the standard deviation of all pixel values in the image.
2.3. CNN Models—Transfer Learning
- A global average pooling layer;
- First Dense Layer with 96 neurons, RELU activation function and kernel regularizes l1 = 0.3 and l2 = 0.3;
- Dropout Layer with 40% random neuron rejection;
- Batch normalization Layer;
- Second Dense Layer with 96 neurons, RELU activation functions, and kernel regularizes l1 = 0.5 and l2 = 0.4;
- Dropout Layer with 40% random neuron rejection;
- Batch normalization Layer;
- SoftMax Dense Layer with 2 classes of output.
2.4. Model’s Evaluation Metrics and Methods
- Precision = TP/(TP + TN)%;
- Sensitivity = TP/(TP + FN)%;
- Specificity = TN/(TN + FN)%;
- Accuracy = (TP + TN)/(TP + TN + FP + FN)%.
3. Results
4. Discussion
4.1. Sequence-Wise Analysis
4.2. Model-Wise Analysis
4.3. Explainability and Clinical Correlation
5. Limitations-Future Work
6. Conclusions
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Wild, C. World Cancer Report 2014; Wild, C.P., Stewart, B.W., Eds.; World Health Organization: Geneva, Switzerland, 2014. [Google Scholar]
- Louis, D.N.; Perry, A.; Reifenberger, G.; Von Deimling, A.; Figarella-Branger, D.; Cavenee, W.K.; Ohgaki, H.; Wiestler, O.D.; Kleihues, P.; Ellison, D.W. The 2016 World Health Organization Classification of Tumors of the Central Nervous System: A summary. Acta Neuropathol. 2016, 131, 803–820. [Google Scholar] [CrossRef] [PubMed]
- Mardor, Y.; Pfeffer, R.; Spiegelmann, R.; Roth, Y.; Maier, S.E.; Nissim, O.; Berger, R.; Glicksman, A.; Baram, J.; Orenstein, A.; et al. Early detection of response to radiation therapy in patients with brain malignancies using conventional and high b-value diffusion-weighted magnetic resonance imaging. J. Clin. Oncol. 2003, 21, 1094–1100. [Google Scholar] [CrossRef] [PubMed]
- Castillo, M. History and evolution of brain tumor imaging: Insights through radiology. Radiology 2014, 273, S111–S125. [Google Scholar] [CrossRef]
- Hashemi, R.H.; Bradley, W.G.; Lisanti, C.J. MRI: The Basics: The Basics; Lippincott Williams & Wilkins: Philadelphia, PA, USA, 2012. [Google Scholar]
- Jackson, E.F.; Ginsberg, L.E.; Schomer, D.F.; Leeds, N.E. A review of MRI pulse sequences and techniques in neuroimaging. Surg. Neurol. 1997, 47, 185–199. [Google Scholar] [CrossRef] [PubMed]
- Drevelegas, A.; Papanikolaou, N. Imaging modalities in brain tumors. In Imaging of Brain Tumors with Histological Correlations; Springer: Berlin/Heidelberg, Germany, 2011; pp. 13–33. [Google Scholar]
- Chokshi, F.H.; Hughes, D.R.; Wang, J.M.; Mullins, M.E.; Hawkins, C.M.; Duszak, R. Diagnostic radiology resident and fellow workloads: A 12-year longitudinal trend analysis using national Medicare aggregate claims data. J. Am. Coll. Radiol. 2015, 12, 664–669. [Google Scholar] [CrossRef]
- Huang, Z.; Xu, H.; Su, S.; Wang, T.; Luo, Y.; Zhao, X.; Liu, Y.; Song, G.; Zhao, Y. A computer-aided diagnosis system for brain magnetic resonance imaging images using a novel differential feature neural network. Comput. Biol. Med. 2020, 121, 103818. [Google Scholar] [CrossRef]
- Ramaha, N.T.A.; Mahmood, R.M.; Hameed, A.A.; Fitriyani, N.L.; Alfian, G.; Syafrudin, M. Brain pathology classification of mr images using machine learning techniques. Computers 2023, 12, 167. [Google Scholar] [CrossRef]
- Huang, X.; Liu, Y.; Li, Y.; Qi, K.; Gao, A.; Zheng, B.; Liang, D.; Long, X. Deep learning-based multiclass brain tissue segmentation in Fetal MRIs. Sensors 2023, 23, 655. [Google Scholar] [CrossRef]
- Ahmmed, S.; Podder, P.; Mondal, M.R.H.; Rahman, S.M.A.; Kannan, S.; Hasan, J.; Rohan, A.; Prosvirin, A.E. Enhancing brain tumor classification with transfer learning across multiple classes: An in-depth analysis. BioMedInformatics 2023, 3, 1124–1144. [Google Scholar] [CrossRef]
- Kaur, T.; Gandhi, T.K. Automated Brain Image Classification Based on VGG-16 and Transfer Learning. In Proceedings of the 2019 International Conference on Information Technology (ICIT), Bhubaneswar, India, 19–21 December 2019; pp. 94–98. [Google Scholar] [CrossRef]
- Kumar, S.; Choudhary, S.; Jain, A.; Singh, K.; Ahmadian, A.; Bajuri, M.Y. Brain tumor classification using deep neural network and transfer learning. Brain Topogr. 2023, 36, 305–318. [Google Scholar] [CrossRef]
- Amarnath, A.; Al Bataineh, A.; Hansen, J.A. Transfer-Learning Approach for Enhanced Brain Tumor Classification in MRI Imaging. BioMedInformatics 2024, 4, 1745–1756. [Google Scholar] [CrossRef]
- Talo, M.; Baloglu, U.B.; Yıldırım, Ö.; Acharya, U.R. Application of deep transfer learning for automated brain abnormality classification using MR images. Cogn. Syst. Res. 2019, 54, 176–188. [Google Scholar] [CrossRef]
- Lu, S.; Lu, Z.; Zhang, Y.-D. Pathological brain detection based on AlexNet and transfer learning. J. Comput. Sci. 2019, 30, 41–47. [Google Scholar] [CrossRef]
- Bernal, J.; Kushibar, K.; Asfaw, D.S.; Valverde, S.; Oliver, A.; Martí, R.; Lladó, X. Deep Convolutional Neural Networks for Brain Image Analysis on Magnetic Resonance Imaging: A Review. Artif. Intell. Med. 2019, 95, 64–81. [Google Scholar] [CrossRef] [PubMed]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L.C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Morid, M.A.; Borjali, A.; Del Fiol, G. A scoping review of transfer learning research on medical image analysis using ImageNet. arXiv 2020, arXiv:2004.13175. [Google Scholar] [CrossRef]
- Deepak, S.; Ameer, P.M. Brain tumor classification using deep CNN features via transfer learning. Comput. Biol. Med. 2019, 111, 103345. [Google Scholar] [CrossRef]
- Tan, C.; Sun, F.; Kong, T.; Zhang, W.; Yang, C.; Liu, C. A Survey on Deep Transfer Learning. In Artificial Neural Networks and Machine Learning—ICANN 2018. ICANN 2018; Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I., Eds.; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2018; Volume 11141. [Google Scholar]
- Bianco, S.; Cadene, R.; Celona, L.; Napoletano, P. Benchmark analysis of representative deep neural network architectures. IEEE Access 2018, 6, 64270–64277. [Google Scholar] [CrossRef]
- Shin, H.-C.; Roth, H.R.; Gao, M.; Lu, L.; Xu, Z.; Nogues, I.; Yao, J.; Mollura, D.; Summers, R.M. Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning. IEEE Trans. Med. Imaging 2016, 35, 1285–1298. [Google Scholar] [CrossRef]
- Chelghoum, R.; Ikhlef, A.; Hameurlaine, A.; Jacquir, S. Transfer Learning Using Convolutional Neural Network Architectures for Brain Tumor Classification from MRI Images. In Artificial Intelligence Applications and Innovations: Proceedings of the 16th IFIP WG 12.5 International Conference, AIAI 2020, Neos Marmaras, Greece, 5–7 June 2020; Proceedings, Part I; Springer International Publishing: Cham, Switzerland, 2020; Volume 583, pp. 189–200. [Google Scholar] [CrossRef]
- Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 2015, 115, 211–252. [Google Scholar] [CrossRef]
- Available online: https://keras.io/api/applications (accessed on 25 May 2024).
- Villanueva-Meyer, J.E.; Mabray, M.C.; Cha, S. Current clinical brain tumor imaging. Neurosurgery 2017, 81, 397–415. [Google Scholar] [CrossRef]
- van Dijken, B.R.; van Laar, P.J.; Holtman, G.A.; van der Hoorn, A. Diagnostic accuracy of magnetic resonance imaging techniques for treatment response evaluation in patients with head and neck tumors, a systematic review and meta-analysis. PLoS ONE 2017, 12, e0177986. [Google Scholar]
- Widmann, G.; Henninger, B.; Kremser, C.; Jaschke, W. MRI sequences in head & neck radiology–state of the art. In RöFo-Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren; © Georg Thieme Verlag KG: Leipzig, Germany, 2017; Volume 189. [Google Scholar]
- Dirix, P.; Haustermans, K.; Vandecaveye, V. The value of magnetic resonance imaging for radiotherapy planning. In Seminars in Radiation Oncology; WB Saunders: Philadelphia, PA, USA, 2014; Volume 24. [Google Scholar]
- Bauer, S.; Wiest, R.; Nolte, L.-P.; Reyes, M. A survey of MRI-based medical image analysis for brain tumor studies. Phys. Med. Biol. 2013, 58, R97. [Google Scholar] [CrossRef] [PubMed]
- Lundervold, A.S.; Lundervold, A. An overview of deep learning in medical imaging focusing on MRI. Z. Med. Phys. 2019, 29, 102–127. [Google Scholar] [CrossRef] [PubMed]
- Toğaçar, M.; Ergen, B.; Cömert, Z. BrainMRNet: Brain tumor detection using magnetic resonance images with a novel convolutional neural network model. Med. Hypotheses 2020, 134, 109531. [Google Scholar] [CrossRef]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017. [Google Scholar]
- Tan, M.; Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA, 9–15 June 2019. [Google Scholar]
- Dosovitskiy, A. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
- Liu, Z.; Mao, H.; Wu, C.Y.; Feichtenhofer, C.; Darrell, T.; Xie, S. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022. [Google Scholar]
52 Patients Train 10 Patients Test | Class | |||||
---|---|---|---|---|---|---|
Tumour | Normal | Total | ||||
Train | Test | Train | Test | |||
Sequences | ADC | 146 | 30 | 49 | 16 | 241 |
Diffusion | 142 | 36 | 63 | 17 | 258 | |
FLAIR | 225 | 36 | 107 | 27 | 395 | |
T1 | 78 | 28 | 62 | 20 | 188 | |
T1+C | 131 | 37 | 100 | 20 | 288 | |
T2 | 155 | 32 | 66 | 23 | 276 | |
Total | 877 | 199 | 447 | 123 | 1646 |
Model | Network Depth | Size (Mb) | Parameters | Characteristic Structure | Top-1 % Error at ImageNet |
---|---|---|---|---|---|
VGG16 | 23 | 528 | 138,357,544 | Stacked Convolution Blocks | 0.713 |
MobilenetV2 | 88 | 14 | 3,538,984 | Inverted Residuals and Linear Bottlenecks | 0.713 |
Resnet50 | 50 | 98 | 25,636,712 | Residual Layers | 0.749 |
InceptionV3 | 159 | 92 | 23,851,784 | Concatenated Different-Sized Convolutional Filters | 0.779 |
Image Size | 224 × 224 × 3 |
Training Epochs | 600 |
Batch Size | 16 |
Loss Function | Categorical Cross-Entropy |
Optimizer | Adam |
Learning Rate | 0.00001 |
Choosing Criteria | The model with the best test accuracy |
Accuracy | FLAIR | T1+C | ADC | T1 | Diffusion | T2 | Avg/Sequence | SD |
---|---|---|---|---|---|---|---|---|
VGG16 | 0.984 | 0.965 | 0.891 | 0.833 | 0.811 | 0.836 | 0.887 | 0.073 |
MobnetV2 | 0.921 | 0.930 | 0.957 | 0.896 | 0.830 | 0.800 | 0.889 | 0.061 |
Resnet50 | 0.968 | 0.947 | 0.891 | 0.833 | 0.887 | 0.836 | 0.894 | 0.056 |
InceptionV3 | 0.937 | 0.895 | 0.870 | 0.938 | 0.849 | 0.800 | 0.881 | 0.053 |
Avg/Model | 0.952 | 0.934 | 0.902 | 0.875 | 0.844 | 0.818 | ||
SD | 0.029 | 0.030 | 0.038 | 0.051 | 0.032 | 0.021 |
Sensitivity | FLAIR | T1+C | ADC | T1 | Diffusion | T2 | Avg/Sequence | SD |
---|---|---|---|---|---|---|---|---|
VGG16 | 0.972 | 0.946 | 1.000 | 1.000 | 0.972 | 0.781 | 0.945 | 0.083 |
MobilenetV2 | 0.944 | 0.919 | 0.967 | 1.000 | 0.917 | 0.750 | 0.916 | 0.087 |
Resnet50 | 1.000 | 1.000 | 0.900 | 1.000 | 0.944 | 0.813 | 0.943 | 0.076 |
InceptionV3 | 0.972 | 0.865 | 0.967 | 1.000 | 0.889 | 0.781 | 0.912 | 0.083 |
Avg/Model | 0.972 | 0.932 | 0.958 | 1.000 | 0.931 | 0.781 | ||
SD | 0.023 | 0.056 | 0.042 | 0 | 0.036 | 0.026 |
Specificity | FLAIR | T1+C | ADC | T1 | Diffusion | T2 | Avg/Sequence | SD |
---|---|---|---|---|---|---|---|---|
VGG16 | 1.000 | 1.000 | 0.688 | 0.600 | 0.471 | 0.913 | 0.779 | 0.224 |
MobilenetV2 | 0.889 | 0.950 | 0.938 | 0.750 | 0.647 | 0.870 | 0.841 | 0.118 |
Resnet50 | 0.926 | 0.850 | 0.875 | 0.600 | 0.765 | 0.870 | 0.814 | 0.117 |
InceptionV3 | 0.889 | 0.950 | 0.688 | 0.850 | 0.765 | 0.826 | 0.828 | 0.093 |
Avg/Model | 0.926 | 0.938 | 0.797 | 0.700 | 0.662 | 0.870 | ||
SD | 0.052 | 0.063 | 0.129 | 0.122 | 0.139 | 0.035 |
Precision | FLAIR | T1+C | ADC | T1 | Diffusion | T2 | Avg/Sequence | SD |
---|---|---|---|---|---|---|---|---|
VGG16 | 1.000 | 1.000 | 0.860 | 0.780 | 0.800 | 0.930 | 0.895 | 0.097 |
MobilenetV2 | 0.920 | 0.970 | 0.970 | 0.850 | 0.850 | 0.890 | 0.908 | 0.055 |
Resnet50 | 0.950 | 0.930 | 0.930 | 0.780 | 0.890 | 0.900 | 0.897 | 0.061 |
InceptionV3 | 0.920 | 0.970 | 0.850 | 0.900 | 0.890 | 0.860 | 0.898 | 0.044 |
Avg/Model | 0.948 | 0.968 | 0.903 | 0.828 | 0.858 | 0.895 | ||
SD | 0.038 | 0.029 | 0.057 | 0.059 | 0.043 | 0.029 |
AUC scores | FLAIR | T1+C | ADC | T1 | Diffusion | T2 | Avg/Sequence | SD |
---|---|---|---|---|---|---|---|---|
VGG16 | 1.000 | 0.991 | 0.887 | 0.875 | 0.755 | 0.867 | 0.896 | 0.091 |
MobilenetV2 | 0.965 | 0.978 | 0.983 | 0.925 | 0.853 | 0.852 | 0.926 | 0.060 |
Resnet50 | 0.965 | 0.957 | 0.844 | 0.818 | 0.907 | 0.894 | 0.898 | 0.059 |
InceptionV3 | 0.924 | 0.93 | 0.819 | 0.900 | 0.864 | 0.852 | 0.882 | 0.044 |
Avg/Model | 0.964 | 0.964 | 0.883 | 0.880 | 0.845 | 0.866 | ||
SD | 0.031 | 0.027 | 0.072 | 0.046 | 0.064 | 0.020 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Stathopoulos, I.; Serio, L.; Karavasilis, E.; Kouri, M.A.; Velonakis, G.; Kelekis, N.; Efstathopoulos, E. Evaluating Brain Tumor Detection with Deep Learning Convolutional Neural Networks Across Multiple MRI Modalities. J. Imaging 2024, 10, 296. https://doi.org/10.3390/jimaging10120296
Stathopoulos I, Serio L, Karavasilis E, Kouri MA, Velonakis G, Kelekis N, Efstathopoulos E. Evaluating Brain Tumor Detection with Deep Learning Convolutional Neural Networks Across Multiple MRI Modalities. Journal of Imaging. 2024; 10(12):296. https://doi.org/10.3390/jimaging10120296
Chicago/Turabian StyleStathopoulos, Ioannis, Luigi Serio, Efstratios Karavasilis, Maria Anthi Kouri, Georgios Velonakis, Nikolaos Kelekis, and Efstathios Efstathopoulos. 2024. "Evaluating Brain Tumor Detection with Deep Learning Convolutional Neural Networks Across Multiple MRI Modalities" Journal of Imaging 10, no. 12: 296. https://doi.org/10.3390/jimaging10120296
APA StyleStathopoulos, I., Serio, L., Karavasilis, E., Kouri, M. A., Velonakis, G., Kelekis, N., & Efstathopoulos, E. (2024). Evaluating Brain Tumor Detection with Deep Learning Convolutional Neural Networks Across Multiple MRI Modalities. Journal of Imaging, 10(12), 296. https://doi.org/10.3390/jimaging10120296