Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model
Abstract
:1. Introduction
- The UNet model is modified by changing the number of filters and feature map dimensions from the first layer to the last layer for deep feature extraction. Moreover, the depth of the UNet model is also enhanced by adding one more convolution block to the encoder as well as the decoder section. To accurately identify the glomerular position in the kidney images, two convolution layers, one batch normalization layer, and one max pooling layer were added to the encoder, and one convolutional layer, one upsampling layer, and one concatenate layer were added to the decoder.
- To achieve better results, the proposed model was tuned with different hyperparameters like optimizers, epochs, and batch sizes.
- The performance of the proposed model was evaluated in terms of accuracy, precision, recall, and F1-score. Moreover, its performance is compared with different state-of-the-art models.
2. Related Work
3. Material and Methods
3.1. Datasets Description
3.2. Extraction of Tiles from Whole-Slide Images
3.3. Data Augmentation of Glomeruli Tiles
3.4. Proposed Modified UNet Model Implementation
4. Result Analysis
4.1. Analysis Using Different Optimizers
4.2. Analysis Using Different Batch Sizes with Adam Optimizer
4.3. Analysis Using Different Epochs with Adam Optimizer and Batch Size 8
4.4. Analysis of Proposed Modified UNet Model with Adam Optimizer, Batch Size 8 and Epochs 50
4.4.1. Visual Analysis Based on Predicted Masks
4.4.2. Analysis Based on Confusion Matrices
4.5. Comparison with State of the Art
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Iseki, K.; Miyasato, F.; Uehara, H.; Tokuyama, K.; Toma, S.; Nishime, K.; Yoshi, S.; Shiohira, Y.; Oura, T.; Tozawa, M.; et al. Outcome study of renal biopsy patients. Kidney Int. 2004, 66, 914–919. [Google Scholar] [CrossRef] [PubMed]
- Ledbetter, D.; Ho, L.; Lemley, K.V. Prediction of kidney function from biopsy images using convolutional neural networks. arXiv 2017, arXiv:1702.01816. [Google Scholar] [CrossRef]
- Fuchs, T.J.; Buhmann, J.M. Computational pathology: Challenges and promises for tissue analysis. Comput. Med. Imaging Graph. 2011, 35, 515–530. [Google Scholar] [CrossRef] [PubMed]
- Jin, K.; Yan, Y.; Wang, S.; Yang, C.; Chen, M.; Liu, X.; Terasaki, H.; Yeo, T.-H.; Singh, N.G.; Wang, Y.; et al. iERM: An interpretable deep learning system to classify epiretinal membrane for different optical coherence tomography devices: A multi-center analysis. J. Clin. Med. 2023, 12, 400. [Google Scholar] [CrossRef] [PubMed]
- Zaidi, S.S.A.; Ansari, M.S.; Aslam, A.; Kanwal, N.; Asghar, M.; Lee, B. A survey of modern deep learning based object detection models. Digit. Signal Process. 2022, 126, 103514. [Google Scholar] [CrossRef]
- Anderson, J.; Glynn, L.G. Definition of chronic kidney disease and measurement of kidney function in original research papers: A review of the literature. Nephrol. Dial. Transplant. 2011, 26, 2793–2798. [Google Scholar] [CrossRef] [PubMed]
- Kimmelstiel, P.; Wilson, C. Intercapillary lesions in the glomeruli of the kidney. Am. J. Pathol. 1936, 12, 83. [Google Scholar]
- Bohr, A.; Memarzadeh, K. The rise of artificial intelligence in healthcare applications. In Artificial Intelligence in Healthcare; Academic Press: Cambridge, MA, USA, 2020; pp. 25–60. [Google Scholar]
- Gao, Z.; Pan, X.; Shao, J.; Jiang, X.; Su, Z.; Jin, K.; Ye, J. Automatic interpretation and clinical evaluation for fundus fluorescein angiography images of diabetic retinopathy patients by deep learning. Br. J. Ophthalmol. 2022. [Google Scholar] [CrossRef]
- Mo, Y.; Wu, Y.; Yang, X.; Liu, F.; Liao, Y. Review the state-ofthe-art technologies of semantic segmentation based on deep learning. Neurocomputing 2022, 493, 626–646. [Google Scholar] [CrossRef]
- Cascarano, G.D.; Debitonto, F.S.; Lemma, R.; Brunetti, A.; Buongiorno, D.; De Feudis, I.; Guerriero, A.; Venere, U.; Matino, S.; Rocchetti, M.T.; et al. A neural network for glomerulus classification based on histological images of kidney biopsy. BMC Med. Inform. Decis. Mak. 2021, 21, 300. [Google Scholar] [CrossRef]
- Kannan, S. Segmentation of Glomeruli Within Trichrome Images Using Deep Learning. Kidney Int. Rep. 2019, 7, 955–962. [Google Scholar] [CrossRef] [PubMed]
- Zeng, C.; Nan, Y.; Xu, F.; Lei, Q.; Li, F.; Chen, T.; Liang, S.; Hou, X.; Lv, B.; Liang, D.; et al. Identification of glomerular lesions and intrinsic glomerular cell types in kidney diseases via deep learning. J. Pathol. 2020, 252, 53–64. [Google Scholar] [CrossRef] [PubMed]
- Chandan, B.K.; Jayachandran, A. Identifying glomeruli in human kidney tissue images using pattern recognition methods. Eur. J. Mol. Clin. Med. 2022, 3. [Google Scholar]
- Gu, Y.; Ruan, R.; Yan, Y.; Zhao, J.; Sheng, W.; Liang, L.; Huang, B. Glomerulus Semantic Segmentation Using Ensemble of Deep Learning Models. Arab. J. Sci. Eng. 2022, 47, 14013–14024. [Google Scholar] [CrossRef]
- Han, X.; Zhang, G.; Wang, X. Glomerular Microscopic Image Segmentation Based on Convolutional Neural Network. In Proceedings of the 2019 Chinese Control Conference (CCC), Guangzhou, China, 27–30 July 2019. [Google Scholar]
- Altini, N.; Cascarano, G.D.; Brunetti, A.; Marino, F.; Rocchetti, M.T.; Matino, S.; Venere, U.; Rossini, M.; Pesce, F.; Gesualdo, L.; et al. Semantic segmentation framework for glomeruli detection and classification in kidney histological sections. Electronics 2020, 9, 503. [Google Scholar] [CrossRef]
- Gallego, J.; Pedraza, A.; Lopez, S.; Steiner, G.; Gonzalez, L.; Laurinavicius, A.; Bueno, G. Glomerulus classification and detection based on convolutional neural networks. J. Imaging 2018, 4, 20. [Google Scholar] [CrossRef]
- Gadermayr, M.; Dombrowski, A.K.; Klinkhammer, B.M.; Boor, P.; Merhof, D. CNN cascades for segmenting sparse objects in gigapixel whole slide images. Comput. Med. Imaging Graph. 2019, 71, 40–48. [Google Scholar] [CrossRef]
- Kato, T.; Relator, R.; Ngouv, H.; Hirohashi, Y.; Takaki, O.; Kakimoto, T.; Okada, K. Segmental HOG: New descriptor for glomerulus detection in kidney microscopy image. BMC Bioinform. 2015, 16, 316. [Google Scholar] [CrossRef]
- Temerinac-Ott, M.; Forestier, G.; Schmitz, J.; Hermsen, M.; Bräsen, J.H.; Feuerhake, F.; Wemmert, C. Detection of glomeruli in renal pathology by mutual comparison of multiple staining modalities. In Proceedings of the 10th International Symposium on Image and Signal Processing and Analysis, Ljubljana, Slovenia, 18–20 September 2017; pp. 19–24. [Google Scholar]
- Ginley, B.; Lutnick, B.; Jen, K.-Y.; Fogo, A.B.; Jain, S.; Rosenberg, A.; Walavalkar, V.; Wilding, G.; Tomaszewski, J.E.; Yacoub, R.; et al. Computational segmentation and classification of diabetic glomerulosclerosis. J. Am. Soc. Nephrol. 2019, 30, 1953–1967. [Google Scholar] [CrossRef]
- Saikia, F.N.; Iwahori, Y.; Suzuki, T.; Bhuyan, M.K.; Wang, A.; Kijsirikul, B. MLP-UNet: Glomerulus Segmentation; IEEE Access: Piscataway, NJ, USA, 2023. [Google Scholar]
- Shubham, S.; Jain, N.; Gupta, V.; Mohan, S.; Ariffin, M.M.; Ahmadian, A. Identify glomeruli in human kidney tissue images using a deep learning approach. Soft Comput. 2023, 27, 2705–2716. [Google Scholar] [CrossRef]
- Li, X.; Davis, R.C.; Xu, Y.; Wang, Z.; Souma, N.; Sotolongo, G.; Bell, J.; Ellis, M.; Howell, D. Deep Learning Segmentation of Glomeruli on Kidney Donor Frozen Sections. J. Med. Imaging 2021, 8, 067501. [Google Scholar] [CrossRef] [PubMed]
- Jiang, L.; Chen, W.; Dong, B.; Mei, K.; Zhu, C.; Liu, J.; Cai, M.; Yan, Y.; Wang, G.; Zuo, L.; et al. A deep learning-based approach for glomeruli instance segmentation from multistained renal biopsy pathologic images. Am. J. Pathol. 2021, 191, 1431–1441. [Google Scholar] [CrossRef] [PubMed]
- Available online: https://www.kaggle.com/competitions/hubmap-kidney-segmentation/ (accessed on 17 November 2020).
- Luo, Z.; Zhang, Y.; Zhou, L.; Zhang, B.; Luo, J.; Wu, H. Micro-vessel image segmentation based on the AD-UNet model. IEEE Access 2019, 7, 143402–143411. [Google Scholar] [CrossRef]
- Anand, V.; Gupta, S.; Koundal, D.; Nayak, S.R.; Nayak, J.; Vimal, S. Multi-class skin disease classification using transfer learning model. Int. J. Artif. Intell. Tools 2022, 31, 2250029. [Google Scholar] [CrossRef]
- Bhatia, V.; Choudhary, S.; Ramkumar, K.R. A comparative study on various intrusion detection techniques using machine learning and neural network. In Proceedings of the 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India, 4–5 June 2020. [Google Scholar]
- Lodhi, S.; Kukreja, V. Deep Neural Network for Recognition of Enlarged Mathematical Corpus. In Proceedings of the 2022 International Conference on Decision Aid Sciences and Applications (DASA), Chiangrai, Thailand, 23–25 March 2022. [Google Scholar]
- Hung, A.J.; Chen, J.; Che, Z.; Nilanon, T.; Jarc, A.; Titus, M.; Oh, P.J.; Gill, I.S.; Liu, Y. Utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes. J. Endourol. 2018, 32, 438–444. [Google Scholar] [CrossRef]
- Gupta, I.; Gupta, S.; Singh, S. Architectures Based on Deep Learning for the Detection of Invasive Ductal Carcinoma. ECS Trans. 2022, 107, 5469. [Google Scholar] [CrossRef]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Sutskever, I.; Martens, J.; Dahl, G.; Hinton, G. On the importance of initialization and momentum in deep learning. In Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA, 17–19 June 2013; pp. 1139–1147. [Google Scholar]
- Zeiler, M.D. Adadelta: An adaptive learning rate method. arXiv 2012, arXiv:1212.5701. [Google Scholar]
- Hinton, G.; Srivastava, N.; Swersky, K. Lecture 6a Overview of Mini-Batch Gradient Descent Course. In Neural Networks for Machine Learning; 2012. Available online: https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf (accessed on 30 August 2023).
- Aggarwal, S.; Juneja, S.; Rashid, J.; Gupta, D.; Gupta, S.; Kim, J. Protein Subcellular Localization Prediction by Concatenation of Convolutional Blocks for Deep Features Extraction from Microscopic Images. IEEE Access 2022, 11, 1057–1073. [Google Scholar] [CrossRef]
- Aggarwal, S.; Gupta, S.; Kannan, R.; Ahuja, R.; Gupta, D.; Juneja, S.; Belhaouari, S.B. A convolutional neural network-based framework for classification of protein localization using confocal microscopy images. IEEE Access 2022, 10, 83591–83611. [Google Scholar] [CrossRef]
- Kawazoe, Y.; Shimamoto, K.; Yamaguchi, R.; Shintani-Domoto, Y.; Uozaki, H.; Fukayama, M.; Ohe, K. Faster R-CNN-based glomerular detection in multistained human whole slide images. J. Imaging 2018, 4, 91. [Google Scholar] [CrossRef]
- Simon, O.; Yacoub, R.; Jain, S.; Tomaszewski, J.E.; Sarder, P. Multi-radial LBP features as a tool for rapid glomerular detection and assessment in whole slide histopathology images. Sci. Rep. 2018, 8, 2032. [Google Scholar] [CrossRef] [PubMed]
- Barros, G.O.; Navarro, B.; Duarte, A.; Dos-Santos, W.L. PathoSpotter-K: A computational tool for the automatic identification of glomerular lesions in histological images of kidneys. Sci. Rep. 2017, 7, 46769. [Google Scholar] [CrossRef] [PubMed]
- Lo, Y.C.; Juang, C.F.; Chung, I.F.; Guo, S.N.; Huang, M.L.; Wen, M.C. Glomerulus detection on light microscopic images of renal pathology with the faster r-cnn. In International Conference on Neural Information Processing; Springer: Cham, Switzerland, 2018; pp. 369–377. [Google Scholar]
Name of Layer | Number of Layers in Original UNet Model | Number of Layers in Proposed Modified UNet Model | Role of the Layers |
---|---|---|---|
Convolutional | 12 | 15 | It enhances the model’s capacity to effectively capture complex features, which further helps the model to obtain detailed features for glomerular position identification. |
Max pooling | 4 | 5 | It is used in the UNet architecture’s encoder to reduce the spatial dimensions of feature maps, which is useful for capturing larger-scale features in images. |
Upsampling | 4 | 5 | In the decoder portion of the UNet architecture, the upsampling layer is used to increase the spatial dimensions of the feature maps. It helps in the recovery of spatial information lost during the downsampling operations of the encoder. |
Normalization | 4 | 5 | The normalization layer is applied to each layer’s feature maps to stabilize and accelerate training by normalizing the activations in a small batch. |
Concatenate | 4 | 5 | The inclusion of additional concatenate layers creates more opportunities for the decoder to integrate features from various scales or levels of abstraction. This can potentially enhance the fusion of low-level and high-level features, ultimately leading to improved accuracy in segmentation. |
Optimizer | Validation Loss | Validation Accuracy | ||||
---|---|---|---|---|---|---|
Original UNet Model | UNet Model with EfficientNetb3 | Proposed Modified UNet Model | Original UNet Model | UNet Model with EfficientNetb3 | Proposed Modified UNet Model | |
Adadelta | 70.34 | 69.32 | 67.61 | 84.25 | 86.35 | 86.42 |
RMSprop | 52.65 | 50.56 | 48.25 | 85.04 | 86.78 | 86.83 |
SGD | 69.23 | 68.21 | 66.04 | 78.47 | 80.72 | 82.76 |
Adam | 29.67 | 27.45 | 23.12 | 85.28 | 87.98 | 88.77 |
Batch Size | Validation Loss | Validation Accuracy | ||||
---|---|---|---|---|---|---|
Original UNet Model | UNet Model with EfficientNetb3 | Proposed Modified UNet Model | Original UNet Model | UNet Model with EfficientNetb3 | Proposed Modified UNet Model | |
8 | 25.78 | 24.9 | 23.12 | 82.67 | 84.98 | 87.77 |
16 | 27.7 | 26.90 | 24.05 | 82.34 | 83.39 | 84.02 |
24 | 26.56 | 25.90 | 23.02 | 80.32 | 82.79 | 84.79 |
Epochs | Validation Loss | Validation Accuracy | ||||
---|---|---|---|---|---|---|
Original UNet Model | UNet Model with EfficientNetb3 | Proposed Modified UNet Model | Original UNet Model | UNet Model with EfficientNetb3 | Proposed Modified UNet Model | |
1 | 34.45 | 33.28 | 30.28 | 80.67 | 82.02 | 84.19 |
10 | 25.78 | 24.9 | 23.12 | 82.67 | 84.98 | 87.77 |
20 | 22.56 | 21.95 | 20.06 | 83.2 | 86.03 | 89.9 |
30 | 20.78 | 19.34 | 18.33 | 84.67 | 88.90 | 91.3 |
40 | 18.86 | 17.04 | 17.15 | 86.17 | 90.19 | 92.1 |
50 | 16.53 | 17.1 | 16.41 | 91.88 | 87.46 | 92.6 |
Metrics | Accuracy (%) | Precision (%) | Recall (%) | F1-Score (%) |
---|---|---|---|---|
Image I | 97.5 | 97.6 | 99.5 | 98.5 |
Image II | 96.9 | 96.5 | 99.9 | 98.2 |
Image III | 98.9 | 95.8 | 99.6 | 99.6 |
Image IV | 99.8 | 99.9 | 99.8 | 99.9 |
Image V | 95.3 | 96 | 98.4 | 97.2 |
Ref. | Species | Staining | Number of WSIs | Number of Cropped Images with Size (pixels) | Technique Used | Performance Parameters | |||
---|---|---|---|---|---|---|---|---|---|
Accuracy (%) | Precision (%) | Recall (%) | F1-Score (%) | ||||||
Cascarano et al. [11] | Human | PAS | 26 | 2772 (656 × 656) | CAD | 95 | 98.4 | 93.1 | 95.6 |
Ye Gu et al. [15] | Human | PAS | --- | --- | FCN+ ResNet/DeepLab v3 | --- | --- | --- | 91.5 |
Altini et al. [17] | Human | PAS | 26 | 2772 (656 × 656) | SegNet, | --- | 83.4 | 88.6 | 83.8 |
DeepLab v3+ | --- | 93.5 | 91.3 | 89.7 | |||||
Kato et al. [16] | Rat | Desmin | 20 | 200 × 200 | R-HOG + SVM | --- | 77.7 | 91.1 | 85.9 |
S-HOG + SVM | --- | 87.4 | 89.7 | 92.4 | |||||
Davis et al. [25] | Human | PAS | 258 | 24,133 (256 × 256) | UNet with 9 layers of CNN | --- | 90 | 96 | 93 |
Jiang et al. [26] | Human | PAS | --- | 1123 | Mask region based CNN | --- | --- | --- | 91.4 |
Kawazoe et al. [40] | Human | PAS | 200 | 4029 (1100 × 1100) | Faster R-CNN | --- | 93.1 | 91.9 | 92.5 |
PAM | 200 | 4029 (1100 × 1100) | --- | 93.9 | 91.8 | 92.8 | |||
MT | 200 | 4029 (1100 × 1100) | --- | 91.5 | 87.8 | 89.6 | |||
Azan | 200 | 4029 (1100 × 1100) | --- | 90.4 | 84.9 | 87.6 | |||
Simon et al. [41] | Human | PAS | 25 | 1649 (576 × 576) | MrcLBP + SVM | --- | 91.7 | 76.1 | 83.2 |
Barros et al. [42] | Human | PAS/H&E | --- | 811 | LoG + KNN | 88.3 | 92.3 | 88 | 90.08 |
Lo et al. [43] | Human | PAS/H&E | 40 | 3473 | Faster-RNN | --- | 86.5 | 91.5 | 88.9 |
Proposed model | Human | PAS | 20 | 50,486 (512 × 512) | Modified UNet model | 95.7 | 97.2 | 96.4 | 96.7 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kaur, G.; Garg, M.; Gupta, S.; Juneja, S.; Rashid, J.; Gupta, D.; Shah, A.; Shaikh, A. Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model. Diagnostics 2023, 13, 3152. https://doi.org/10.3390/diagnostics13193152
Kaur G, Garg M, Gupta S, Juneja S, Rashid J, Gupta D, Shah A, Shaikh A. Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model. Diagnostics. 2023; 13(19):3152. https://doi.org/10.3390/diagnostics13193152
Chicago/Turabian StyleKaur, Gurjinder, Meenu Garg, Sheifali Gupta, Sapna Juneja, Junaid Rashid, Deepali Gupta, Asadullah Shah, and Asadullah Shaikh. 2023. "Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model" Diagnostics 13, no. 19: 3152. https://doi.org/10.3390/diagnostics13193152
APA StyleKaur, G., Garg, M., Gupta, S., Juneja, S., Rashid, J., Gupta, D., Shah, A., & Shaikh, A. (2023). Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model. Diagnostics, 13(19), 3152. https://doi.org/10.3390/diagnostics13193152