Genetic Clustered Federated Learning for COVID-19 Detection
Abstract
:1. Introduction
- The clients are grouped based on the hyper-parameters thereby increasing the learning efficiency per training unit.
- Genetic algorithm is used to tune the hyper-parameters and better model aggregation in a cluster.
2. Literature Survey
2.1. Federated Learning
2.2. Evolutionary Algorithms
3. Proposed Methodology
Algorithm 1 Clustering and initial broadcasting N = no. of clients : Learning Rate : Learning Rate List |
|
Algorithm 2 FL-based genetic optimization for clustered data. rounds: The number of loops required to train the decentralized approach |
|
Dataset Description
- genetic algorithms optimize hyper-parameters based on mutation, crossover, and evolution;
- clients receive optimized hyper-parameters for cluster-based servers;
- every client is prepared using a set of parameters;
- combining client models produces the most effective server model.
4. Results and Discussion
4.1. COVID Dataset Performance for Genetic CFL Architecture
4.2. Genetic CFL Performance Analysis
5. Conclusions and Future Directions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
COVID-19 | Coronavirus |
AI | Artificial Intelligence |
ML | Machine Learning |
FL | Federated Learning |
GCFL | Genetic Clustered Federated Learning |
PCFL | Privacy-preserving and Communication efficient scheme for Federated Learning |
FedCD | Federated Cloning-and-Deletion |
FedMA | Federated Matched Averaging |
IFCA | Iterative Federated Clustering Algorithm |
FeSEM | Federated Stochastic Expectation Maximization method |
WOA | Whale Optimization Algorithm |
MLP | Multilayer Perceptrons |
ANN | Artificial Neural Networks |
References
- Liu, B.; Yan, B.; Zhou, Y.; Yang, Y.; Zhang, Y. Experiments of federated learning for covid-19 chest X-ray images. arXiv 2020, arXiv:2007.05592. [Google Scholar]
- Hageman, J.R. The coronavirus disease 2019 (COVID-19). Pediatr. Ann. 2020, 49, 99–100. [Google Scholar] [CrossRef] [PubMed]
- Saleem, K.; Saleem, M.; Zeeshan, R.; Javed, A.R.; Alazab, M.; Gadekallu, T.R.; Suleman, A. Situation-aware BDI reasoning to detect early symptoms of covid 19 using smartwatch. IEEE Sens. J. 2022. [Google Scholar] [CrossRef]
- Mothukuri, V.; Parizi, R.M.; Pouriyeh, S.; Huang, Y.; Dehghantanha, A.; Srivastava, G. A survey on security and privacy of federated learning. Future Gener. Comput. Syst. 2021, 115, 619–640. [Google Scholar] [CrossRef]
- Zhang, W.; Zhou, T.; Lu, Q.; Wang, X.; Zhu, C.; Sun, H.; Wang, Z.; Lo, S.K.; Wang, F.Y. Dynamic-Fusion-Based Federated Learning for COVID-19 Detection. IEEE Internet Things J. 2021, 8, 15884–15891. [Google Scholar] [CrossRef]
- Lian, X.; Zhang, C.; Zhang, H.; Hsieh, C.J.; Zhang, W.; Liu, J. Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. Adv. Neural Inf. Process. Syst. 2017, 30, 1–11. [Google Scholar]
- Manoj, M.; Srivastava, G.; Somayaji, S.R.K.; Gadekallu, T.R.; Maddikunta, P.K.R.; Bhattacharya, S. An incentive based approach for COVID-19 planning using blockchain technology. In Proceedings of the 2020 IEEE Globecom Workshops GC Wkshps, Taipei, Taiwan, 7–11 December 2020; pp. 1–6. [Google Scholar]
- Alazab, M.; Tang, M. Deep Learning Applications for Cyber Security; Springer: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
- Kumar, R.; Khan, A.A.; Kumar, J.; Golilarz, N.A.; Zhang, S.; Ting, Y.; Zheng, C.; Wang, W. Blockchain-federated-learning and deep learning models for covid-19 detection using ct imaging. IEEE Sens. J. 2021, 21, 16301–16314. [Google Scholar] [CrossRef]
- Yarradoddi, S.; Gadekallu, T.R. Federated Learning Role in Big Data, Jot Services and Applications Security, Privacy and Trust in Jot a Aurvey. In Trust, Security and Privacy for Big Data; CRC Press: Boca Raton, FL, USA, 2022; pp. 28–49. [Google Scholar]
- Nilsson, A.; Smith, S.; Ulm, G.; Gustavsson, E.; Jirstrand, M. A performance evaluation of federated learning algorithms. In Proceedings of the Second Workshop on Distributed Infrastructures for dEep Learning, Rennes, France, 10 December 2018; pp. 1–8. [Google Scholar]
- Victor, N.; Alazab, M.; Bhattacharya, S.; Magnusson, S.; Maddikunta, P.K.R.; Ramana, K.; Gadekallu, T.R. Federated Learning for IoUT: Concepts, Applications, Challenges and Opportunities. arXiv 2022, arXiv:2207.13976. [Google Scholar]
- Sattler, F.; Müller, K.R.; Wiegand, T.; Samek, W. On the byzantine robustness of clustered federated learning. In Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Virtual, 4–8 May 2020; pp. 8861–8865. [Google Scholar]
- Malekijoo, A.; Fadaeieslam, M.J.; Malekijou, H.; Homayounfar, M.; Alizadeh-Shabdiz, F.; Rawassizadeh, R. Fedzip: A compression framework for communication-efficient federated learning. arXiv 2021, arXiv:2102.01593. [Google Scholar]
- Fang, C.; Guo, Y.; Hu, Y.; Ma, B.; Feng, L.; Yin, A. Privacy-preserving and communication-efficient federated learning in internet of things. Comput. Secur. 2021, 103, 102199. [Google Scholar] [CrossRef]
- Alazab, M.; Huda, S.; Abawajy, J.; Islam, R.; Yearwood, J.; Venkatraman, S.; Broadhurst, R. A hybrid wrapper-filter approach for malware detection. J. Netw. 2014, 9, 1–14. [Google Scholar] [CrossRef]
- Ozfatura, E.; Ozfatura, K.; Gündüz, D. Time-correlated sparsification for communication-efficient federated learning. In Proceedings of the 2021 IEEE International Symposium on Information Theory (ISIT), Melbourne, VI, Australia, 12–20 July 2021; pp. 461–466. [Google Scholar]
- Briggs, C.; Fan, Z.; Andras, P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19–24 July 2020; pp. 1–9. [Google Scholar]
- Zhao, Y.; Li, M.; Lai, L.; Suda, N.; Civin, D.; Chandra, V. Federated learning with non-iid data. arXiv 2018, arXiv:1806.00582. [Google Scholar] [CrossRef]
- Wang, H.; Yurochkin, M.; Sun, Y.; Papailiopoulos, D.; Khazaeni, Y. Federated learning with matched averaging. arXiv 2020, arXiv:2002.06440. [Google Scholar]
- Yao, X.; Huang, T.; Wu, C.; Zhang, R.; Sun, L. Towards faster and better federated learning: A feature fusion approach. In Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22–29 September 2019; pp. 175–179. [Google Scholar]
- Ghosh, A.; Chung, J.; Yin, D.; Ramchandran, K. An efficient framework for clustered federated learning. Adv. Neural Inf. Process. Syst. 2020, 33, 19586–19597. [Google Scholar] [CrossRef]
- Kopparapu, K.; Lin, E.; Zhao, J. Fedcd: Improving performance in non-iid federated learning. arXiv 2020, arXiv:2006.09637. [Google Scholar]
- Gadekallu, T.R.; Pham, Q.V.; Huynh-The, T.; Bhattacharya, S.; Maddikunta, P.K.R.; Liyanage, M. Federated learning for big data: A survey on opportunities, applications, and future directions. arXiv 2021, arXiv:2110.04160. [Google Scholar]
- Zhang, C.; Xie, Y.; Bai, H.; Yu, B.; Li, W.; Gao, Y. A survey on federated learning. Knowl.-Based Syst. 2021, 216, 106775. [Google Scholar] [CrossRef]
- Bonawitz, K.; Eichner, H.; Grieskamp, W.; Huba, D.; Ingerman, A.; Ivanov, V.; Kiddon, C.; Konečnỳ, J.; Mazzocchi, S.; McMahan, B.; et al. Towards federated learning at scale: System design. Proc. Mach. Learn. Syst. 2019, 1, 374–388. [Google Scholar]
- Xie, M.; Long, G.; Shen, T.; Zhou, T.; Wang, X.; Jiang, J.; Zhang, C. Multi-center federated learning. arXiv 2020, arXiv:2005.01026. [Google Scholar]
- Chai, Z.; Ali, A.; Zawad, S.; Truex, S.; Anwar, A.; Baracaldo, N.; Zhou, Y.; Ludwig, H.; Yan, F.; Cheng, Y. Tifl: A tier-based federated learning system. In Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing, Stockholm, Sweden, 23–26 June 2020; pp. 125–136. [Google Scholar]
- Likas, A.; Vlassis, N.; Verbeek, J.J. The global k-means clustering algorithm. Pattern Recognit. 2003, 36, 451–461. [Google Scholar] [CrossRef]
- Kim, J.Y.; Cho, S.B. Evolutionary optimization of hyperparameters in deep learning models. In Proceedings of the 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand, 10–13 June 2019; pp. 831–837. [Google Scholar]
- Xiao, X.; Yan, M.; Basodi, S.; Ji, C.; Pan, Y. Efficient hyperparameter optimization in deep learning using a variable length genetic algorithm. arXiv 2020, arXiv:2006.12703. [Google Scholar]
- Aljarah, I.; Faris, H.; Mirjalili, S. Optimizing connection weights in neural networks using the whale optimization algorithm. Soft Comput. 2018, 22, 1–15. [Google Scholar] [CrossRef]
- Beruvides, G.; Quiza, R.; Rivas, M.; Casta no, F.; Haber, R.E. Online detection of run out in microdrilling of tungsten and titanium alloys. Int. J. Adv. Manuf. Technol. 2014, 74, 1567–1575. [Google Scholar] [CrossRef]
- Agrawal, S.; Sarkar, S.; Alazab, M.; Maddikunta, P.K.R.; Gadekallu, T.R.; Pham, Q.V. Genetic CFL: Hyperparameter optimization in clustered federated learning. Comput. Intell. Neurosci. 2021, 2021, 7156420. [Google Scholar] [CrossRef] [PubMed]
- Zhang, H.; Zhang, M.; Liu, X.; Mohapatra, P.; DeLucia, M. FedTune: Automatic Tuning of Federated Learning Hyper-Parameters from System Perspective. arXiv 2021, arXiv:2110.03061. [Google Scholar]
- Khodak, M.; Tu, R.; Li, T.; Li, L.; Balcan, M.F.F.; Smith, V.; Talwalkar, A. Federated hyperparameter tuning: Challenges, baselines, and connections to weight-sharing. Adv. Neural Inf. Process. Syst. 2021, 34, 19184–19197. [Google Scholar]
- Ibraimi, L.; Selimi, M.; Freitag, F. BePOCH: Improving federated learning performance in resource-constrained computing devices. In Proceedings of the 2021 IEEE Global Communications Conference (GLOBECOM), Madrid, Spain, 7–11 December 2021; pp. 1–6. [Google Scholar]
- Taheri, R.; Shojafar, M.; Alazab, M.; Tafazolli, R. FED-IIoT: A robust federated malware detection architecture in industrial IoT. IEEE Trans. Ind. Inform. 2020, 17, 8442–8452. [Google Scholar] [CrossRef]
- Qayyum, A.; Ahmad, K.; Ahsan, M.A.; Al-Fuqaha, A.; Qadir, J. Collaborative federated learning for healthcare: Multi-modal covid-19 diagnosis at the edge. arXiv 2021, arXiv:2101.07511. [Google Scholar]
- Arikumar, K.; Prathiba, S.B.; Alazab, M.; Gadekallu, T.R.; Pandya, S.; Khan, J.M.; Moorthy, R.S. FL-PMI: Federated learning-based person movement identification through wearable devices in smart healthcare systems. Sensors 2022, 22, 1377. [Google Scholar] [CrossRef]
Ref. No | Technologies Used | Key Contributions | Limitations |
---|---|---|---|
[20] | Federated matched averaging (FedMA) algorithm | FedMA builds the shared global model layer by layer based on feature extraction signatures of hidden elements. | Privacy, data bias |
[21] | Feature fusion mechanism | The accuracy and generalization abilities of FedFusion outperform baselines while reducing communication rounds by more than 60 percent. | The issue of high communication costs must be addressed immediately. |
[22] | Iterative Federated Clustering Algorithm (IFCA) | The convergence rate of the population loss function under proper initialization ensures both convergences of the training loss and generalization to test data simultaneously. | Data heterogeneity is to be addressed. |
[27] | Multi-center aggregation mechanism | The proposed objective function is optimized using the Federated Stochastic Expectation Maximization method (FeSEM). | Data heterogeneity is to be addressed. |
[31] | Genetic algorithms | Convolutional neural networks can be efficiently tuned by using a variable-length genetic algorithm. | In the case of networks with fewer layers, the size could be too small for the problem, resulting in underfitting. |
[32] | Whale optimization algorithm (WOA) | For training multilayer perceptrons (MLP), the WOA was applied because of its high local optimization avoidance and fast convergence speed. | Slow convergence speed and local optima stagnation are the main disadvantages of conventional training algorithms. |
[39] | Clustered federated learning | It proposes a collaborative learning framework to intelligently process visual data at the edge device by developing a multi-modal ML algorithm that is capable of diagnosing COVID-19 in both X-ray and Ultrasound images. | The major challenge here is regarding the performance of CFL when the number of samples per client varies. |
[13] | Clustered federated learning | It addresses the issue of suboptimal results when the local clients’ data distributions diverge by separating the client population into different groups based on the pairwise cosine similarities. | The main cluster is separated from some suspicious clients after a few rounds, which poses a major challenge. |
This paper | Genetic CFL algorithm | Genetic algorithms are used to optimize the hyper-parameters such as batch size, and learning rate of the clustered FL models. | Client training has a significant impact on FL efficiency. |
Symbol | Meaning |
---|---|
N | no. of customers |
Learning Rate | |
b | Batch Size |
customer | |
Weights of models | |
Weights of models of customer |
Column Name | Description |
---|---|
test date | Date of arrival of the test in the laboratory. It should be noted that this is the citizen’s first test, in the DD/MM/YYYY format. |
cough | Did cough symptoms appear before the test? 1—Yes, 0—No, NULL—Unknown. |
fever | Did fever appear before the test? 1—Yes, 0—No, NULL—Unknown. |
sore throat | Did a sore throat appear before the test? 1—Yes, 0—No, NULL—Unknown. |
shortness of breath | Did breathing difficulties occur before the test? 1—Yes, 0—No, NULL—Unknown. |
headache | Did a headache appear before the test? 1—Yes, 0—No, NULL—Unknown. |
corona result | Results of the first corona test performed on the subject. Categorical variable, 3 categories:
|
age 60 and above | Indicator 60 years or older (1) or under 60 years (0). |
gender | Sex of the subjects. Male/Female/Null (unknown). |
test indication | What is the indication for testing?
|
Client Ratio | Rounds | FL | ||||
---|---|---|---|---|---|---|
Accuracy | Loss | Precision | Recall | F1-Score | ||
0.1 | 3 | 0.9148 | 0.2609 | 0.9132 | 0.9146 | 0.9145 |
6 | 0.9212 | 0.2524 | 0.9203 | 0.9211 | 0.9210 | |
10 | 0.9367 | 0.2115 | 0.9362 | 0.9366 | 0.9365 | |
0.15 | 3 | 0.9156 | 0.2608 | 0.9151 | 0.9155 | 0.9154 |
6 | 0.9197 | 0.2560 | 0.9192 | 0.9196 | 0.9195 | |
10 | 0.9224 | 0.2554 | 0.9216 | 0.9221 | 0.9220 | |
0.3 | 3 | 0.9068 | 0.2758 | 0.9059 | 0.9064 | 0.9063 |
6 | 0.9166 | 0.2648 | 0.9158 | 0.9164 | 0.9163 | |
10 | 0.9187 | 0.2629 | 0.9179 | 0.9185 | 0.9184 |
Client Ratio | Rounds | Genetic CFL | ||||
---|---|---|---|---|---|---|
Accuracy | Loss | Precision | Recall | F1-Score | ||
0.1 | 3 | 0.9271 | 0.2469 | 0.9267 | 0.9269 | 0.9268 |
6 | 0.9271 | 0.2467 | 0.9268 | 0.9270 | 0.9269 | |
10 | 0.9271 | 0.2460 | 0.9270 | 0.9269 | 0.9268 | |
0.15 | 3 | 0.9223 | 0.2552 | 0.9218 | 0.9221 | 0.9220 |
6 | 0.9223 | 0.2551 | 0.9220 | 0.9222 | 0.9221 | |
10 | 0.9223 | 0.2549 | 0.9222 | 0.9221 | 0.9220 | |
0.3 | 3 | 0.9208 | 0.2627 | 0.9199 | 0.9206 | 0.9205 |
6 | 0.9208 | 0.2625 | 0.9205 | 0.9207 | 0.9206 | |
10 | 0.9208 | 0.2621 | 0.9207 | 0.9206 | 0.9205 |
Round No | Epochs | Genetic CFL | |||
---|---|---|---|---|---|
Training Loss | Training Accuracy | Validation Loss | Validation Accuracy | ||
1 | 1 | 0.6276 | 0.9126 | 0.6172 | 0.9126 |
2 | 0.6090 | 0.9109 | 0.5986 | 0.9125 | |
3 | 0.5906 | 0.9109 | 0.5803 | 0.9125 | |
2 | 1 | 0.4883 | 0.9109 | 0.4787 | 0.9125 |
2 | 0.4726 | 0.9109 | 0.4636 | 0.9125 | |
3 | 0.4580 | 0.9109 | 0.4494 | 0.9125 | |
3 | 1 | 0.3857 | 0.9109 | 0.3791 | 0.9125 |
2 | 0.3754 | 0.9109 | 0.3694 | 0.9125 | |
3 | 0.3663 | 0.9109 | 0.3609 | 0.9125 | |
4 | 1 | 0.3262 | 0.9109 | 0.3220 | 0.9125 |
2 | 0.3204 | 0.9109 | 0.3165 | 0.9125 | |
3 | 0.3153 | 0.9109 | 0.3116 | 0.9125 | |
5 | 1 | 0.2926 | 0.9109 | 0.2892 | 0.9125 |
2 | 0.2895 | 0.9109 | 0.2863 | 0.9125 | |
3 | 0.2868 | 0.9109 | 0.2836 | 0.9125 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kandati, D.R.; Gadekallu, T.R. Genetic Clustered Federated Learning for COVID-19 Detection. Electronics 2022, 11, 2714. https://doi.org/10.3390/electronics11172714
Kandati DR, Gadekallu TR. Genetic Clustered Federated Learning for COVID-19 Detection. Electronics. 2022; 11(17):2714. https://doi.org/10.3390/electronics11172714
Chicago/Turabian StyleKandati, Dasaradharami Reddy, and Thippa Reddy Gadekallu. 2022. "Genetic Clustered Federated Learning for COVID-19 Detection" Electronics 11, no. 17: 2714. https://doi.org/10.3390/electronics11172714
APA StyleKandati, D. R., & Gadekallu, T. R. (2022). Genetic Clustered Federated Learning for COVID-19 Detection. Electronics, 11(17), 2714. https://doi.org/10.3390/electronics11172714