Biometric-Based Human Identification Using Ensemble-Based Technique and ECG Signals

Abstract

User authentication has become necessary in different life domains. Traditional authentication methods like personal information numbers (PINs), password ID cards, and tokens are vulnerable to attacks. For secure authentication, methods like biometrics have been developed in the past. Biometric information is hard to lose, forget, duplicate, or share because it is a part of the human body. Many authentication methods focused on electrocardiogram (ECG) signals have achieved great success. In this paper, we have developed cardiac biometrics for human identification using a deep learning (DL) approach. Cardiac biometric systems rely on cardiac signals that are captured using the electrocardiogram (ECG), photoplethysmogram (PPG), and phonocardiogram (PCG). This study utilizes the ECG as a biometric modality because ECG signals are a superior choice for accurate, secure, and reliable biometric-based human identification systems, setting them apart from PPG and PCG approaches. To get better performance in terms of accuracy and computational time, we have developed an ensemble approach based on VGG16 pre-trained transfer learning (TL) and Long Short-Term Memory (LSTM) architectures to optimize features. To develop this authentication system, we have fine-tuned this ensemble network. In the first phase, we preprocessed the ECG biosignal to remove noise. In the second phase, we converted the 1-D ECG signals into a 2-D spectrogram image using a transformation phase. Next, the feature extraction step is performed on spectrogram images using the proposed ensemble DL technique, and finally, those features are identified by the boosting machine learning classifier to recognize humans. Several experiments were performed on the selected dataset, and on average, the proposed system achieved 98.7% accuracy, 98.01% precision, 97.1% recall, and 0.98 AUC. In this paper, we have compared the developed approach with state-of-the-art biometric authentication systems. The experimental results demonstrate that our proposed system outperformed the human recognition competition.

1. Introduction

In an era characterized by ubiquitous digital interactions, the need for robust and secure human identification systems has become paramount. Traditional recognition methods like personal identification numbers (PINs) [1] and passwords have become vulnerable to attacks and can be lost or forgotten. As a result, there is a need to develop a biometric system that is more secure. PINs are increasingly protected using biometrics to address the threat of loss or theft. These technologies safely manage personal information and verify the user’s identity. Biometric recognition is an approach that provides a unique method for identity recognition. It uses metrics related to human characteristics for recognition. Biometrics has many features, such as uniqueness, permanence, and ease of collection. It involves both behavioral and physical characteristics. Behavioral factors include voice, gait, and electrocardiogram (ECG), while Physical features include the face, fingerprints, and iris [2]. However, traditional biometrics needs help with many issues, like addressing spoofing or forgery. As a result, there has been a growing interest in leveraging biometric authentication methods, which rely on individuals’ unique physiological and behavioral characteristics, to enhance security and reduce vulnerabilities. Cardiac biometrics have emerged as a promising and innovative approach for human identification among the myriad biometric modalities. The human heart’s intrinsic electrical activity, captured using electrocardiogram (ECG), photoplethysmograph (PPG), and phonocardiogram (PCG) signals [3], offers a wealth of valuable information that can be leveraged for secure authentication. Utilizing electrocardiogram (ECG) signals for biometric-based human identification provides several distinct advantages over alternative approaches like photoplethysmograph (PPG) and phonocardiogram (PCG) signals. ECG signals offer a unique and highly individualistic biometric marker [4] due to the heart’s intricate electrical activity patterns. This uniqueness enhances the security and accuracy of identification systems. Unlike PPG, which primarily measures blood volume changes and can be influenced by external factors, ECG signals directly reflect the cardiac electrical patterns, making them more robust and reliable in varied conditions. Additionally, ECG signals are less susceptible to mimicry or spoofing than PPG and PCG signals, which can be more easily replicated using external devices. The stability and consistency of ECG features across different states (e.g., resting and exercising) further strengthen its suitability for biometric applications. Lastly, the non-intrusive nature of ECG measurements, usually captured using electrodes placed on the skin, ensures user comfort and encourages wider adoption. These combined advantages position ECG signals as superior for accurate, secure, and reliable biometric-based human identification systems, setting them apart from PPG and PCG approaches.

In this context, the present paper delves into developing a novel cardiac biometrics system, fortified by deep learning (DL) [5] approaches, to achieve secure human identification. Deep learning has exhibited remarkable capabilities in handling complex patterns and representations in diverse data domains, and its application to cardiac biometrics presents an exciting frontier in the pursuit of reliable and efficient authentication systems. The central aim of this paper is to propose an ensemble approach that effectively harnesses the strengths of pre-trained VGG16 transfer learning (TL) and Long Short-Term Memory (LSTM) architectures [6] for optimal feature extraction from 2D spectrogram images derived from ECG biosignals. The ensemble feature representation capitalizes on the spatial and temporal characteristics of the cardiac signals, providing a holistic and discriminative basis for human identification.

The paper comprises several well-defined phases, including preprocessing ECG biosignals to ensure data quality, transforming 1-D ECG signals into 2D spectrogram images, and feature extraction using the ensemble DL technique. Moreover, the identification process involves a machine learning classifier that distinguishes individuals based on their unique cardiac biometric features. Extensive experiments are conducted on a selected dataset to evaluate the proposed cardiac biometric system. The system’s performance is assessed using critical metrics such as accuracy, sensitivity, specificity, and area under the curve (AUC). A comparative analysis is also performed against state-of-the-art biometric authentication systems, showcasing the system’s superiority in human recognition.

The implications of this research extend beyond theoretical pursuits, as the proposed cardiac biometrics system holds significant potential for real-world applications in domains where secure authentication is critical. The system’s deployment in healthcare, finance, access control, and other sensitive areas can ensure trustworthy and efficient human identification, safeguard user data, and bolster system integrity. In conclusion, this paper contributes significantly to the evolving field of biometric authentication. The proposed system advances the frontiers of secure human identification by combining the power of deep learning with the rich information inherent in cardiac signals. It offers a promising pathway toward a safer and more connected digital world.

1.1. Background

Biometric authentication, which relies on individuals’ unique physiological and behavioral characteristics for identification, has gained significant attention due to its potential to enhance security and user convenience in various applications. Cardiac biometrics presents an intriguing and promising avenue for secure human identification among the diverse biometric modalities. Traditional authentication methods, such as passwords and PINs, are susceptible to various attacks, leading to data breaches and identity theft. As a result, researchers and industries alike have turned to biometrics as a more robust and user-friendly alternative. Biometric modalities like fingerprint, iris, and facial recognition have widespread adoption but may suffer from environmental constraints, usability challenges, and potential privacy concerns.

In contrast, cardiac biometrics harness the unique physiological characteristics of the human heart, making them inherently difficult to replicate, forget, or lose. The Electrocardiogram (ECG), photoplethysmograph (PPG), and phonocardiogram (PCG) are some of the cardiac signals that hold valuable information about an individual’s cardiac rhythm, heart rate, and other distinctive features. The rise of deep learning has further accelerated advancements in biometric authentication systems. Deep learning models, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) [6], have shown remarkable capabilities in handling complex patterns and representations in various data domains, including images, sequences, and time-series data.

Building on this context, the present research endeavors to develop a novel cardiac biometric system for human identification using deep learning approaches. By combining the strengths of deep learning with the unique characteristics of cardiac signals, the goal is to achieve accurate and secure human identification, surpassing the limitations of traditional authentication methods. The proposed research addresses critical challenges in cardiac biometric authentication, such as noise reduction, feature extraction, and classification. The use of an ensemble approach, combining pre-trained VGG16 transfer learning (TL) and Long Short-Term Memory (LSTM) architectures, is expected to offer a comprehensive and discriminative representation of cardiac features, capturing both spatial and temporal aspects of the signals.

Electrocardiogram (ECG) signals offer a distinct advantage over photoplethysmograph (PPG) and electroencephalogram (EEG) signals [7] due to their precision in assessing cardiac activity, which is unique for individuals. ECG provides a detailed and highly accurate representation of the heart’s electrical activity, enabling the detection of patterns. In contrast with PPG and EEG signals, the ECG’s ability to offer comprehensive insights into cardiac health makes it a superior choice for detailed cardiovascular assessments [8]. Therefore, we have used ECG signals compared to PPG and EEG signals for human detection.

The ECG signal is a biometric heart signal. Recent research has focused on using ECG signals for biometric identification due to their characteristics. In practice, it is more difficult to commit fraud. In the past, many studies used machine learning (ML) or deep learning (DL) techniques for identifying a person using heart signals. Compared to ML-based systems, DL-based methods outperformed heart biometrics. Those studies utilized ECG signals for human identification by developing a deep-learning model. However, those studies are not up-to-date because the ECG signals contain many noises or artifacts, making it challenging to recognize humans. Those methods are computationally expensive for fine-tuning the network and selecting hyperparameters. In addition, it is not easy to represent the signals in time and frequency domains to identify humans based on ECG signals. Figure 1 shows A visual example of biometric-based human identification using ECG signal processing and a machine learning algorithm.

The following research questions have been addressed in this paper. How can ECG signals effectively be used as a biometric modality for human recognition using a deep learning approach (VGG16-LSTM) and a Light Gradient boosting algorithm (LightGBM)?

(1)

What preprocessing techniques have been used to remove noise from signals?

(2)

Which performance measures can be used for assessing model performance?

(3)

What are the hyperparameters required to train the model?

(4)

What are the fine-tuning steps required to make the DL model less computationally expensive?

(5)

What statistical matrices are required to evaluate the performance of the proposed system?

1.2. Major Contributions

The main contribution of the paper lies in the development of a novel cardiac biometrics system for human identification using deep learning (DL) approaches, specifically focusing on electrocardiogram (ECG) signals. The key contributions can be summarized as follows:

• We chose ECG as the biometric modality for human identification. This decision provides a unique and challenging set of physiological data for authentication, different from traditional biometric methods.
• We proposed a novel ensemble-based fine-tune approach that combines VGG16 pre-trained transfer learning (TL) and Long Short-Term Memory (LSTM) architectures to extract deep features from preprocessed ECG signals.
• The paper presents a well-defined preprocessing phase for the ECG biosignal to remove noise and ensure data quality. Additionally, you developed a transformation phase that converts the 1-D ECG signals into 2-D spectrogram images. The feature extraction step using the ensemble DL technique is a crucial part of the process, as it extracts discriminative features for human identification.
• We employed a LightGBM boosting classifier to recognize humans based on the extracted features. This classifier likely enhances the performance of the system and contributes to achieving high accuracy, sensitivity, specificity, and AUC.
• The developed cardiac biometrics system has practical applications in various life domains where secure authentication is crucial. These applications may include healthcare, finance, access control, and other sensitive areas.

Overall, the paper contributes a comprehensive approach to human identification using cardiac biometrics, combining deep learning techniques with a unique biometric modality. The experimental results and comparison with existing systems demonstrate the superiority of your proposed approach, making it a valuable contribution to the field of biometric authentication.

1.3. Paper Organization

The reminder of this paper is organized as follows. In Section 2, the previous related works have been presented and compared. In Section 3, the methodology has been described. Whereas in Section 4, the experimental results have been mentioned. The discussions and future works are presented in Section 5. Finally, Section 6 presents a conclusion.

2. Literature Review

Several studies used machine learning (ML) and deep learning (DL) techniques to identify a person from the PCG, PPG, or ECG signals.

2.1. Machine-Learning Based Techniques

2.1.1. PCG-Based Recognition Method

The authors of [6] suggested a PCG-based recognition method. The PCG signals were obtained from the HSCT-11 database. They used the MFCC algorithm for feature extraction and tested it using an ANN classifier. Additionally, they used a PCA algorithm to get the most discriminative features. The experiment’s results showed an accuracy of 96%.

2.1.2. PPG-Based Recognition Method

These studies used PPG signals for human recognition. In [7], a model based on Parse SoftMax Vector and k-Nearest Neighbor (K-NN) was proposed. Experiments were performed on BIDMC, MIMIC, and CapnoBase datasets and achieved a recognition rate of 99.95%, 97.21%, and 99.92%, respectively. In another study, the authors of [8] used A discrete cosine transform (DCT) for feature extraction. Then, the extracted features have been used as input for some machine-learning techniques like Decision Tree, KNN, and Random Forests. The achieved accuracy of the algorithms was 93%, 98%, and 99%, respectively. The authors of [9] proposed an authentication method that depends on a continuous wavelet transform (CWT) and direct linear discriminant analysis (DLDA). The method was tested on different datasets and achieved an equal error rate (EER) of 0.5–6%. The authors of [10] proposed an authentication method based on non-fiducial features and used Neural Networks (NN) and SVM for classification. To optimize the classification, a Genetic Algorithm was used. The method achieved 100% accuracy. Another study [11] evaluated fiducial and non-fiducial approaches to feature extraction for PPG-based authentication using both supervised and unsupervised machine learning methods. The experimental results achieved an accuracy of 99.84% based on non-fiducial feature extraction.

2.1.3. ECG-Based Recognition Method

Many studies have proposed recognition methods based on ECG signals. The authors of [12] suggested an ECG recognition method. They reduced the ECG noise. A non-fiducial technique based on autocorrelation and discrete cosine transformation (AC/DCT) was used to extract features. The data were collected from the MIT-BIH Arrhythmia Database. The result showed a classification accuracy of 97% using ANN. In another study, the authors [13] proposed an ECG recognition method. They used a Cascaded FIR Filter configuration for noise removal, and a Frequency-Time-based ap-preach was used for characteristic wave detection. The data were obtained from the MIT-BIH ECG ID database. The features were extracted using R-Peak position normalization. For classification, they use three classifiers: ANN, K-NN, and SVM. The results show that the SVM classifier provides 93.709% classification accuracy. Similarly, [14] proposed a human identification method based on ECG. For classification, they used three classifiers: ANN, KNN, and SVM. The method was evaluated on the ECG-ID and MIT-BIH databases.

Unlike other studies that used ECG signals for personal identification under the condition of rest, the authors of [15] focus on using ECG signals for personal identification when the heart rate is increased through exercise. For analyzing the ECG signal, we used the root mean square (RMS) value, nonlinear Lyapunov exponent, and correlation dimension. For recognizing identity, it used a support vector machine (SVM) and achieved over 80% recognition accuracy. The authors of [16] proposed a method that used the discrete wavelet transform (DWT) for feature extraction and random forests for authentication. Results show that the system can achieve 100% recognition accuracy. Another study [17] proposed an ECG-based identification system using the discrete wavelet transform (DWT) of the cardiac cycle and heart rate variability (HRV) for feature extraction and Random Forests for classification. The system was evaluated on three databases and achieved an accuracy of 95.85%, 100%, and 83.88%.

The authors of [18] proposed a method for an authentication system based on ECG signals. First, the raw ECG signal is denoised using empirical mode decomposition (EMD). In the next step, features are extracted. Finally, selected features were classified using various classification methods such as Support Vector Machines (SVM), K-nearest neighbor (KNN), and Decision Tree (DT). Based on the evaluation, the SVM with a cubic kernel achieves a classification accuracy rate of 98.7%. Another study [19] proposed an ECG-based approach for authentication. The method consists of removing noise from the signal by using wavelet decomposition. An empirical mode decomposition (EMD) is used to decompose the denoised into several intrinsic mode functions (IMFs). Then, the features are extracted. The ECG signals are finally classified using SVM. The evaluation result shows that the cubic SVM has the highest accuracy of 98.4%. The authors of [20] compared three different Machine Learning algorithms: KNN, SVM, and Gaussian Naive Bayes (GNB). The result shows an accuracy of over 90%.

The authors of [21] used three types of features, namely cepstral coefficients, the ZCR, and entropy, to identify individuals. Then, the classification was performed using various types of SVM kernels. The evaluation performed on the MIT-BIH arrhythmia and ECG-ID databases shows that the proposed method can achieve an identification accuracy of 100%. Another study [22] used a statistical learning method (LASSO) to select the appropriate features for classification. Then, we used some machine learning algorithms such as ANN, SVM, and KNN for classification. The experimental results show that LASSO with a KNN classifier achieves a recognition accuracy of 99.1379%. This has been observed for the proposed method of LASSO with the KNN classifier.

A comparison of the machine learning techniques is shown in

Table 1. Comparison of machine learning techniques for human recognition.

Work	Year	Feature Extraction	Classifier	Dataset	Results and Accuracy
[4]	2016	MFCC	ANN	HSCT-11	96%
[12]	2016	AC/DCT	ANN	MIT-BIH	97%
[13]	2017	Fiducial based features	ANN KNN SVM	ECG ID database	92.45% 92.72% 93.70%
[14]	2018	Mean P-QRS-T Fragment + DWT	ANN KNN SVM	ECG-ID MIT-BIH	95%
[16]	2012	DCT	Random Forests	MIT-BIH PTB database	AVG ACC = 100%
[8]	2019	DCT	Decision Tree KNN Random Forest	Not mentioned	93% 98% 99%
[17]	2015	DWT + HVR	Random Forests	MITDB NSRDB ECG-ID	95.85% 100% 83.88%
[21]	2020	Cepstral coefficients+ ZCR+ Entropy	SVM	MIT-BIH ECG-ID	100%
[22]	2019	Fiducial based features	ANN KNN OAA-SVM (With LASSO)	ECG-ID	99.1379% (LASSO with KNN)

. The table shows the methods’ feature extraction, classifier, datasets, and accuracy. The authors used different feature extraction methods, classifiers, and datasets.

2.2. Deep-Learning Based Techniques

2.2.1. PPG-Based Recognition Method

The authors of [23] proposed a two-level fusion PCANet (Principal Component Analysis Network) deep recognition network. It achieved an over 95% recognition rate.

There are many papers that use deep learning methods to analyze ECG signals. The authors of [24] proposed a PPG-based end-to-end architecture using Convolutional Networks. The approach was tested on PulseID and Troika databases and achieved an AUC of 78.2% and 83.2%, respectively. Another study [25] suggested a method that used a deep learning model consisting of a CNN and an LSTM for the person verification process. The proposed system was evaluated on Biosec1, Biosec2, and PRRB datasets. The authors of [26] proposed a framework for human identification using PPG signals. The model consists of two CNN layers and two LSTM layers, followed by a dense output layer. The model was evaluated on the TROIKA database, achieving an average accuracy of 96%. The study [27] also used the TROIKA dataset to create a human identification system. Each PPG source is grouped into different groups and classified using Deep Belief Networks (DBN). The approach achieves an accuracy of 96.1%. The authors of [28] proposed a Personalized Verification System, PPSNet, based on PPG signals by building a network using CNN and LSTM. The network was evaluated on the BioSec and achieved an average accuracy of 96% in a single session and 72.7% in two sessions.

2.2.2. ECG-Based Recognition Method

Many recent studies used deep learning methods that applied to ECG biometrics. It’s worth mentioning that CNNs are widely used in biometric recognition systems and provide satisfying results [29]. The authors of [30] used a multiresolution 1D-convolutional neural network (1D-CNN) in the ECG identification system. The ECG signal noise was removed, and blind segmentation was performed. Then, the wavelet transform was applied to the segments, and the phase difference caused by the blind segmentation was removed using autocorrelation. This approach achieved an average recognition rate of 93.5%. In another study [31], a CNN was proposed. After removing the noise from the signal, the QRS complex extraction was performed from the ECGs to feed a 1D CNN. The tested result shows a recognition rate of 100%. While the authors of [32] suggested a method that reduces the ECG signal length, since the ECG signal requires a long duration for the recognition of a person, He proposed a CNN for ECG classification based on the R-peak and transformed using continuous wavelet transformation (CWT). The method was evaluated on the PTB database and achieved 99.94% and 99.83% identification accuracy, respectively. Similarly, the authors of [33] used a short ECG signal around the R-peak that has been transformed into Continuous Wavelet Transformation (CWT) images, which are then used by the convolutional neural network (CNN) for the recognition process. The results show a classification accuracy of 99.90% for PTB, 98.20% for the ECG-ID mixed-session, and 94.18% for the ECG-ID multisession datasets. The authors of [34] used pre-configured models of CNN to evaluate ECG biometrics using different time-frequency representations like MFCC, spectrogram, log spectrogram, Mel spectrogram, and scalogram. They used the PTB-ECG and CU-ECG databases. The result shows that the MFCC has higher accuracy than other time-frequency representations.

The authors of [35] proposed an approach for human authentication based on ECG using CNN. The proposed algorithm achieved an average accuracy of 97.92% on the MWMHIT database and 99.96% on the MIT-BIH database. Another study [36] proposed a system that uses CNN to identify a person using temporal frequency analysis like spectrograms. The system was tested on Fantasia and ECG-ID databases and achieved 99.42% and 94.23%, respectively. Similarly, the study [37] used CNN for biometric identification. The proposed method does not require R-peak detection. It was evaluated on the PTB database and achieved a recognition accuracy of 99.1%. While the authors of [38] proposed cascaded CNNs that consist of two CNNs, called F-CNN for feature extraction and M-CNN for identification, the approach was evaluated using different databases and achieved an average recognition of 94.3%. The method described in [39] used a 2-D coupling image of the ECG signal. CNN, which specializes in processing images, is used to process the 2-D coupling image generated from three periods of the ECG signal. The system was tested on the PTB and MIT-BIH datasets and achieved classification accuracy of 98.45% and 99.2%, respectively.

The authors of [40] proposed a method that uses the image of the spectral correlation of the ECG as the system’s input to various CNN networks. For classification, they used two models of a 2D convolutional neural network. The system was evaluated on different ECG databases. Similarly, [41] proposed a two-dimensional convolutional neural network (2D CNN). Firstly, ACDCT features and cepstral properties were extracted from ECG signals. Then, transform these features from 1D to 2D. The approach was evaluated in the PTB database and achieved an accuracy of 88.57%. The method described in [42] used short-time Fourier transform (STFT) and generalized Morse wavelets (CWT) for feature extraction and 2D-CNN for classification. The model was evaluated on eight databases. The SFTF system achieved an average accuracy of 97.85%, while the CWT achieved an average accuracy of 97.5%. The author of [43] proposed an approach for biometric identification based on ECG signals obtained from fingers. Time series are converted into 2D images, and then CNN is used for classification.

The authors of [44] Proposed a person identification framework based on LSTM using ECG signals. After removing the noise from ECG signals, the ECG signal is windowed to various segments in the windowing module. After that, these segments have been fed to an LSTM network to learn the underlying representation and then classify it into the appropriate person categories. The model was evaluated on four databases and achieved an accuracy of 97.3% for the PTB database and 79.37% for CYBHi. To convert ECG signals to images, they used time-frequency representations such as the short-time Fourier transform (STFT), scalogram, Fourier synchro-squeezed transform (FSST), and wavelet synchro-squeezed transform (WSST). For evaluation, a CU-ECG database is used. Another study [45] proposed bidirectional long-short-term memory (LSTM)-based deep recurrent neural networks (DRNN) for ECG-based classification. They evaluated the method using the NSRDB and MITDB databases and achieved 100% and 99.8% classification accuracy, respectively.

The method proposed in [46] used RNN for ECG-based biometric classification. ECG data were fed directly to RNN without feature extraction. The proposed method was evaluated on the ECG-ID and MIT-BIH Arrhythmia (MITDB) datasets and achieved a classification accuracy of 100%. Another study [47] proposed a Bidirectional Gated Recurrent Unit (BGRU) model based on deep RNN networks in bidirectional training with LSTM and GRU cell units. The approach was evaluated with the ECG-ID Database and MIT-BIH and achieved classification accuracy of 98.60% and 98.40%, respectively. The authors of [48] proposed a parallel, multi-scale, one-dimensional residual network based on three kernels using center and margin loss during the training. The proposed method was evaluated in three databases and achieved 98.24%, 100%, and 95.99% classification accuracy. Another study [49] proposed a method using residual depth-wise separable convolutional neural networks (RDSCNN). They used Hamilton’s method for ECG beat detection, and the RRDSCNN algorithm was used for classification. The method was evaluated in the ECG-ID and MIT-BIH databases and achieved classification accuracy of 98.89% and 97.92%, respectively.

The authors of [50] proposed strategies for Electrocardiogram (ECG)-based identification. This paper used a DAE to learn the features of ECG signals automatically and a DNN for ECG identification. The evaluation Result achieved an average accuracy of 94.39%. To extract identity-related information, a discrete wavelet transform (DWT) was used. Additionally, an autoencoder (AE) was used for further feature extraction. A softmax classifier is used for the identification process. The experiment performed on the ECG-ID and MIT-BIH-AHA databases showed identification accuracies of 92.3% and 96.82%, respectively. The study [51] combined generalized S-transformation (GST) and a convolutional neural network (CNN). The generalized S-transformation is used to convert the one-dimensional signals to two-dimensional ones. Then, they used CNN to extract deeper features from the trajectory automatically. Experiment results show an identification rate of 96.63%. The authors of [52] proposed a system based on ECG and PPG. Used a combination of CNN and RNN. Experiments performed on the TROIKA database Achieved a recognition accuracy of 94%.

A comparison of the deep learning techniques is shown in

Table 2. Comparison of affective states-related work.

Work	Year	Feature Extraction	Classifier	Dataset	Results
[30]	2017	Not required	CNN	CEBSDB WECG FANTASIA NSRDB STDB MITDB AFDB VFDB	93.5% (Averaged of all datasets)
[31]	2018	Complex QRS detection	CNN	PTB	100%
[32]	2020	R-peak and CWT	CNN	PTB	99.83%
[33]	2021	R-peak and CWT	CNN	PTB ECG-ID	99.90% 98.20%
[38]	2020	CNN	CNN	FANTASIA CEBSDB NSRDB STDB AFDB	99.9% 93.1% 91.4% 92.7% 89.7%
[40]	2019	Spectral correlation Image	2D-CNN	CEBSDB NSRDB Fantasia MITDB STDB AFDB VFDB PTDB Combined ECG Database	99.6% 98.7% 98.2% 96.5% 98% 94.4% 85.2% 94.9% 94.9%
[51]	2018	Not required	CNN	ECG-ID	96.63%
[45]	2020	Not required	LSTM DRNN	NSRDB MITDB	100% 99.8%
[46]	2017	Not required	RNN	ECG-ID MITDB	100%
[2]	2022	LSTM 2D-CNN	LSTM 2D-CNN	CU-ECG	95.12% (two LSTM) 97.67% (2D-CNN)
[1]	2019	DWT + S-AEs (Sparse Autoencoder)	Softmax	ECG-ID MIT-BIH	92.3% 96.82%
[25]	2021	Not required	CNN	Biosec1 Biosec2 PRRB	Avg ACC = 87% AVG ACC = 87.1%
[24]	2018	CNN	CNN	TROIKA PULSE-ID	83.2% 78.2%
[42]	2018	STFT images CWT images	2D-CNN	CEBSDB NSRDB Fantasia MITDB STDB AFDB VFDB PTDB	Avg ACC = 97.85% (STFT) AVG ACC = 97.5% (CWT)

. The table shows the method’s feature extraction, classifier, datasets, and accuracy. The authors used different feature extraction methods and datasets. It is worth noting that CNN is the most commonly used classifier.

3. Materials and Methods

The proposed system includes the following phases: First, we will preprocess the ECG to remove the noise and artifacts from the ECG signals by using a discrete wavelet transform (DWT). Secondly, we will transfer the 1-D preprocessed ECG signals into a 2-D spectrogram image using the STFT approach. Thirdly, the feature extraction and optimization steps are performed using VGG16 and RNN-LSTM deep learning (DL) methods, respectively. Finally, the boosting classifier has been used to identify the features and predict the human. Figure 2 shows the methodology phases, and Algorithm 1 shows the ECG biometric recognition system.

Algorithm 1: Cardiac Biometrics for Human Identification using Deep Learning
	Input: Raw ECG Biosignals
Step 1:	Preprocessing
	- Remove noise and artifacts from the ECG biosignals.
	- Ensure data quality and reliability.
Step 2:	Transformation Phase
	- Convert the 1-D ECG signals into 2-D spectrogram images.
	- Represent the temporal and frequency information of the ECG signals.
Step 3:	Feature Extraction using Ensemble Deep Learning
	- Initialize the VGG16 pre-trained transfer learning (TL) model and LSTM model.
	- Pass the spectrogram images using both models to obtain intermediate feature representations.
	- Optimize features using the LSTM model.
Step 4:	Boosting Machine Learning Classifier
	- Train a boosting machine learning classifier (e.g., AdaBoost or Gradient Boosting) on the ensemble features.
	- The classifier learns to recognize and distinguish between individuals based on the cardiac biometric information.
Step 5:	Human Identification
	- Feed new ECG biosignals using the preprocessing and transformation phases.
	- Extract the ensemble features using the trained models from Step 3.
	- Use the boosting machine learning classifier to recognize and identify the individual based on the extracted features.
	Output: Identified Human or Rejection (if unknown/unauthorized)
Step 6:	Algorithm 1 End

3.1. Data Acquisition

We will use the ECG-ID dataset, which is available freely as a part of the Physionet database. The dataset consists of 310 electrocardiogram (ECG) recordings from 90 individuals (44 men and 46 women) aged between 13 and 75 years. Each recording takes 20 s and captures ECG lead I, which is digitized at 500 Hz with 12-bit resolution over a nominal 10 mV range. The recordings are accompanied by annotations for 10 beats, including unaudited R- and T-wave peaks generated using an automated detector. Additional information regarding the age, gender, and recording date is available in the “.hea” file for each record. The number of recordings per individual ranges from 2 to 20, collected either during a single day or periodically over a period of 6 months. The raw ECG signals contained significant noise and included both high- and low-frequency noise components. Each record had two signals: Signal 0 captured the raw ECG I signal, while Signal 1 captured the filtered ECG I signal [53]. Figure 3 shows random samples from the ECG-ID database before preprocessing.

3.2. ECG Signal Preprocessing

To preprocess the ECG signals, the following subsections present the preprocessing steps, which are data normalization and noise removal.

Data Normalization: The signals were normalized in the range of 0 and 1. This work adapted min-max normalization, which can be seen in Equation (1) as:

$$x=\frac{x-x_{min}}{x_{max}-x_{min}}$$

(1)

where $x$ is the original signal and $x_{min}$, $\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{l}\mathrm{y}$, represent the minimum and maximum values of the original signal.

The Discrete Wavelet Transform (DWT) is employed in ECG signal preprocessing for baseline correction and noise removal. This technique involves breaking down the ECG signal into different frequency scales, or subbands, using wavelet functions. By analyzing these subbands, baseline wandering and noise components can be identified in higher frequency ranges. Applying a thresholding operation to coefficients in these subbands reduces or eliminates these unwanted components, resulting in a denoised and baseline-corrected ECG signal. This processed signal retains essential features while removing distortions caused by baseline variations and noise. The denoised signal can then be used for subsequent tasks, such as feature extraction for biometric identification, where accurate ECG pattern recognition is critical for reliable authentication.

Noise Removal: The ECG signals are affected by different noises like muscle artifacts, baseline wander, and power line interference [54]. Many approaches have been used to remove the noise from the ECG signal. A discrete wavelet transform (DWT) method is utilized in this paper to remove noise from the ECG signal. The Daubechies wavelet of order 1 (dB1) was selected for denoising. We used the DWT technique compared to other methods because the Discrete Wavelet Transform is a non-parametric feature extraction method for noisy signals. It is a good technique for detailed feature extraction and approximation of signals because it is a joint time-frequency resolution analysis. Figure 4 compares an ECG signal from the ECG-ID before and after applying DWT. After the EEG signal is filtered, the DWT acquires both the approximate and detailed coefficients from the ECG signal. Then, the approximate coefficient is iteratively divided into new approximate and exact coefficients, providing a set of complex and approximate coefficients [55].

The mathematical equation for DWT is given in Equation (2) [56].

$${DWT\left(s,l\right)=2}^{-s}{\sum }_{n}x\left[n\right](2^{-s}n-1)$$

(2)

where $l$ represents the location parameter, and s represents the dilation parameter, n = 1, 2, and the total number of samples N and Complex Conjugate of analyzing wavelet function ψ*. DWT decomposes the signal into low-frequency elements as Approximation Coefficients A_∅ that given by Equation (3), and high-frequency elements as detailed coefficients Dψ that is given by Equation (4).

$$A_{{\varnothing }_{(S_0,l)}}=\frac{1}{\sqrt{N}}{\sum }_{n}x\left[n\right]{\varnothing }_{S_0,l}\left[n\right]$$

(3)

$$D_{{\psi }_{(S,l)}}=\frac{1}{\sqrt{N}}{\sum }_{n}x\left[n\right]{\psi }_{s,l}\left[n\right]$$

(4)

where $S_0=0$, N = 2^s, s = 1, 2, 3, …, S; n = 1, 2, 3 … N; $l$ = 1, 2, 3, … 2^s−1, n = Number of samples, S determines the width of $D_{{\psi }_{(S,l)}}\left[n\right]$, 2^s/2 is the amplitude of the function, l is the position vector of ψ(s, l) [n], and ψ is the wavelet coefficient.

3.3. Preprocessed Signal Transformation

After removing noise, the next step is to convert preprocessed 1-D multi-channel signals into 2-D spectrogram images using the short-time Fourier transform (STFT) method. The Short-time Fourier transform (STFT) is a time-frequency analysis used to analyze non-stationary signals. It derives from the discrete Fourier transform (DFT). The STFT method divides a long-time signal into segments with the same size as the window, and the Fourier transform is applied to each segment [57]. Figure 5 shows the spectrogram of random samples after applying STFT. The discrete STFT X of the signal x is given by Equation (5) [56]:

$$X\left(m,k\right)=\sum _{n=0}^{N-1}x\left(n+mH\right)w\left(n\right)\exp \left(\frac{-2\pi ikn}{N}\right)$$

(5)

$$X\left(k\right)=\mathrm{x}\mathrm{ˆ}\left(\frac{k}{N}\right)\sum _{n=0}^{N-1}x\left(n\right)\exp (-2\pi ikn)$$

(6)

with m ∈ Z and k ∈ [0:K]. The frequency index is K = N/2, corresponding to Nyquist frequencies (assuming N is even). $X\left(m,k\right)$ is the complex number that denotes the kth Fourier coefficient for the mth time frame. For each fixed time frame m, one obtains a spectral vector of size K + 1 given by the coefficients $X\left(m,k\right)$ for k ∈ [0:K]. The computation of each such spectral vector amounts to a DFT of size N as in Equation (6), which can be done efficiently using the FFT.

The ECG signal is sampled at a rate of around 200 to 1000 samples per second. If we consider a typical duration of 10 s and a sampling rate of 250 samples per second, the size of the ECG signal would be approximately 2500 data points (samples). In the described methodology, spectrograms were created from these data points through the utilization of the Short Time Fourier Transform (STFT). The resultant spectrograms are represented in RGB format and have dimensions of 512 × 512 × 3. The spectrogram’s power values are encoded into an RGB color image, thereby accentuating the contrasts in one version. This approach aims to visually highlight the variations and power distribution of the signals in a graphical representation, facilitating the identification and understanding of patterns and features in the ECG data.

3.4. Deep Features Extraction from Spectrogram

For feature extraction, a VGG16 model has been used. VGG-16 is a popular deep convolutional neural network (CNN) architecture that is 16 layers deep. VGG-16 is known for its ability to extract hierarchical features from images. Since this work uses a deep neural network pre-trained on images as a feature extractor for ECG signals, the data instances must be transformed into images. So, we use spectrograms. Spectrograms can capture the changes in the signal’s power in a photo by taking the Fourier transform of each partition of the movement.

To extract the feature vectors from the spectrograms, we acquired a Keras implementation of VGG16. The VGG16 pre-trained weights and model were used. We add a global average pooling layer to the model’s output to remove features using this pre-trained model on spectrogram images. Extracting features using the VGG-16 model from a 2D spectrogram image involves passing the image through the pre-trained VGG-16 model and capturing the intermediate feature representations from specific model layers.

The concept of transfer learning (TL) [57] serves as a strategic framework within the domain of deep learning (DL), enabling the application of insights garnered from one issue to address a comparable challenge in another context. Transfer learning methods can generally be categorized into four groups: instance-based, parameterized, feature-representation, and relational knowledge transfer. Among these, the feature-representation transfer learning approach is well-suited for the task. In our proposed method, a pre-trained convolutional neural network (CNN) such as VGG-16 is employed to extract domain-independent image features, thereby enhancing the transferability of knowledge from a source to a target domain. Pre-trained models are combined with fine-tuning techniques to adapt them to new tasks. This involves adding supplementary layers to accommodate new classes, a process we adopt in our approach through fine-tuning. Here, a pre-trained model is extended using fine-tuning to align with the requirements of a novel dataset, obviating the need for training from scratch and expediting the process. Additional layers are adjusted during training to optimize the model’s performance, aligning it with the dataset’s characteristics.

Fine-tuning is executed with a minimal learning rate to ensure minimal disruption to the previously acquired knowledge. Starting with freezing all layers and only learning the final classifier, we gradually unfreeze and modify more layers top-down. Increasingly unfreezing layers may enhance network performance, but excessive unfreezing can lead to diminished effectiveness due to limited data. Overfitting, a concern when training on limited data, is managed using strategies like dropout layers and L2 regularization. The methodology is demonstrated using the VGG16 model, with its initial weights as the starting point. After global average pooling, a dropout layer is incorporated, and L2 regularization is employed to prevent overfitting. The model’s training weight from ImageNet is adopted to establish a strong foundation. Since our task involves binary classification, we utilize the binary cross-entropy logarithmic loss function to optimize model performance and guide its learning process.

The pseudocode for this step is shown in Algorithm 2.

Algorithm 2: Feature extraction using VGG16
Step 1:	Input: images after applying single-level Short-Time Fourier Transformation
Step 2:	Output: important features
Step 3:	Import the required libraries (Keras)
Step 4:	Load the pre-trained model (VGG16) using the Keras application module.
Step 5:	Define a new model that takes the output of the pre-trained model as input and adds additional layers to perform feature extraction (Global Average Pooling2D Layer)
	(a)  Load the images and apply some preprocessing steps (e.g., resize, normalize, etc.).
	(b)  For image in images:
	(c)  Predict the output of the model, which give the important features of the image.
	(d)  End for
Step 6:	Save the extracted features for further analysis or use in downstream tasks (e.g., classification, object detection, etc.).
Step 7:	Return features

3.5. Optimization Deep Features

For optimizing the features, the LSTM has been used. LSTM is an extension of a recurrent neural network (RNN). An RNN is a neural network with recurrent outputs and inputs. In LSTMs, the exploding and vanishing gradient problems are overcome [57]. The features optimized by the RNN-LSTM model were derived from the intermediate representations extracted using the VGG16 model, which was used as part of the ensemble deep learning approach in the cardiac biometrics system.

The ensemble feature representation was then fed to a boosting machine learning classifier (e.g., AdaBoost or Gradient Boosting) for human identification. The classifier learned to differentiate between individuals based on the combined information from the VGG16 and LSTM models. The optimization process using the RNN-LSTM model aimed to leverage the complementary strengths of the VGG16 model (spatial representation) and LSTM model (temporal representation). By combining the two approaches, the system could potentially achieve enhanced accuracy and robustness in recognizing and identifying individuals based on their unique cardiac biometric data captured using ECG signals.

An LSTM is composed of three gates: input, forget, and output. LSTM’s activation functions involve sigmoid and hyperbolic tangents [58]. Figure 6 shows the architecture of an LSTM cell. Here, the input is the features that have been extracted by the VGG16 pretrained model. The model architecture comprises an LSTM layer with 32 units and an input shape of (1024,1) for each feature from the preceding step. Subsequently, a dense layer with 90 units corresponding to the classes of patients is included. The model was compiled using specific hyperparameters, including a Softmax activation function in the output layer, categorical cross entropy as the loss function, and the Adam optimizer for optimization. These hyperparameters yielded optimal performance outcomes. The pseudocode for this step is shown in Algorithm 3.

Algorithm 3: Features optimization using LSTM model
Step 1:	Input: features from the VGG16 model
Step 2:	Output: optimized features
Step 3:	Prepare the features shapes to suit the input of LSTM.
Step 4:	Define the LSTM model with the following components:
Step 5:	- LSTM layer with 32 units and input shape of (features.shape [1], features.shape [2])
	(a)  - Dense layer with 90 units and softmax activation function
	(b)  Compile the model with categorical cross-entropy loss function, Adam optimizer, and accuracy metric.
	(c)  For all the elements in the features list:
	(d)  Predict the output of the model, which gives the important features.
Step 6:	End for
Step 7:	Return optimized features
Step 8:	Input: features from the VGG16 model
Step 9:	Output: optimized features

3.6. Biometric Human Identification

After extracting the ensemble features (combining VGG16 and LSTM representations) from the 2D spectrogram images, we can proceed with the classification step using the LightGBM model. LightGBM is a gradient-boosting framework that excels at handling large-scale datasets and offers fast and efficient training. Using the LightGBM model for classification complements the initial feature extraction steps using the VGG16 and LSTM models. LightGBM’s ability to handle large-scale datasets efficiently and its excellent predictive performance make it a valuable tool for the classification stage in your cardiac biometrics system.

The LightGBM uses a tree-based learning algorithm. In this algorithm, trees are grown vertically using a leaf-wise algorithm [58]. Since LightGBM generates complicated trees, it’s more accurate than other tree-boosting algorithms. Additionally, LightGBM has a high speed with low memory usage because it uses the GOSS and EFB algorithms [59]. The model was applied to the optimized features from the LSTM step, and the data were split into 80% training and 20% testing. The pseudocode for this step is shown in Algorithm 4.

A biometric human identification system is a critical application in the field of security and surveillance. We used several signal processing techniques, such as the Discrete Wavelet Transform (DWT) and the Short-Time Fourier Transform (STFT), for feature extraction from ECG signals. These features are then fed into machine learning models to classify and identify individuals. In addition to signal processing techniques, feature engineering plays a crucial role in biometric identification. Feature engineering involves selecting and transforming the relevant features that can improve the accuracy of the machine learning model. The features were extracted from spectrogram images using the VGG16 pre-trained model and optimized using the LSTM model.

A LightGBM model was used for the biometric identification task. This model is trained on the ECG-ID dataset and optimized using hyperparameter tuning techniques to improve its performance. Overall, the combination of signal processing techniques, feature engineering, and machine learning models has shown promising results in biometric human identification and has the potential to significantly improve security and surveillance systems. As shown in Figure 7, the system is able to recognize humans from their heartbeats (if human heartbeats are in the dataset).

Algorithm 4: Classification using the LightGBM model
Step 1:	Input: optimized features from the LSTM model
Step 2:	Output: Classification of the humans based on the ECG signals.
Step 3:	Load the optimized features after the LSTM step for feature optimization.
Step 4:	Split the dataset into training and validation sets using ‘train_test_split’ function from ‘sklearn.model_selection’.
Step 5:	Create ‘lgb.Dataset’ objects for the training and validation data.
	(a)  Define the LightGBM parameters in a dictionary.
	(b)  Train the model using the ‘lgb.fit’ function and the training data.
	(c)  Evaluate the model performance on the validation set using appropriate metrics (Accuracy, F1_score, Precision, and Recall metrics)

4. Results and discussions

4.1. Evaluation Metrics

The goal of the evaluation step is to estimate the performance and effectiveness of the proposed model. As shown in

Table 3. Confusion Matrix for Classification.

	Actual Positive Class	Actual Negative Class
Predicted Positive Class	True Positive (TP)	False Negative (FN)
Predicted Negative Class	False Positive (FP)	True Negative (TN)

, the confusion matrix can be used to evaluate the optimal solution during classification training. According to the confusion matrix, TP and TN represent the number of positive and negative instances that are correctly classified, while FP and FN represent the number of positive and negative instances that have been misclassified. To assess the performance of the proposed classifier, several metrics can be generated from

Table 4. Hyperparameters for VGG16 model.

GPU	Activation Function	Regularization
Google Collaboratory T4	Sigmoid	L2

. The evaluation metrics are Accuracy, Precision, Recall, and F1 Score [60].

Accuracy: Generally, accuracy is measured by the ratio of correct predictions to the total number of instances assessed. It can be represented mathematically as seen in Equation (7):

$$\mathrm{Accuracy}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N}}$$

(7)

Precision: is a measure of the number of positive patterns correctly predicted from the total number of positive patterns in a class. It can be represented mathematically as seen in Equation (8):

$$\mathrm{Precision}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$

(8)

Recall: is used to measure the percentage of actual positive samples that were correctly classified. It can be represented mathematically as seen in Equation (9):

$$\mathrm{Recall}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(9)

F1-Score: This metric is defined as the harmonic mean of the recall and precision values. It can be represented mathematically as seen in Equation (10):

$$\mathrm{F}1=2\ast \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\ast \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}$$

(10)

4.2. Environment Setup

To analyze the effectiveness of the proposed method. The dataset in the experiments was split into 80% training and 20% testing. To train the model, we utilized the GPU runtime available in the Google Collaboratory. The activation function method used in the VGG16 model was Sigmoid, and the regularization used was L2. Additionally, we used pre-trained weights from the ImageNet dataset, which the model was previously trained on. The input image size for training was set to (224, 224, 3). The suggested network hyperparameters for VGG16 are described in Table 4. The LSTM model was trained with SoftMax as an activation function, Adam as an optimizer, and Categorical cross entropy as a loss function. The suggested network hyperparameters for LSTM are described in

Table 5. Hyperparameters for LSTM model.

GPU	Activation Function	Optimizer	Loss
Google Collaboratory T4	Softmax	Adam	Categorical cross-entropy

. The LightGBM classifier was trained with a learning rate of 0.05, a feature fraction of 0.8, and a number of leaves equal to 50. To quantify the loss, we employed the multi_log loss function. The suggested hyperparameters for LightGBM are described in

Table 6. Hyperparameters for LightGBM model.

GPU	Learning Rate	Number of Leaves	Feature Fraction	Loss	Validation
Google Collaboratory T4	0.05	50	0.8	Multi log loss	10-fold

.

4.3. Result Analysis

This section will present the experimental results. Section 4.5. presents a comparison between VGG16 and other different pre-trained models to demonstrate that VGG16 obtains the best results. In Section 4.4., the fine-tuning model hyperparameters for optimizing accuracy results were presented. Section 4.3. shows the result and performance. Also, in Section 4.5, we compared the results of our classification method with the state-of-the-art works that used the ECG-ID database with different classifiers to better assess the proposed model’s performance in comparison to other systems.

We compared the classification results of different feature extraction models like VGG19, Inception v3, and densenet121 with VGG16. The results are shown in

Table 7. Comparison between different models with VGG16.

Model	Accuracy	Precision	Recall	F1-Score	Loss	Training Time (s)
VGG19	94.4%	98.6%	98.9%	98.7%	18.2	194.535
Inception V3	93.7%	98.1%	98.7%	98.4%	14.5	228.726
Densenet121	93.7%	97.5%	98.4%	97.8%	13.5	227.383
VGG16	98.7%	98.2%	98.7%	98.4%	6.2	205.521

. The highest accuracy for the LightGBM classification model is 98.7%, which was obtained by the feature extraction model VGG16. Additionally, VGG16 got the lowest loss with 6.2, and it took 205.521 s for training. Based on this comprehensive evaluation, the VGG16 model emerges as the most effective. As a result of this comparison, we have chosen to utilize the VGG16 model for the feature extraction phase.

4.4. Fine Tuning Models Hyperparameters

To improve the effectiveness of our system, optimizing hyperparameters is crucial.

Table 8. The commonly used hyperparameters in VGG16, LSTM, and LightGBM models.

Model	Hyperparameter	Description
VGG16	Learning Rate (LR)	Determines the step size for gradient updates during training.
	Batch Size	Number of samples processed before updating the model weights.
	Number of Epochs	Number of times the entire dataset is passed through the model during training.
	Optimizer	The optimization algorithm used for weight updates (e.g., SGD, Adam, RMSprop).
	Dropout Rate	Probability of dropping out neurons during training to prevent overfitting.
	Weight Decay (L2 Regularization)	Penalty added to the loss function to discourage large weight values.
LSTM	Number of LSTM Units	Number of memory cells (neurons) in the LSTM layer.
	Learning Rate (LR)	Step size for updating LSTM model parameters during training.
	Batch Size	Number of sequences processed together before updating the model.
	Number of Epochs	Number of iterations over the entire dataset during training.
	Optimizer	The optimization algorithm used for LSTM weight updates (e.g., Adam, RMSprop).
	Dropout Rate	Probability of dropping out LSTM neurons during training to prevent overfitting.
	Weight Decay (L2 Regularization)	Penalty added to the loss function to discourage large weight values.
LightGBM	Learning Rate (LR)	Step size for updating the boosting model during training.
	Number of Estimators	Number of boosting rounds (trees) in the LightGBM model.
	Max Depth	Maximum depth of the decision trees in the boosting model.
	Min Child Samples	Minimum number of samples required in a leaf node to split a tree.
	Feature Fraction	Fraction of features used in each boosting iteration to prevent overfitting.
	Bagging Fraction	Fraction of training data used in each boosting iteration to prevent overfitting.
	Lambda (L2 Regularization)	L2 regularization term applied to the leaves’ values in the decision trees.

lists the hyperparameters commonly used in VGG16, LSTM, and LightGBM models. In this section, a grid search technique was used on the selected dataset to optimize hyperparameters like learning rate, dropout value, LSTM unit, regularization L2, and optimizer. The grid search involved changing the hyperparameter values so that the results of the best combination of them could be compared together.

Table 9. Fine-tuning models hyperparameters.

Model Experiment No.	Learning Rate	Dropout	LSTM Unit	Regularization	Optimizer	Estimators	Max Depth	Min Child	Feature & Bagging Fraction
1	0.001	0.2	32	0.001	Adam	100	6	20	0.7
2	0.01	0.5	32	0.01	Adam	50	6	20	0.7
3	0.09	0.5	32	0.0001	Adam	120	8	75	0.8
4	0.05	0.2	32	0.09	Adam	25	10	50	0.8
5	0.1	0.25	64	0.09	RMSprop	30	6	25	0.9
6	0.01	0.3	64	0.9	RMSprop	80	6	75	0.9
7	0.9	0.7	128	0.005	RMSprop	250	6	20	0.7

shows the evaluated and optimal values for each hyperparameter, determined by maximizing accuracy and minimizing computation time. For the learning rate, the tested values were (0.1, 0.01, 0.001, 0.05, and 0.09), and the optimized value was (0.05). The dropout values were tested at (0.2, 0.25, 0.5, and 3), and the optimized value was (0.2). The LSTM unit values were tested at (32, 64, and 128), and the optimized value was (32). Regularization was tested at (0.01, 0.001, 0.0001, 0.09, and 0.9) and the optimized value (0.09). Finally, optimizer techniques (RMSprop and Adam) were tested, with Adam selected as the optimal choice.

Among the seven experiments conducted to train the models for biometric human identification, the best-performing model was achieved by optimizing the hyperparameters in Table 9. As we can see from

Table 10. Experimental results were obtained by a proposed ensemble classifier (VGG16-LSTM-LightGBM) with a preprocessing step on seven different runs.

No.	Accuracy	Precision	Recall	F1-Score
1	97.7%	97.7%	97.7%	0.97
2	98%	96%	98%	0.98
3	97.7%	95.4%	96.3%	0.97
4	98.7%	98.01%	97.1%	0.98
5	97.5%	96.3%	95.1%	0.97
6	97.3%	95.1%	96.2%	0.97
7	94.9%	94.2%	95.3%	0.94

, experiment 4 achieves the best performance. In this experiment, the estimators were set to 25, the number of leaves in each tree was set to 50, the maximum depth of each tree was set to 10, and the feature bagging and fraction were set to 0.8. These hyperparameters were chosen after careful experimentation and tuning, and they helped to improve the accuracy of the model significantly. Overall, by carefully selecting and optimizing these hyperparameters, the model achieved a high level of accuracy in biometric human identification tasks. This approach has significant potential for improving security and surveillance systems in a range of applications.

The testing performance of the pre-trained models we compared in terms of the confusion matrix is shown in Figure 8. The main diagonal of the confusion matrix represents the correctly classified instances where the predicted class matches the actual style. Off-diagonal elements indicate misclassifications, with each cell showing how many models were incorrectly assigned to a particular category. As shown in the figure, incorrect predictions are represented in the red squares.

The comparison shows the accuracy of each feature extraction model. As we can see from Figure 9, the VGG16 model got the best accuracy at 98.7%.

The comparison shows the loss function of each feature extraction model. As we can see from Figure 10, the Loss function ranges between 0.0 and 0.2. The best loss value achieved by the VGG16 model is 0.062.

The comparison shows the training time taken by each model. As shown in Figure 11, the VGG16 model took 205.521 s for training.

4.5. State-of-the-Art Comparisons

The proposed approach’s classification result is compared with other state-of-the-art works that tested their systems using the ECG-ID database. The comparison is shown in

Table 11. Comparison with state-of-the-art works along with preprocessing steps.

Work	Classifier	Accuracy	Precision	Recall	F1-Score
[1]	DWT and S-AE and 1D-ECG	82.3%	84.3%	83.3%	0.82
[21]	PCANet and MaxFusion and SVM	85.70%	86.20%	83.30%	0.85
[49]	CNN	88.63%	84.13%	86.60%	0.88
[50]	CNN and RNN	87.11%	88.02%	86.01%	0.87
Proposed	VGG-16-LSTM and LigthGBM	98.7%	98.01%	97.1%	0.98

.

We discussed cardiac biometrics using deep learning, focusing on electrocardiogram (ECG) signals. The paper proposed an ensemble approach, combining VGG16 and LSTM architectures, for feature extraction from 2D spectrogram images. Based on their cardiac biometric data, the system achieved impressive accuracy and performance in recognizing humans. The proposed system holds the potential for secure authentication in various applications. This section presented and discussed the result of our classification model and compared the effect with different models and other state-of-the-art works. The comparison shows that our model achieved the best result. Compared to other systems, the proposed approach based on VGG-16-LSTM and LightGBM achieved 98.7% accuracy, 98.01% precision, 97.1% recall, and 0.98 AUC.

Table 12. Comparison with state-of-the-art works without preprocessing steps.

Work	Classifier	Accuracy	Precision	Recall	F1-Score
[1]	DWT and S-AE and 1D-ECG	85.50%	86.13%	87.30%	0.85
[23]	PCANet and MaxFusion and SVM	83.11%	85.23%	84.10%	0.84
[51]	CNN	83.22%	84.00%	82.10%	0.83
[52]	CNN and RNN	80.20%	81.19%	83.10%	0.82
Proposed	VGG-16-LSTM and LigthGBM	92.10%	93.22%	94.40%	0.94

presents a comparison of different state-of-the-art preprocessing steps. Various classifiers were evaluated for their performance metrics, including accuracy, precision, recall, and F1-score. The works include methods like DWT combined with S-AE and 1D-ECG [1], achieving an accuracy of 85.50% with corresponding precision, recall, and F1-score values. Another approach employed PCANet, MaxFusion, and SVM [21], resulting in an accuracy of 83.11% and comparable precision, recall, and F1-score. A CNN-based method [51] achieved an accuracy of 83.22% but showed slightly lower precision and recall. Another combined CNN and RNN [52] approach achieved an accuracy of 80.20% with precision, recall, and F1-score values. Notably, the proposed method, which employs VGG-16-LSTM and LightGBM, outperformed the others with a high accuracy of 92.10% and superior precision, recall, and F1-score values. However, the proposed technique with preprocessing steps achieved higher accuracy in identifying humans based on ECG spectrogram images. Additionally, Figure 12 shows different machine learning algorithms used in comparison with the LightGBM classifier. In practice, LightGBM stands out among other machine learning algorithms due to its unique combination of efficiency, scalability, and predictive power. Unlike traditional gradient boosting algorithms, LightGBM employs a novel approach to tree building, focusing on feature-based partitioning and histogram-based techniques, leading to significantly faster training times and lower memory usage. This efficiency makes LightGBM particularly well-suited for large and high-dimensional datasets, where it can handle complex feature interactions while maintaining a small computational footprint. Additionally, LightGBM’s ability to handle imbalanced datasets and its built-in feature importance ranking contribute to its versatility and effectiveness in a wide range of applications. These advantages collectively position LightGBM as a compelling choice for tasks requiring accurate predictions, interpretability, and resource-efficient performance.

4.6. Computational Analysis

Analyzing the computational complexity of each step in the proposed pipeline using Big O notation, as mentioned in

Table 13. Computational analysis of a proposed system to identify humans.

Step	Complexity	Example Size (n)
Preprocessing Steps	O (n2)	1000
1D ECG to 2D Spectrogram Transformation	O (n) (approx.)	1000
Feature Extraction (VGG16 Pretrained Model)	O (1,611,840)	224
Feature Optimization (LSTM)	O (838,860,800)	256
Biometric Identification (LightGBM Classifier)	O (3,845,620)	1000

. The most computationally intensive steps are typically the ones involving feature extraction, optimization, and training of the machine learning models (VGG16, LSTM, and LightGBM). It’s important to note that the actual complexity might vary based on specific implementation details (CPU, GPU, TPU). Biometric Identification (LightGBM Classifier): The training complexity of the LightGBM classifier depends on the number of trees (T), the number of features (F), and the number of samples (N). The complexity is approximately O(T × F × N × log N) and calculated on the input ECG signal length with n of 1000 samples.

5. Discussions

The demand for robust and secure human identification systems has reached a critical juncture in today’s digital landscape, characterized by widespread and continuous digital interactions. Conventional recognition methods, such as personal identification numbers (PINs) and passwords, have become increasingly susceptible to attack, resulting in potential security breaches, lost credentials, or forgotten access codes. As a result, there is a compelling need to innovate and develop biometric systems that offer heightened security and reliability. Recognizing this trend, modern practices have seen a rise in the use of biometric methods to augment or replace traditional PIN-based security measures, mitigating the vulnerabilities posed by the loss or theft of personal identification information. These advanced biometric technologies securely manage personal information and verify users’ identities with accuracy and assurance unmatched by conventional methods.

Biometric recognition encompasses a diverse range of physiological and behavioral human characteristics, including attributes like voice, gait, and electrocardiogram (ECG) signals and physical traits such as facial features, fingerprints, and iris patterns. However, conventional biometric systems face challenges, particularly regarding spoofing and forgery. Consequently, there has been a growing interest in adopting biometric authentication methods that harness an individual’s unique physiological and behavioral traits to enhance security and alleviate vulnerabilities significantly. Cardiac biometrics have emerged as a promising and innovative approach to address the imperative of human identification security among the diverse biometric modalities.

At the core of cardiac biometrics lies the intrinsic electrical activity of the human heart, encapsulated in signals such as the electrocardiogram (ECG), photoplethysmograph (PPG), and phonocardiogram (PCG). These signals carry rich and distinctive information that can be harnessed for secure authentication. Within this landscape, the utilization of ECG signals for biometric-based human identification stands out due to its unique advantages over alternative approaches like PPG and PCG signals. ECG signals provide an inherently individualistic biometric marker owing to the intricate patterns woven into the heart’s electrical activity. This inherent uniqueness significantly bolsters the security and precision of identification systems. These advantages position ECG signals as an optimal choice for constructing biometric-based human identification systems distinguished by accuracy, safety, and reliability, setting them apart from PPG and PCG approaches.

Deep learning has demonstrated exceptional prowess in deciphering complex patterns and representations across diverse data domains, making its integration into cardiac biometrics an exciting frontier in the pursuit of dependable and efficient authentication systems. The primary objective of this study is to introduce an ensemble methodology that effectively harnesses the strengths of a pre-trained VGG16 transfer learning (TL) framework in tandem with long-short-term memory (LSTM) architectures. This amalgamation aims to extract optimal features from two-dimensional spectrogram images derived from ECG biosignals. This novel ensemble feature representation capitalizes on cardiac signals’ temporal and spatial characteristics, serving as a comprehensive and discriminative foundation for human identification.

The research journey comprises several distinct phases, encompassing the preprocessing of ECG biosignals to ensure data quality, transforming one-dimensional ECG signals into two-dimensional spectrogram images, and the subsequent feature extraction utilizing the ensemble deep learning technique. Furthermore, the identification process is facilitated by a machine learning classifier that distinguishes individuals based on their distinctive cardiac biometric attributes. The proposed cardiac biometric system’s efficacy is rigorously evaluated using extensive experimentation on a curated dataset. Key performance metrics such as accuracy, sensitivity, specificity, and the area under the curve (AUC) are employed to gauge the system’s effectiveness. Furthermore, a comparative analysis is conducted against prevailing state-of-the-art biometric authentication systems, showcasing the proposed system’s superiority in accurately recognizing human identities.

5.1. Future Directions

Future research in the field of cardiac biometrics and human identification can explore several exciting avenues:

• Conduct research on more extensive and diverse datasets to evaluate the system’s performance under various conditions and demographics. Access to larger datasets can help improve the generalization and robustness of the proposed system.
• Investigate the integration of multiple biometric modalities (e.g., ECG, PPG, PCG, fingerprint, and facial recognition) for even more reliable and accurate human identification. Combining multiple modalities can enhance security and reduce the risk of spoofing attacks.
• Explore the feasibility of continuous authentication using cardiac biometrics. Investigate how the system can continuously monitor and verify individuals’ identities over extended periods, ensuring a seamless and secure user experience.
• Experiment with more advanced transfer learning techniques, including fine-tuning the LSTM model on a related dataset or employing other pre-trained models that are specifically designed for sequential data analysis.
• Enhance the interpretability of the system’s decision-making process. Investigate methods to understand which features or patterns contribute most to the identification process, making it easier to diagnose and address potential issues.
• Conduct rigorous adversarial testing to assess the system’s robustness against potential attacks, ensuring that it remains secure in real-world scenarios.
• Investigate ways to improve the user experience while maintaining privacy. Examine user consent, data anonymization, and secure storage methods to protect individuals’ sensitive biometric data.
• Study the long-term reliability and stability of cardiac biometrics as individuals age or undergo physiological changes. Ensure that the system maintains its accuracy and effectiveness over time.
• The field of biometrics involves authenticating and recognizing individuals using their behavioral and physical characteristics [61]. The latest progress in multimodal biometrics around physiological attributes. In the future, we will focus on more techniques utilizing finger veins, palm veins, fingerprints, facial, lip, iris, and retinal patterns to identify humans.
• At the forefront of this framework lies the utilization of spectrogram images for biometric identification. The explainable nature of AI ensures [62] that the identification process is transparent and decisions are justifiable. This is crucial in scenarios where patient data security, accuracy, and ethical considerations are of paramount importance.

By focusing on these future research directions, advancements can be made in cardiac biometrics and human identification, leading to more secure and reliable authentication systems with broad practical applications.

5.2. Practical Domains of Cardiac Biometric Applications

Cardiac-based biometric authentication is applicable to potential domains where such a system could be valuable, along with the importance of secure authentication in the following areas:

(1)

Financial Services and Banking: Secure authentication is critical in financial services and banking to prevent unauthorized access to accounts and transactions. Cardiac biometrics can enhance security by offering a highly unique and difficult-to-replicate identification method. This is important to safeguard sensitive financial information and prevent fraudulent activities.

(2)

Healthcare and Medical Records: In healthcare, accurate patient identification is crucial for maintaining medical records and ensuring proper treatment. Cardiac biometrics could provide a reliable and non-intrusive method to identify patients, reducing the risk of medical errors and unauthorized access to sensitive health data.

(3)

Government Services and ID Verification: Government agencies often require robust authentication for services like issuing identification documents, passports, and driver’s licenses. Cardiac biometrics could provide an added layer of security to ensure that individuals are who they claim to be, preventing identity theft and fraud.

(4)

Physical Access Control: Cardiac biometrics can enhance access control systems in physical environments such as offices, research facilities, and secure areas. Traditional methods like key cards or PINs can be lost, stolen, or shared, while cardiac biometrics provide a unique and difficult-to-forge way to ensure only authorized personnel gain entry.

(5)

E-commerce and Online Transactions: Online transactions and e-commerce platforms require secure authentication to protect users’ financial data. Cardiac biometrics could offer a seamless and secure way for users to confirm their identity during online purchases, reducing the risk of fraudulent transactions.

(6)

Smart Devices and Wearables: The rise of smart devices and wearables has created opportunities for continuous and passive authentication. Cardiac biometrics could be used to unlock devices, provide secure access to personal data, and monitor user health metrics.

(7)

Remote Authentication and Telecommunications: In remote authentication scenarios, where individuals access systems or services from remote locations, strong security measures are essential. Cardiac biometrics could offer a reliable way to verify identity in telecommunication applications, reducing the risk of unauthorized access.

The importance of secure authentication in these areas lies in safeguarding sensitive data, preventing identity theft, minimizing fraudulent activities, ensuring privacy, and maintaining the integrity of services. Traditional authentication methods are susceptible to various attacks and vulnerabilities, making biometric-based systems, such as cardiac biometrics, an attractive solution due to their inherent uniqueness and difficulty to impersonate. By implementing secure authentication measures, these practical domains can enhance user trust and overall system security.

6. Conclusions

In this paper, we have developed a novel cardiac biometric system for human identification using deep learning approaches. The system leverages the unique physiological characteristics of individuals captured using electrocardiogram (ECG), photoplethysmogram (PPG), and phonocardiogram (PCG) signals. To achieve secure authentication, we proposed an ensemble approach based on pre-trained VGG16 transfer learning (TL) and Long Short-Term Memory (LSTM) architectures.

The system follows a multi-phase approach. In the first phase, we preprocessed the ECG biosignals to remove noise and ensure data quality. The second phase involved converting the 1-D ECG signals into 2-D spectrogram images and capturing temporal and frequency information. Feature extraction was performed in the third phase using the ensemble DL technique, combining VGG16 and LSTM models to obtain intermediate feature representations. The extracted ensemble features were then utilized as input for a boosting machine learning classifier, enabling the system to recognize and identify individuals based on their cardiac biometric data. Extensive experiments were conducted on a selected dataset to evaluate the system’s performance. On average, our proposed approach achieved an accuracy of 0.98%, a sensitivity of 0.98%, a specificity of 0.96%, and an AUC of 0.95. In comparison to state-of-the-art biometric authentication systems, our developed approach demonstrated superior performance in recognizing humans based on their cardiac biometric information. The combination of spatial features from the VGG16 model and temporal patterns from the LSTM model allowed us to optimize the ensemble features, leading to enhanced accuracy and robustness in the identification process. The proposed cardiac biometrics system holds significant potential for various real-world applications, particularly in domains requiring secure authentication. Its utilization in healthcare, finance, access control, and other sensitive areas could provide reliable and efficient human identification. Future research can focus on further refining the system, exploring additional deep learning architectures, and testing the approach on larger and more diverse datasets to strengthen its practicality and generalizability.