Personalized PPG Normalization Based on Subject Heartbeat in Resting State Condition

Abstract

Physiological responses are currently widely used to recognize the affective state of subjects in real-life scenarios. However, these data are intrinsically subject-dependent, making machine learning techniques for data classification not easily applicable due to inter-subject variability. In this work, the reduction of inter-subject heterogeneity was considered in the case of Photoplethysmography (PPG), which was successfully used to detect stress and evaluate experienced cognitive load. To face the inter-subject heterogeneity, a novel personalized PPG normalization is herein proposed. A subject-normalized discrete domain where the PPG signals are properly re-scaled is introduced, considering the subject’s heartbeat frequency in resting state conditions. The effectiveness of the proposed normalization was evaluated in comparison to other normalization procedures in a binary classification task, where cognitive load and relaxed state were considered. The results obtained on two different datasets available in the literature confirmed that applying the proposed normalization strategy permitted increasing the classification performance.

1. Introduction

In recent years, sensor technology has significantly improved and wearable devices have become increasingly popular, allowing easily registering subjects’ physiological responses during their daily activities [1,2]. Physiological signals are successfully used to measure arousal [3,4,5]. Arousal is an uncontrolled human reaction, related to attention and cognitive alertness, activated by stimuli that require high psycho-physical engagement, and thus activated in particular during cognitive tasks and stressful conditions. Although it has been proven that, sometimes, stress can have a positive effect on a person by improving his/her alertness state or his/her ability to react [6], it has also been proven that a high and continuous level of stress or cognitive load can affect the physical and mental well-being of the subject. Illnesses such as depression, anxiety, and sleep disorders are, indeed, often due to excessive stress or workload [7].

In view of its importance, the automatic recognition of stress and excessive cognitive load has recently become an object of study, even in different application areas. For instance, systems able to recognize emotion and, above all, stress experienced by subjects can be used in working or academic environments [8] in order to monitor and identify the emotional state of employees or students. In this regard, it has been proven that a high level of stress or cognitive load due to excessive workload can increase the level of fatigue, decrease the subject’s working capability, and consequently, bring physical and mental illness, which can lead to workplace absence [9]. Similarly, automatic stress recognition systems can be used in the context of vehicle driving for the detection of excessive mental fatigue states, which can reduce a person’s driving skills [10]. Finally, algorithms of stress detection can also be used in recreational areas for the development of systems able to modify their parameters based on the user’s emotional state. Concrete examples concern music-retrieval systems able to interact with a user to suggest a music playlist using both external inputs and his/her physiological signals [11] or video games in which some internal game parameters, such as the difficulty, are set based on the player’s emotions and stress level [12]. Moreover, systems able to recognize the subject’s emotional state can also be involved in the medical area. For instance, emotion recognition systems can be used to monitor the health state of convalescent patients [6] or to help elderly subjects during their daily activities [13,14]. In all of these contexts, the development of systems able to recognize, interpret, and simulate human affect can be seen as a necessary step to make technologies user friendly and able to interact actively with people.

Several works in the literature have focused their attention on the analysis of physiological signals as an honest indicator of humans’ emotion and mood [3,4]. In particular, the heart rate of a person is positively correlated with the perceived level of arousal and stress [15], thus resulting in being promising in the recognition of this specific emotional state [16]. Several stressors have been adopted in the literature to elicit arousal and mental stress [17]. Among them, mental arithmetic calculus or the Stroop test has been applied as a stressor in numerous experiments to evoke a high level of cognitive load in the participants [18]. In many of these studies, the use of physiological signals related to heartbeat has allowed positive performance both in statistical [19,20] and classification analysis [21,22,23].

Despite the progress in sensor technology and the relative simplicity of acquiring physiological signals from the human body, there are still some critical issues that must be addressed to fully exploit the potential that the analysis of physiological signals can offer. Although it is currently easier to acquire data, which can overcome the low cardinality of datasets, the application of machine learning techniques is still limited by inter-subject heterogeneity. Even in the same resting condition, without external stimuli, physiological signals appear to be significantly subject-dependent.

In this work, the reduction of inter-subject heterogeneity was faced in the case of the heartbeat, mainly detected through Photoplethysmography (PPG) [24]. The PPG signal is one of the most-used signals to measure arousal [5,25] and, consequently, to detect stress and evaluate the experienced cognitive load [20,26]. The PPG signal of each subject appears different, both in terms of amplitude and beat frequency. Regarding the amplitude, differences can be due to the subjects’ skin characteristics or to different sensor adherence during the acquisition phase. Concerning the diversity in terms of heartbeat, according to the American Heart Association, the heartbeat frequency of a resting adult can vary in the range between 60 and 100 beats per minute, and it depends on many different factors, both personal (such as age, sex, ethnicity, sports ability, diet, illnesses, prescribed medications, etc.) and environmental (humidity, temperature, etc.) [27]. Normalization procedures based on data rescaling are often used [28] to overcome amplitude variability within subjects. The main strategies adopted involve rescaling to the range $[0,1]$ [29], normalizing by dividing by the maximum value of the signal [30], and applying the Z-score [31]. None of these methods, however, take into account the effective differences in the subjects’ heartbeat, which are not only related to amplitude, but also to frequency.

The aim of this work was to solve this inter-subject variability, proposing a novel personalized PPG normalization based on the heartbeat of the subject in a resting state condition.

To validate the normalization procedure presented here, PPG data belonging to two different datasets were considered: the Cognitive Load and Affective Walkability in Different Age Subjects(CLAWDAS) dataset, partially introduced in [32,33,34], and the Cognitive Load, Affect and Stress recognition (CLAS) dataset, available in [35].

The proposed normalization was analyzed considering a binary classification task to discriminate cognitive load in the relaxed state and compared with normalization strategies adopted in the literature.

The paper is organized as follows. In Section 3, the two considered datasets are described, while the preliminary signal preprocessing strategies applied to each of them are reported in Section 4. The novel personalized PPG normalization strategy, based on the subject’s resting state heartbeat, is presented in Section 5. Extracted features, classification strategies, and the adopted cross-validation approach are reported in Section 6. The comparison of the classification performances on the two datasets, obtained with different normalization strategies, is then reported and analyzed in Section 7. Finally, the conclusions are drawn in the last section.

2. State-of-the-Art

In the last decade, the number of publications and citations related to heartbeat signals collected by photopletysmography has widely increased [36]. In particular, the growing popularity of cheaper and non-invasive wearable devices has allowed extending the analysis of such signals from health applications to other areas of interest such as entertainment [25,37], driving monitoring [38,39], or emotional analysis [40,41]. In many of these applications, the pipeline used to pre-process the PPG signals is similar and includes noise and motion artifact removal, as well as signal de-trending [42]. In [43], the PPG signals were pre-processed using a two-step procedure: first, each signal was filtered using a three-order low-pass Bessel filter and a notch filter to remove basic noise and baseline shift, then a direct comparison between PPG and ECG signals was performed to detect and eliminate the motion artifacts. Similarly, in [44], a bandpass filter was used to remove power line interference and motion artifacts and de-trend the signals, while a visual analysis was performed to remove the high peaks of noise. Other PPG denoising strategies, including methods based on empirical mode decomposition or wavelet decomposition, were summarized in [45]. In several literature works, the denoising and de-trending steps were followed by an amplitude normalization step performed to take into account subjects’ heterogeneity. The amplitude of the PPG signals varies from person to person, and it is both influenced by subjective and environmental factors [45,46].

Many strategies of data rescaling have been proposed in the literature to normalize the signals of each person in a similar range of values [47]. A min–max normalization was applied to the filtered signal of each participant to rescale the heart rate data in the range $[0,1]$ in [29,48,49]. Similarly, in [5], the amplitude of each subject’s PPG signal was rescaled in the range $[0,1000]$ using a min–max normalization strategy followed by the multiplication by a constant factor $\alpha =1000$. Another normalization strategy usually applied in the literature is the Z-normalization or Z-score normalization. In this method, the amplitude of each subject’s signal is standardized using the formula $X_n=(X_i-\mu )/\sigma$, where $\mu$ and $\sigma$ are, respectively, the mean and variance of the analyzed PPG signal. This method was applied in [50] as the last step of the pre-processing phase, while in [51], it was applied as an intermediate operation between the denoising step and the motion artifact removal step. Finally, in [19,52], an amplitude normalization based on the subject’s baseline was considered using the formula $(X_i-{\mu }_b)/max|X_i-{\mu }_b|$, where $X_i$ is the PPG signal, ${\mu }_b$ is the mean baseline value, and $max|X_i-{\mu }_b|$ is the maximum range of each subject.

While the amplitude normalization is usually considered in the analysis of PPG signals, the definition of a normalization strategy that considers the differences in the subjects’ heartbeat from a frequency point of view is still an open area of research. In [5], the PPG signals of each subject were segmented into single-peak frames, and each frame was used as a separate signal in the analysis. This method allows solving the issue of subject’s heartbeat heterogeneity, but restricts the features involved in the analysis to only peak morphological characteristics. In the following sections, our novel personalized PPG normalization based on the heartbeat of the subject in a resting state condition is introduced and described. In particular, a cognitive load classification task is considered in the analysis. We selected this task because it is widely analyzed in the literature, with positive results in terms of accuracy, making the validation of our proposal easier.

3. Dataset Description

Several datasets exist in the literature that acquire multimodal physiological data in the field of emotion recognition, among them DEAP [53], MAHNOB-HCI [54], EMDB [55], AMIGOS [56], ASCERTAIN [57], CASE [58], CLAS [35], and CLAWDAS [32,33]. Three main criteria have been applied for the selection of the two datasets, CLAS and CLAWDAS, considered here: the presence of cognitive load tasks, the use of wearable devices for physiological signals’ acquisition, and the presence of a baseline in the resting state condition. Both selected datasets fit these criteria, thus resulting in being suitable for the analysis proposed. In particular, CLAWDAS was selected as the proprietary dataset, while CLAS was chosen for the high number of participants involved, as well as for the use of the same wearable devices adopted in CLAWDAS. Below, the two datasets are described in detail.

In the CLAS dataset, the physiological signals of 60 healthy volunteers (mostly students between 20 and 27 years old, 17 women) were acquired while they were performing interactive or perceptive tasks. In particular, the interactive tasks were introduced to evaluate the level of concentration and the cognitive capacity of different individuals by solving Math Problems, Logic Problems, and Stroop Tests. In the perceptive tasks, different emotions were elicited in the participants by images and video selected from the DEAP dataset [53]. During the whole experiment, three types of physiological signals were simultaneously recorded by means of Shimmer sensors [34] Electrocardiography (ECG), Plethysmography (PPG), and Electrodermal Activity (EDA). The signals were acquired with a sampling rate of 256 Hz and a resolution of 16 bits per sample. In addiction, for each subject, 3D accelerometer data and metadata were also collected.

For the purpose of this work, only PPG signals collected during the interactive tasks were considered. For each participant, this phase of the experiment was characterized by the following steps:

• 1 min of Baseline (BL) in the resting state condition;
• 3 min of Math Problems in which the participant solves different simple Mathematical Problems (MPs) in a limited interval of time;
• 30 s of the Neutral State (NS), in which neutral audio-visual stimuli are displayed;
• 3 min of Stroop Tests (STs), where the user is expected to correctly match the color of the text with the meaning of the word, having a strict time constraint for each assignment;
• 30 s of the Neutral State (NS), in which neutral audio-visual stimuli are displayed;
• 5 min of Logic Problems (LPs) consisting of several simple logical problems often used during IQ tests;
• 30 s of the Neutral State (NS), in which neutral audio-visual stimuli are displayed.

Moreover, the 3 Neutral State repetitions of 30 s for each subject in the picture test of the perceptive session were also considered. For further details of this dataset, please refer to [35].

CLAWDAS is a dataset collected in a controlled laboratory environment at the Research Center for Advanced Science and Technology (RCAST) at The University of Tokyo. The experiments performed at RCAST were focused on finding differences in physiological responses related to different ages of the subjects, involved in several tasks, from cognitive to listening and walking ones.

The experiments involved two different groups of subjects: a population of 16 Japanese young adults with average age = 24.7 years old (4 women) and a population of 20 Japanese elderly people with average age = 65.15 years old (10 women). During the whole experiment, the heartbeat of each participant was collected though Photopletysmography (PPG) using the Shimmer3 GSR+ Unit [34] with a sampling frequency of 128 Hz. In addition to PPG, the Electrodermal Activity (EDA) of each subject was acquired using the same sensor. The Shimmer3 GSR+ Units are non-invasive and completely painless sensors that could be easily worn by the participants, as shown in Figure 1.

CLAWDAS data acquired during cognitive and listening tasks were considered here. The experimental protocol was composed of the following steps:

• 3 min of questionnaires to collect the personal details and the current emotional state of each participant using STAI Questionnaires;
• 1 min of Baseline (BL) in the resting state condition;
• 6 min of Reading (R) and Comprehension (C) tasks composed by two repetitions (trials) of 2 min of R followed by 1 min of self-assessment and C questions;
• 1 min of Baseline (BL) in the resting state condition;
• A 15 min sequence composed of six repetitions of the following two tasks:
  • 2 min of Audio Listening (AL); in this task, relaxation was induced by natural sounds (Figure 2, right);
  • 30 s of cognitive load, induced by mental Math Calculations (MCs) that involve sums, subtractions, and multiplications (Figure 2, left).
  Each repetition has a different audio track and Math Calculation.
• 1 min of Baseline (BL) in the resting state condition

The experimental protocol was reviewed and approved by the Research Ethics Committee at The University of Tokyo, Japan (Nos. 19-283 and 19-376). The CLAWDAS dataset was partitioned into two distinct subsets, according to the age of the participants: CLAWDAS Young, which included all the signals acquired from young adults, and CLAWDAS Elderly, which grouped all the signals collected from the elderly. In the following analysis, the two groups were considered separately.

Table 1. Number of instances for each task in the CLAS dataset (first 5 columns) and in the CLAWDAS dataset (last 5 columns), distinguished into CLAWDAS Young and CLAWDAS Elderly. BL = Baseline, MP = Math Problem, ST = Stroop Test, LP = Logic Problem, NS = Neutral State, MC = Math Calculation, R = Reading, C = Comprehension.

	Num Subj.	BL	MP	ST	LP	NS	BL	MC	AL	R	C
CLAS	60	60	60	60	60	360	-	-	-	-	-
CLAWDAS Young	16	-	-	-	-	-	46	96	96	32	32
CLAWDAS Elderly	20	-	-	-	-	-	60	120	120	40	40

reports the number of instances for each task in the CLAS and CLAWDAS datasets, keeping distinct the two subsets of CLAWDAS related to subjects’ age (Young and Elderly).

4. Signal Preprocessing: Denoising and Amplitude Normalization

In this section, the preprocessing operations applied to raw PPG data are detailed and differentiated for CLAS and CLAWDAS, respectively.

4.1. Denoising Strategies

The raw PPG signals are usually corrupted by noise and motion artifacts, which can undermine their interpretation and use [59]. In the CLAS dataset, the signals had already been preprocessed by the authors during the acquisition phase [35], and thus, no further denoising procedure was applied.

Concerning the CLAWDAS, the PPG raw signals of each subject were preprocessed by a multiresolution wavelet denoising strategy, as suggested by [59,60]. The signal was divided into frequency sub-bands using the Stationary Wavelet Transform (SWT) [61] with the Fejer–Korovkin mother wavelet [62] and four levels of decomposition. A soft thresholding was applied to the detail coefficients of each sub-band. The universal threshold calculated by the formula $T_k=\sqrt{2log\left(N_j\right)}$ was adopted, where $N_j$ is the length of the j-th wavelet coefficient and k is the sub-band [63]. The SWT was implemented with the à trous algorithm [64]. A preliminary operation of replicate padding was applied to the signal in order to obtain a length divisible by $2^{level}$ [61], with level$=4$.

4.2. Amplitude Normalization

In order to normalize the signals with respect to the amplitude, a Z-score operation, defined by the formula $Z=(x-\mu )/\sigma$, was applied to the PPG recordings after the denoising procedure.

In CLAWDAS, the amplitude normalization, (AmpN), as well as the denoising preprocessing were applied to the signal of each subject, before splitting it into the different experimental trials.

A similar procedure was also applied to the CLAS dataset signals. However, in this case, the authors already split the data into single trials, according to their experimental protocol (see Section 3), with no preliminary amplitude normalization. Thus, in order to apply a similar procedure to both datasets, the segmented trials of each subject were concatenated to re-build the original acquired signal. Then, the Z-score amplitude normalization was applied to each subject signal. Finally, the amplitude normalized signals were split back into the trials, related to single tasks, using the markers properly defined during the previous phase of concatenation.

5. Personalized PPG Normalization Based on Subject Resting State Heartbeat

The American Heart Association has underlined that the heart rate frequency of an adult in a resting state can vary in the range between 60 and 100 beats per minute. This inter-subject variability depends on many different factors, both personal and environmental. In the case of the CLAS and CLAWDAS datasets, the subjects’ heartbeat range of the baseline recordings belongs to the one reported by the literature, as depicted in Figure 3, where the average heartbeat distribution for each dataset is plotted.

In order to remove this inter-subject variability, the core idea of our normalization procedure is to map PPG signals, defined in the Discrete Time Domain (DTD), into a new Subject Normalized discrete Domain (SND), applying a mapping procedure based on the resting state heartbeat frequency. In this SND, all the subjects have the same resting state heart frequency. For each subject, a subject-based resampling frequency can be calculated, so that for all the subjects, the heart frequency of the resting state in the SND is equal, despite the original subject-peculiar frequency in the Continuous Time Domain (CTD).

Then, the PPG data acquired during all the experimental tasks were also mapped into this new domain, applying the calculated subject-based resampling frequency and obtaining subject normalized PPG signals that can be considered for population-based analysis.

Defining $f_c\left[\frac{\mathrm{sample}}{\mathrm{second}}\right]$ as the sampling frequency of the PPG signal, given by the acquisition device, and $f_b\left[\frac{\mathrm{beat}}{\mathrm{second}}\right]$ as the heartbeat frequency in the resting state condition in the CTD, the corresponding normalized heartbeat frequency $f_{Nb}\left[\frac{\mathrm{beat}}{\mathrm{sample}}\right]$ in the DTD is:

$${\mathit{f}}_{\mathit{Nb}}=\frac{f_b\frac{\mathrm{beat}}{\sec \mathrm{ond}}}{f_c\frac{\mathrm{sample}}{\sec \mathrm{ond}}}$$

(1)

We now define the subject normalized heartbeat frequency of the resting state in the SND as $f_{SNb}\left[\frac{\mathrm{beat}}{\mathrm{SNsample}}\right]$, where SNsample stands for Subject Normalized sample, which is the independent variable of the SND.

The subject-based resampling frequency that permits mapping the DTD PPG signal into the SND one is defined as $f_{SNc}\left[\frac{\mathrm{SNsample}}{\mathrm{sample}}\right]$ and can be calculated as follows:

$${\mathit{f}}_{\mathit{SNc}}=\frac{f_{Nb}\frac{\mathrm{beat}}{\mathrm{sample}}}{f_{SNb}\frac{\mathrm{beat}}{\mathrm{SNsample}}}$$

(2)

As our goal was to obtain a domain where the inter-subject variability is discounted, the $f_{SNb}$ for all subjects’ baseline should be the same. Once this constant value is chosen, the $f_{SNc}$ resampling frequency for each subject can be calculated from Equation (2). Then, all the PPG data of the same subject can be resampled accordingly and mapped into the SND, making the SN data reliable for population-based analysis. In

Table 2. Correspondences between the three domains.

Domain	Continuous Time	Discrete Time	Subject Normalized
Acronym	CTD	DTD	SND
Heartbeat	$f$ [$\frac{\mathrm{beat}}{\mathrm{second}}$]	$f_N$ [$\frac{\mathrm{beat}}{\mathrm{sample}}$]	$f_{SN}$ [$\frac{\mathrm{beat}}{\mathrm{SNsample}}$]
Resting state Heartbeat	$f_b$ [$\frac{\mathrm{beat}}{\mathrm{second}}$]	$f_{Nb}$ [$\frac{\mathrm{beat}}{\mathrm{sample}}$]	$f_{SNb}$ [$\frac{\mathrm{beat}}{\mathrm{SNsample}}$]
Sampling frequency	–	$f_c$ [$\frac{\mathrm{sample}}{\mathrm{second}}$]	$f_{SNc}$ [$\frac{\mathrm{SNsample}}{\mathrm{sample}}$]

, the notation introduced is summarized for the sake of clarity. Taking this into account, Equations (1) and (2) can be rewritten as follows:

$${\mathit{f}}_{\mathit{SNc}}=\frac{f_b}{f_c}\ast \frac{1}{f_{SNb}}$$

(3)

The resting state heartbeat in the SND ($f_{SNb}$) can be arbitrarily chosen, only paying attention to possible aliasing effects. In our calculations, we set $f_{SNb}=\frac{1}{128}\left[\frac{\mathrm{beat}}{\mathrm{SNsample}}\right]$, which corresponds to one beat on 128 SNsamples in the SND.

To analyze the effect of our normalization proposal, let us consider some numerical examples. In the case of a sampling frequency of $f_c=128$ Hz, as in the case of the CLAWDAS dataset, we can observe from Equation (3) that, in the case of a subject with a baseline heartbeat frequency of $60\frac{\mathrm{beat}}{\mathrm{minute}}$, corresponding to $1\frac{\mathrm{beat}}{\sec \mathrm{ond}}$, the subject-based resampling frequency is $f_{SNc}=1\frac{\mathrm{SNsample}}{\mathrm{sample}}$, meaning that there are no differences between the signal in the DTD and in the SND. Note that $60\frac{\mathrm{beat}}{\mathrm{minute}}$ is generally considered as the minimum value for normal people. For heartbeat frequencies higher than $60\frac{\mathrm{beat}}{\mathrm{minute}}$, the mapping from the DTD to the SND implies an over-sampling, while for lower frequencies, the consequent under-sampling does not introduce aliasing, as 128 samples are guaranteed between two consecutive peaks.

In Figure 4, PPG signals corresponding to the first baseline in the CLAWDAS Elderly dataset of Subjects 11 and 18, respectively, are considered. In the first row, the signals in the DTD are reported, showing the difference between the two subjects’ heartbeat frequencies: for Subject 11, $f_{Nb}=\frac{1}{78}\frac{\mathrm{beat}}{\mathrm{sample}}$, corresponding to $84\frac{\mathrm{beat}}{\mathrm{minute}}$, while for Subject 18, $f_{Nb}=\frac{1}{94}\frac{\mathrm{beat}}{\mathrm{sample}}$, corresponding to about $82\frac{\mathrm{beat}}{\mathrm{minute}}$. In the second row, the same two signals resampled in the SND are shown. Note that we assumed in defining our procedure that the heartbeat during a resting state is a stationary and periodic signal; however, this is not the case in real life, justifying not having $f_{Nb}$ strictly equal to $\frac{1}{128}\frac{\mathrm{beat}}{\mathrm{SNsample}}$ for both subjects in the SND.

In the case of multiple baseline signals, the heartbeat frequency is evaluated as the average of the heartbeat frequency of all of them. This procedure was applied, for example, during the normalization of the CLAWDAS signals. In this case, in fact, three different baseline signals were acquired from each subject.

The pseudocode that allows applying the personalized normalization to the PPG signals of a generic subject “s” is reported in Algorithm 1. The first two rows initialize the counter variable “i” to 0 and set the SND resting state heartbeat frequency (FSNB) to the value arbitrarily chosen. In Rows [3–11], the resting state heartbeat of subject “s” is computed. Using this value, the subject-based resampling frequency “fsnc” is evaluated in Row 12. Finally, in Rows [13–15], all the signals collected from subject “s” are resampled using the new sampling frequency “fsnc”.

Algorithm 1: Pseudocode of the personalized PPG normalization applied to the signals of subject “s”.

1: $i\leftarrow 0$   2: $FSNB\leftarrow$resting state heartbeat frequency in subject normalized domain   3: foreach baseline signal “bl” collected from subject “s”do   4:     $fbtrail\left[i\right]\leftarrow$ compute the heartbeat frequency of “bl” as beats/second   5:     $i\leftarrow i+1$   6: end for   7: ifsubject “s” has more than one “bl” signalthen   8:     $fb\leftarrow$ mean of the value in fbtrial   9: else 10:     $fb\leftarrow$ the only element in fbtrail 11: end if 12: $fsnc\leftarrow$multiply fb for FSNB 13: foreach signal “t” acquired from subject “s”do 14:     resample “t” using the new subject normalized sampling frequency fsnc 15: end for

6. Classification Setting

A binary classification task is herein proposed on CLAS, CLAWDAS Young, and CLAWDAS Elderly, to evaluate the performance of the personalized PPG normalization presented in this work. In particular, a cognitive load recognition task was considered in the analysis. We selected this task of classification because it is widely analyzed in the physiological signals’ literature with also positive performance [21,22]. Two classes were thus considered: the class corresponding to signals collected during high cognitive load tasks: High CL, and the class related to low cognitive load tasks: Low CL. The tasks used as representative of each class changed according to the dataset considered.

In particular, according to [35], in the CLAS dataset, the PPG data collected during the three cognitive tasks (MP, ST, and LP) were labeled as High CL, while the NS data were labeled as Low CL. In order to make the two classes balanced, MP, ST, and LP signals were split into two non-overlapped segments of equal length, reaching a cardinality of 360 instances for both classes.

In CLAWDAS Young and CLAWDAS Elderly, the data collected during the MC task were selected for the High CL class, while the data collected during the AL task were chosen as representatives of the Low CL class. Thereby, the two classes were equally balanced, with 96 instances each for CLAWDAS Young and 120 instances each for CLAWDAS Elderly. In this study, the Math Calculation task was preferred over the Reading and Comprehension tasks as the High CL class because it is able to elicit high mental stress according to what has been defined in the literature [17].

In all the performed analysis, seven handcrafted features were extracted as characteristics useful to describe the PPG signals:

• Minima, Maxima, Mean, and Standard Deviation of the signal;
• Peak Rate, which represents the mean number of peaks;
• Inter-Beat Interval (IBI), which represents the mean distance between two peaks in a row;
• Root Mean Square of Successive Distance (RMSSD), which represents the variance of the distance between two consecutive peaks [65].

The last three features were evaluated in the discrete domain and reported with respect to samples. For the sake of clarity, it is recalled that the meaning of samples changes according to the type of normalization strategy adopted. In particular, samples refer to subject normalized samples when the features are evaluated on signals with personalized normalization based on resting state heartbeat, while this term refers to discrete time samples in all the other cases. All the features so evaluated were also standardized by applying the z-score before being used as the input to the different classifiers.

Using the the seven features introduced above, three binary classification experiments were performed for each of the two datasets considered, comparing the following three normalization strategies:

• AmpN: Amplitude Normalization, as described in Section 4.2;
• SubjFeatN: amplitude normalization followed by a Subject Feature Normalization. This feature normalization was performed with respect to the subject baseline on the Peak Rate, IBI, and RMSSD features as follows:

  $$featNor{m}_i=\frac{featur{e}_i-\overline{featureB{L}_i}}{\overline{featureB{L}_i}}$$

  (4)

  where $i\in PeakRate,IBI,RMSSD$; $featur{e}_i$ represents the feature value before the normalization; $featNor{m}_i$ is the new normalized value; and $\overline{featureB{L}_i}$ is the mean value of the $i-th$ feature evaluated on the subject resting state;
• PersFreqN: amplitude normalization followed by the Personalized Normalization based on the resting state heartbeat, described in Section 5.

The pre-processing and feature extraction operations applied to each signal and the three considered normalization strategies are summarized in Figure 5.

For each analysis, four different classification models were tested: a Classification and Regression Tree (CART) with Gini’s diversity index as the criterion of splitting and 100 as the max number of decision splits, and three Support Vector Machines (SVMs) with different kernels: Linear (SVM Linear), Gaussian (SVM Gaussian), and polynomial Cubic (SVM Cubic). In particular, for the Gaussian kernel SVM, the kernel scale was set to 3.3 in order to consider a medium Gaussian SVM.

A Leave One Subject Out (LOSO) cross-validation [66] was applied to evaluate the performance of the trained classifiers. At each iteration, the data used to train the classifier consisted of the signals collected from all the subjects, except one, whose instances were instead used to test the performance of the model. An overall confusion matrix was finally generated, joining the single confusion matrices resulting from each iteration. From this confusion matrix, several well-known evaluation metrics were extracted. In particular, we selected accuracy to evaluate the general performance of the classifier and the single class F1-score [67] to assess, instead, the goodness of the classification model in recognizing the single classes. The classification settings described above are summarized in

Table 3. Summary of the classification settings.

Types of Normalization Considered	AmpN, SubjFeatN, PersFreqN
Features Used	Maximum, Minimum, Mean, Standard Deviation, Peak Rate, IBI, RMSSD
Classifier Involved	SVM Linear, SVM Cubic, SVM Gauss, and CART
Performance Evaluation Method	LOSO
Evaluation Metrics	Accuracy, single-class F1-score
Dataset	CL High Class		CL Low Class
	Task	Num. of signals	Task	Num. of signals
CLAS	Math, Stroop, and Logic Test	360	Neutral State	360
CLAWDAS Young	Math Calculation	96	Audio Listening	96
CLAWDAS Elderly	Math Calculation	120	Audio Listening	120

.

7. Results and Discussion

The three different types of normalization, AmpN, SubjFeatN, and PersFreqN, are compared in this section, considering the high class and low class cognitive load binary classification task for the two chosen datasets. The performance corresponding to each of the classification settings described in Table 3 are reported in

Table 4. Performance comparison on the CLAS dataset, varying the normalization strategies (columns) and classification models (rows). Two performance metrics are evaluated using a LOSO validation strategy: Accuracy (Acc) and single-class F1-score. The best performances reached for each type of normalization are underlined, while the highest accuracy value of all is highlighted in bold.

	AmpN			SubjFeatN			PersFreqN
Classifier		High CL	Low CL		High CL	Low CL		High CL	Low CL
	Acc	F1-Score	F1-Score	Acc	F1-Score	F1-Score	Acc	F1-Score	F1-Score
SVM Linear	66%	0.65	0.67	67%	0.68	0.67	76%	0.77	0.76
SVM Cubic	74%	0.74	0.73	73%	0.74	0.72	81%	0.81	0.81
SVM Gauss	72%	0.72	0.71	73%	0.72	0.73	78%	0.78	0.77
CART	72%	0.71	0.73	66%	0.65	0.66	73%	0.72	0.73

,

Table 5. Performance comparison on the CLAWDAS Young dataset, varying the normalization strategies (columns) and classification models (rows). Two performance metrics are evaluated using a LOSO validation strategy: Accuracy (Acc) and single-class F1-score. The best performances reached for each type of normalization are underlined, while the the highest accuracy value of all is highlighted in bold.

	AmpN			SubjFeatN			PersFreqN
Classifier		High CL	Low CL		High CL	Low CL		High CL	Low CL
	Acc	F1-Score	F1-Score	Acc	F1-Score	F1-Score	Acc	F1-Score	F1-Score
SVM Linear	66%	0.62	0.69	68%	0.63	0.72	79%	0.77	0.80
SVM Cubic	66%	0.66	0.66	68%	0.67	0.68	72%	0.72	0.72
SVM Gauss	63%	0.59	0.66	66%	0.61	0.70	76%	0.75	0.76
CART	56%	0.59	0.51	68%	0.67	0.69	64%	0.63	0.64

and

Table 6. Performance comparison on CLAWDAS Elderly, varying the normalization strategies (columns) and classification models (rows). Two performance metrics are evaluated using a LOSO validation strategy: Accuracy (Acc) and single-class F1-score. The best performances reached for each type of normalization are underlined, while the the highest accuracy value of all is highlighted in bold.

	AmpN			SubjFeatN			PersFreqN
Classifier		High CL	Low CL		High CL	Low CL		High CL	Low CL
	Acc	F1-Score	F1-Score	Acc	F1-Score	F1-Score	Acc	F1-Score	F1-Score
SVM Linear	59%	0.58	0.60	69%	0.65	0.72	80%	0.80	0.81
SVM Cubic	59%	0.55	0.62	64%	0.61	0.66	72%	0.71	0.73
SVM Gauss	63%	0.61	0.64	75%	0.74	0.76	78%	0.78	0.78
CART	54%	0.51	0.57	68%	0.68	0.67	75%	0.75	0.74

, for CLAS, CLAWDAS Young, and CLAWDAS Elderly, respectively. In particular, the results of the different normalization strategies are reported in terms of accuracy and single-class F1-score generated using the LOSO cross-validation approach.

Note that, despite the considered classifier, SVM Linear, SVM Cubic, SVM Gauss, and CART, in all the experiments carried out, the normalization strategy herein proposed, PersFreqN, outperformed the other two normalization procedures, for all the datasets. This observation is further supported by a visual comparison of the performance of the classifiers reported in terms of accuracy in the bar plot of Figure 6, varying the normalization strategy and the involved dataset. In particular, PersFreqN achieved the best accuracy for all the dataset and type of classifier, with the sole exception of the CART classifier for the CLAWDAS Young dataset, where the best performance was achieved by the SubjFeatN strategy. On the other hand, the AmpN normalization was in general the worst approach.

Comparing the datasets, the best performance was observed on the CLAS dataset. In this case, the proposed PersFreqN allowed reaching an accuracy of 81% adopting the SVM classifier with the polynomial cubic kernel. This result significantly outperformed the accuracy of 74% reached using AmpN and the 73% obtained using SubjFeatN.

For the CLAWDAS datasets, the highest accuracies achieved were 79% for CLAWDAS Young and 80% for CLAWDAS Elderly, both obtained with the proposed PersFreqN and SVM with a linear kernel. These values show a significant improvement with respect to the other normalization strategies AmpN and SubjFeatN, which were always lower than 66%.

Another consideration regards the classifier that allowed reaching the best results. In general, from the three Table 4, Table 5 and Table 6, it emerged that the highest accuracy values were usually achieved by the SVM classifiers with the linear or cubic kernel, whereas the lowest ones were generally obtained by the CART classifier.

Finally, a last consideration should be made on the SubjFeatN normalization strategy. In general, this normalization produced a performance higher than that of AmpN, even if it appeared less effective compared to the proposed PersFreqN strategy. These results confirmed that a normalization strategy that takes into account not only amplitude normalization, but also subject’s characteristics should be adopted to remove inter-subject variability.

As a final remark, all the adopted classification settings were able to classify with comparable performance both classes, as indicated by the values of the single-class F1-score in all the tables. However, the introduction of the proposed PersFreqN seemed to produce even more balanced classification results.

8. Conclusions

While considering physiological data, signal normalization not only with respect to amplitude, but also with respect to personal characteristics is mandatory to perform subject-based analysis, especially if machine learning techniques should be applied. Personalized normalization on PPG data, both at the feature level, SubjFeatN, and with respect to heartbeat frequency, PersFreqN, introduced an increase in the classification performance, considering different classification models and datasets. In particular, the personalized PPG normalization based on subject heartbeat herein proposed, PersFreqN, outperformed the other strategies and permitted significantly reducing inter-subject heterogeneity. Moreover, the proposed normalization could also be useful for intra-subject analysis, especially when comparing the physiological responses of the same subject, on different days or even at different moments of the day: it is well known, in fact, that the physiological responses not only depend on external stimuli, but also on physical and internal conditions, which can significantly vary for the same subject with respect to time.