BiomacEMG: A Pareto-Optimized System for Assessing and Recognizing Hand Movement to Track Rehabilitation Progress

Abstract

One of the most difficult components of stroke therapy is regaining hand mobility. This research describes a preliminary approach to robot-assisted hand motion therapy. Our objectives were twofold: First, we used machine learning approaches to determine and describe hand motion patterns in healthy people. Surface electrodes were used to collect electromyographic (EMG) data from the forearm’s flexion and extension muscles. The time and frequency characteristics were used as parameters in machine learning algorithms to recognize seven hand gestures and track rehabilitation progress. Eight EMG sensors were used to capture each contraction of the arm muscles during one of the seven actions. Feature selection was performed using the Pareto front. Our system was able to reconstruct the kinematics of hand/finger movement and simulate the behaviour of every motion pattern. Analysis has revealed that gesture categories substantially overlap in the feature space. The correlation of the computed joint trajectories based on EMG and the monitored hand movement was 0.96 on average. Moreover, statistical research conducted on various machine learning setups revealed a 92% accuracy in measuring the precision of finger motion patterns.

1. Introduction

Human-computer interaction (HCI) bridges the communication gap between humans and computers [1]. Hand Gesture Recognition is one type of HCI that predicts the class and moment of execution of a given hand action (gesture) [2]. Surface electromyography (sEMG), which captures the electrical activity of muscles, is one possible input for these models [3,4]. On the contrary, electromyography (EMG) is a type of physiological signal recording that detects electrical signals from muscle fibres in response to gestures or movements [5]. The brain creates electrical impulses that are sent to muscle fibres through the spinal cord and nerve fibres to activate movements [6,7]. EMG-based research for medical assistance and smart gadgets, such as stroke assessment [8], analysis of neural impairments [9], heartbeat analysis [10], prosthetics [11], rehabilitation [12,13], physical training assessment [14,15], sports analytics [16], movement recognition and analysis [17], affective computing [18], human-machine interfaces  [19,20], text input for the disabled [21], bio-signal fusion [22], and general healthcare [23], has been active. To aid rehabilitation from a distance, interactive forms for telerehabilitation can be used to measure the development of patients’ range of motion (ROM) in real time using artificial intelligence algorithms, by manipulating the angles of action of limbs about a joint [24]. Latreche et al. [25] then explored an artificial intelligence approach built into the MediaPipe website to measure range of motion with a universal goniometer and a digital angle meter. Anton et al. suggested a WebRTC-based telerehabiliation system framework capable of managing video and audio streams based on network status and available bandwidth to ensure real-time communication performance [26].

The novelty of the study lies in the combination of these elements:

• The study developed a robot-assisted hand-motion therapy system.
• The system utilizes machine learning approaches to recognize and describe hand motion patterns in healthy people.
• The system also tracks rehabilitation progress.

The study used surface electrodes to collect electromyographic (EMG) data from the forearm flexion and extension muscles, which were used as input parameters to the machine learning algorithms to recognize seven hand gestures.

The study’s contributions include demonstrating the technical feasibility of measuring and recognizing finger movement patterns and the potential use of this approach for monitoring rehabilitation progress or other medical processes. The study also analysed nearly 10,500 individual measurements of the specified gestures, which were assigned to a specific movement class and used to develop a classification algorithm. The study findings could potentially lead to the development of more effective and personalized stroke therapy programs that improve hand mobility and function.

The remainder of the paper are as follows. Section 2 discusses the measurement and classification methodology. Section 3 presents and analyses the results. Section 4 provides a discussion. Section 5 outlines the conclusions.

2. Review of Related Approaches

The EMG-based gesture recognition process comprises classifying the unknown observed data into the most likely gesture that corresponds to the target movements [27]. Current research on EMG-based hand gesture categorization encounters challenges, such as poor classification accuracy, low generalization ability, a scarcity of training data, and a lack of robustness [28]. Recent research has focused on developing user-independent interfaces based on gesture recognition to diminish the time of calibration for novice users [29]. However, because EMG signals are erratic, signal analysis performance is generally poor [30,31]. Furthermore, due to the presence of noise and the need for significant processing power, it is very difficult to employ sEMG signals as user control signals [32]. For example, Alfaro et al. [29] offered a user-independent gesture identification system based on sensor fusion technology that combines the data from IMU and EMG sensors. The Myo Armband was employed to evaluate muscle activity and mobility of healthy participants. The data acquired from 22 people were used to classify the movements, and the Adaptive Least-Squares Support Vector Machine (SVM) model was 92.9% accurate. Similarly, Benalcázar et al. [33] provided a real-time approach to recognise five movements on the right hand. These are the exact actions that the Myo armband’s patented technology detects. A shallow artificial feed-forward neural network is used in the proposed system. The correlations among the EMG channels and the result of a bag of five components given to each EMG channel are sent into this network. Fajardo et al. [34] offered a mixed feature approach strategy for EMG classification that classifies signals acquired from a single channel device using handmade features generated from temporal-spectral analysis and deep features produced from a convolutional neural network (CNN). The approach requires only 100 impulses out of each gesture for training, significantly reducing the time necessary to train the system. Such techniques suggested combine hand-crafted characteristics from a time-spectral investigation with deep features to build the feature vector [35]. Subsequently, the EMG characteristic vector is often classified using a multilayer perceptron classifier (MLPC) [36]. In [37], EMG-based hand/finger motion classifiers were developed based on fixed electrode placement. Three channels of MG signals were monitored and each channel yielded six time-domain (TD) characteristics. A total of 18 attributes were used to create subject-specific classifiers for 10 gestures using an artificial neural network (ANN) and several machine learning classifiers. ANN had the highest mean accuracy and the lowest intersubject variability in accuracy, showing that subject-specific variance in EMG data had low influence on it. The research [38] proposed BoostEMD, a unique version of Huang’s Empirical Mode Decomposition (EMD) approach for computing higher-order intra-mode functions (IMFs) that identify higher frequency oscillations in EMG signals. The authors demonstrated the efficacy of denoising by classifying the EMG data before and after denoising, as well as analysing the characteristics of the recovered noise signal, similarly to [39]. Sahu et al. [40,41] proposed using the beta artificial bee colony (BetaABC) and binary BetaABC (BBABC) to pick important features in the recognition of the EMG pattern to improve the classification performance while reducing the complexity of the classifier, where the EMG signal was decomposed and the characteristics were extracted using the discrete wavelet transform (DWT).

Deep learning methods are a very popular use case for the classification of gestures using EMG signals [42,43]. However, deep learning algorithms are rarely used in the field of EMG-based gesture detection, as they take an unacceptable amount of effort from a single individual [44,45]. Although useful features may be acquired from the large volumes of data created by pooling the signals of several users, reducing the recording effort while improving gesture recognition, there are additional issues, such as bias [46]. Gopal et al. [32] compared machine learning and deep learning classifiers on generalization in general across different classes, and offered a systematic examination of the effect of variables in the time domain and pre-processing factors on the performance of the classification model. Ensemble and deep learning algorithms have exceeded other standard machine learning approaches. In [47], EMG data received from the forearm muscles was classified using Recurrent Neural Network (RNN) to improve the online classification of hand motions. This study contrasts and explains a Feed-Forward Neural Network (FFNN), an RNN, a Long Short-Term Memory network (LSTM), and a Gated Recurrent Unit (GRU). The FFNN, LSTM, RNN, and GRU models exhibited equivalent accuracy, 95% for the DualMyo dataset and 91% for the NinaPro DB5 dataset. Jo et al. [48] suggested Convolutional Recurrent Neural Network (CRNN) model to recognize hand motions in real time by combining LSTM for time-series information classification and CNN for feature extraction. Two grips, three hand signals, and one rest are recognized and categorised as hand gestures. For pre-processing, the Short Time Fourier Transform (STFT), Continuous-time Wavelet Transform (CWT), and the recently suggested Scale Average Wavelet Transform (SAWT) are utilized. The CRNN with SAWT and the overlapping window outperformed the other approaches. The recognition of lower-limb motions using the tunable Q-factor wavelet transform (TQWT) and the Kraskov entropy (KrEn) was presented in [49] with a similarly efficient result. Toro-Ossaba et al. [50] proposed to build a gesture classifier using an RNN model that incorporates LSTM units and dense layers, with the objective of reducing the number of EMG channels and total complexity of the model to increase scalability for embedded systems. The model needs only four EMG channels to identify five hand movements, resulting in a considerable decrease in the number of electrodes. The model was trained using a 20 s dataset of gesture EMG signals recorded with a custom EMG armband. During training and validation, the model obtained an accuracy of up to 97%, and an accuracy of 87% during real-time testing.

Li et al. [51] developed an approach to identify hand movements that uses an improved multichannel CNN (IMC-CNN) to extract properties of time and frequency domains from surface electromyography (sEMG). To recognize the 10 most frequently used hand gestures, the spectral features of sEMG signals, which serve as input to the IMC-CNN model, are utilised. The spectrogram characteristics of the sEMG signals from several channels are integrated into a complete enhanced spectrogram feature, which is sent into IMC-CNN to categorize hand motions. The recognition accuracy of the proposed model was 97.5%. In [52], the arm sEMG signal was acquired to infer four types of signals from nine static motions. To reduce redundant data in EMG signals, enhance recognition performance, and allow for real-time recognition, the action recognition system is built using the principal component analysis (PCA) method and the general regression neural network (GRNN). The precise action mode is determined by collecting information from human movements. The total recognition rate of the system was achieved by 95.1% after dimension reduction and model training. Shanmuganathan et al. [53] demonstrated that the use of recurrent CNN (R-CNN) in terms of EMG signals for hand gesture identification was demonstrated. The signal is captured using electrodes on the forearm, and the features are extracted using the wavelet packet transform. Using the wavelet power spectrum, the R-CNN model achieves 96.48% accuracy. The purpose of the alternative study [54] was to see if movements of the hand and wrist joints affected the recognition of the EMG pattern. In total, 11 hand and wrist movements were evaluated in able-bodied volunteers using 2 different gesture modalities. Movements of the hand and wrist joints had a substantial influence on the recognition of the EMG patterns. The findings provided a fresh perspective on the parameters that influence the recognition of EMG patterns. To evaluate a similar model, Zhang et al. [55] used a FFNN trained on segmented sEMG signals, which had an average identification rate of 98.7% using real sEMG data from 12 subjects. The system can recognize a gesture before it is completed due to the fast recognition time.

3. Materials and Methods

The research evaluated the technical possibilities of measuring and recognizing the movement of the experimenter’s hand, potentially monitoring the progress of rehabilitation or other medical process. At this stage, in order to check the precision, it was chosen to measure not only the movement of the whole hand or another large part of the body but more precise scenarios, that is, to examine the movement that is performed by the fingers. A total of 7 gestures were performed, namely: (a) raising the index finger, (b) raising the middle finger, (c) raising the ring finger, (d) raising the little finger, (e) thumbs up, (f) palm at rest with all fingers extended, and (g) victory sign displayed.

The 8 channel Mindrove armband (Mindrove, Budapest, HU) was used to capture the EMG data. The wearable equipment was placed on the forearm and used to measure different types of muscle groups in real time.

• The carpi radials of the muscle flexor are responsible for the flexion and radial deviation of the wrist.
• Musculus flexor pollicis longus, which is responsible for the flexion of the pollicis.
• Musculus flexor digitorum, which is responsible for flexion of the fingers.
• The carpi ulnaris muscle flexor, which is responsible for flexion and ulnar deviation of the wrist.
• Muscuclus extensor pollicis longus at brevis, which is responsible for the extension of pollicis.
• The carpi ulnaris muscle extensor is responsible for extension and ulnar deviation of the wrist.
• Musculus extensor digitorum, that is responsible for extension of the fingers.
• Musculus extensor carpi radialis, which is responsible for extension and radial deviation of the wrist.

According to accepted scientific practice, all gestures were recorded with EMG sensors pressed on the forearm to measure arm muscle activity (see Figure 1). The EMG time signal was recorded for up to 5 s while each participant performed a required gesture. A 5 s segment of the signal was analysed using the statistical estimates discussed in the previous section. An experimental study was conducted to determine which statistical estimates of the temporal signal can be used as motion-discriminating features and suitable for developing a classification algorithm. A proprietary software approach was used to process the captured data based on the Pareto optimized principles described below.

The study analysed nearly 10,500 individual measurements of the specified gestures (approximately 1500 for each class). All measurements are assigned to a specific movement class, which is numbered from M1 to M7, as explained above. The number and distribution of classes is shown in the figure below, where the measurement number values are plotted on the horizontal axis, and the motion class numbers are plotted on the vertical axis. The sample of data used is assumed to be balanced.

3.1. Features

Gesture and related muscle activity were recorded with 8 EMG sensors. Statistical estimates of the signal were calculated from the recorded time-varying signal, and features describing that signal were extracted:

• Standard deviation;
• Minimum;
• Maximum;
• Crossing the zero axis;
• Average change in amplitude;
• The first amplitude jump;
• Mean absolute value;
• Wave length;
• Wilson amplitude.

3.1.1. Standard Deviation of the EMG Signal

Standard deviation is a measure of absolute variability that shows how individual observations are grouped with respect to the mean. It is measured in the same units as the mean. If our data are equivalent to the normal distribution law, about 95% subjects have values between minus and plus of the mean, about 2 standard deviations. The other 5% are equally distributed outside these limits. As the sample size increases, the standard deviation does not tend to change as much. The standard deviation is calculated based on Equation (1).

$$\sigma =\sqrt{\frac{{\sum }_{i=1}^N{(x_i-\mu )}^2}{N}}$$

(1)

where $\sigma$ is the standard deviation of the EMG signal, $\mu$ is the average of the EMG signal, x is the value of the EMG signal at the i-th time instant, N is the number of measurements in the EMG signal. The standard deviation was calculated for the EMG signals recorded by all sensors.

3.1.2. EMG Signal Minimum Values

In search of the feature that best discriminates the gesture, it was chosen to include the minimum value operator (min), which is obtained by searching for the minimum value in the EMG signal according to Equation (2):

$$X_{min}=\underset{1\ge I\ge N}{min}\left\{x_i\right\}$$

(2)

3.1.3. EMG Signal Maximum Values

In search of the best discriminating feature of gesture, it was chosen to include the maximum value operator (max), which is obtained by searching for the minimum value in the EMG signal according to Equation (3):

$$X_{max}=\underset{1\ge I\ge N}{max}\left\{x_i\right\}$$

(3)

3.1.4. EMG Signal Zero Crossing

Zero crossing [7,56,57] are a convenient and fast way of estimating the frequency of a sampled sequence of data. A zero crossing is the point where the sign of a function inverts into opposite, as shown by the axis crossing (zero value) on the graph of the function. It is a signal evaluation indicator often employed in electronics, mathematics, acoustics, and image processing.

Zero crossing is calculated using Equations (4) and (5)

$$s(x,y)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\left(x\dot{y}\right)<0\hfill \\ 0\hfill & \mathrm{if}\left(x\dot{y}\right)>0\hfill \end{array}\right.$$

(4)

$$ZC\left(V\right)=\sum _{i=1}^{n-1}s(V_i,V_{i+1})$$

(5)

where $ZC$ is the zero crossing value, x and y are the EMG signal values.

3.1.5. The Mean Amplitude Change of the EMG Signal

The average amplitude is the average magnitude of all instantaneous values in the EMG time signal. The average amplitude is the ratio between the sum of the magnitudes of all values and the number of values in the EMG signal. For all alternating periodic signals, such as sine and cosine waves, the average amplitude of the entire signal is zero. The average amplitude of the signal of the first half is the same as the amplitude of the signal of the second half, but the amplitude is opposite. Therefore, the average amplitude of the entire signal is zero (the average amplitudes of both halves of the signal cancel). The average change in amplitude is calculated according to Equation (6).

$$i_{Avg}=\frac{1}{N}\sum _{i=1}^N{x}_i$$

(6)

3.1.6. Average Absolute Value of the EMG Signal

Mean absolute average error, also known as mean absolute average deviation, is a measure of the accuracy of a forecasting method, such as trend estimation, and is also used as a loss function. Usually, the accuracy is expressed by the ratio defined by Equation (7). The mean absolute value (MAV) is a method to determine and evaluate the level of muscle contraction.

$$MAV=\frac{1}{N}\sum _{i=1}^N\mid x_i\mid$$

(7)

3.1.7. EMG Signal Wavelength

The waveform length (WL) is intuitively the total length of the waveform in a segment. The resulting waveform length count values provide a measure of the amplitude, frequency, and duration of the waveform. The signal wavelength is calculated according to Equation (8).

$$WL=\sum _{i=1}^{N-1}\mid x_{i+1}-x_i\mid$$

(8)

3.1.8. Wilson Amplitude of the EMG Signal

The Wilson amplifier (WAMP) is the number of times the difference between the amplitudes of the sEMG signal of two adjacent segments exceeds a predetermined threshold to reduce the effects of noise. Wilson’s amplitude is related to the level of motor unit action potential (MUAP) and muscle contraction. A suitable value for the threshold parameter is usually chosen between 10 and 100 mV, which depends on the gain setting of the instrument. The Wilson amplitude is calculated using the Equation below.

$$WAMP=\sum _{n=1}^{N-1}f(\mid x_n-x_{n+1}\mid )$$

(9)

$$f\left(x\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}(x\ge y)\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(10)

3.1.9. Summary of EMG Signal Estimates

Each contraction of the arm muscles during one of the seven movements was recorded by 8 EMG sensors. Estimates of temporal signals were calculated from the recorded signals.

3.2. Reducing the Classification Space by Applying a Linear Data Transformation to PCA

PCA is one of the classical statistics methods. The main idea of this method is to perform a linear transformation and reduce the number of dimensions of the data by discarding a part of the components found after the transformation, which have the smallest variance. The PCA method is often referred to as the discrete K–L transformation because it reduces the number of data or dimensions that describe the features of the sample. In particular, the principal component analysis descriptor looks for the principal component P1 with the highest variance. This component crosses the central point of the data and is closest to all data points because the average distance from the component to the points is minimal. The second principal component P2 also crosses the central point of the data and is perpendicular to the first principal component. Each principal component is described by an eigenvector and an eigenvalue. The real vector indicates the direction of the principal component, and the real value describes how the data are distributed in that direction.

The PCA algorithm is performed in four steps:

• Ensure that the data have the same mean and variability.
• Find the covariance matrix C.
• Calculate the real vectors $E_k$ and real values ${\lambda }_k$ of the covariance matrix C.
• Sort the true values of ${\lambda }_k$ in descending order and form the matrix A of the main components from the true vectors $E_i$ according to the list obtained.

Each gesture is measured by 8 EMG sensors, and the temporal signal is estimated using 10 statistical estimators. A total of 80 dimensional vectors are used to describe one movement. To reduce the number of dimensions and simplify the structure of the gesture classification algorithm, the PCA method was applied to the EMG data.

3.3. Random Forest Classifiers for Hand Motion Identification

A random forest classifier includes many decision trees. Mostly, the random forest classifier, which is an advanced artificial intelligence algorithm used in robotics, produces high accuracy and good prediction results. Random forests, or random decision forests, are a machine learning technique for solving classification, regression, and other problems. A random forest classifier is constructed from a large number of decision trees whose classification properties are determined during training. In the research study, a generative array of decision trees was created. In each tree, depending on the training data obtained from the feature vectors, a decision is made on which class to assign the feature vector. The votes of all trees for the classes are checked, and the class that receives the most votes is the output of the classifier.

3.4. Pareto Optimization

Pareto optimization is a multi-objective optimization technique used to find a set of optimal solutions that best balance multiple conflicting objectives. In the context of EMG feature selection, we use Pareto optimization to identify the optimal subset of EMG features that best represent the underlying signal while minimizing the number of features required for classification. To do this, a Pareto front is first constructed by generating multiple solutions that represent different trade-offs between performance and feature number. Each solution is evaluated on the basis of its classification accuracy and the number of features it employs. The set of all solutions that cannot be improved in one objective without sacrificing performance in another objective is called the Pareto front.

The Pareto front combines non-dominated multi-objective optimization solutions. Given a set of solutions $S=s_1,s_2,\dots ,s_n$ with m objectives $f_1,f_2,\dots ,f_m$, the Pareto front $P^{\ast }$ is defined as follows. Let $F\left(x\right)=(f_1\left(x\right),f_2\left(x\right),\dots ,f_m\left(x\right))$ be the m-dimensional objective function vector and X be the decision variable vector. The Pareto front $P^{\ast }$ is defined as: $P^{\ast }=x\in X|$ there is no other $x^{\prime }\in X$ such that $f_j\left(x^{\prime }\right)<=f_j\left(x\right)\forall j\in 1,2,\dots ,m$ with at least one strict inequality. In other words, a solution $s_i$ is on the Pareto front if there does not exist any other solution $s_j$ that is better than $s_i$ in all objectives. The Pareto front contains all non-dominated solutions, which means that there is no solution in the front that is worse than another solution in any objective. Constructing the Pareto front involves finding all non-dominated solutions in the solution set S. Once the Pareto front is constructed, we select a solution that best fits their needs based on their preference for accuracy versus the number of features used. This approach ensures that the selected solution is not only accurate but also parsimonious and less prone to overfitting. We present the EMG feature selection procedure using Pareto optimization as Algorithm 1.

Algorithm 1: EMG feature selection using Pareto optimization

Input: EMG dataset D, maximum number of features m Output: Set of Pareto-optimal feature subsets P $F\leftarrow$ set of all possible feature subsets of size $\le m$ $P\leftarrow \varnothing$ $S\leftarrow$ random subset of features of size m while not stopping criterion met do     $P\leftarrow P\cup$ non-dominated solutions in S     $S\leftarrow$ next subset in F end while Return P

The above algorithm takes as input an EMG dataset D and the maximum number of features to consider m. It first generates a set F of all possible feature subsets of size $\le m$. It then initializes an empty set P to store the Pareto-optimal feature subsets. The algorithm then randomly selects a subset S of m features and begins an iterative process to generate the Pareto-optimal feature subsets. At each iteration, it adds the non-dominated solutions in S to the set P. It then updates S to the next subset in F, and the process continues until a stopping criterion is met. Finally, the algorithm returns the set of Pareto-optimal subsets of features P.

4. Results

Each contraction of the arm muscles during one of the seven movements was recorded by 8 EMG sensors. Estimates of temporal signals were calculated from the recorded signals. The average values of the statistical indicators of the eight sensors are presented in Figure 2. The minimum standard deviation of 0.007 was obtained when measuring the 6th gesture. The highest standard deviation of 0.253 was recorded for the 7th gesture. The minimum value −0.77 was obtained when measuring the 2nd gesture. The highest min value of −0.027 was recorded during the 6th gesture. The minimum (max) value was 0.007 when measuring the 6th gesture. The maximum (max) value of 0.77 was recorded during the 2nd gesture. The minimum value of 5.87 was obtained when measuring the 6th gesture. The highest value of 90.87 was recorded during the 7th gesture. For the average sensor, the minimum amplitude change value of 0.02 was obtained when measuring the 6th gesture. The maximum amplitude change value of 0.735 was recorded during the 7th gesture. The minimum value of the first amplitude jump of 0.018 was obtained when measuring the 7th gesture. The highest value of the first amplitude jump of 0.447 was recorded during the 3rd gesture. The minimum absolute value of 0.005 was obtained when measuring the 6th gesture. The highest absolute value of 0.189 was recorded for the 7th gesture. The minimum wavelength value of 1.19 was obtained during the measurement of the 6th gesture. The highest wavelength value of 46.26 was recorded during the 7th gesture. Minimum Wilson amplitude value of 0.0 was obtained by measuring the 1st, 2nd, 5th, and 6th gestures. The highest Wilson amplitude value of 10 was recorded during the 7th gesture. The minimum, mean, and maximum values of the gesture features are summarized in

Table 1. Minimum values of gesture features.

Gesture	Standard Deviation	Minimum	Maximum	Crossing the Zero Axis	Average Change in Amplitude	First Amplitude Jump	Mean Absolute Value	Wave Length	Wilson Amplitude
1	0.055	−0.229	0.225	49.91	0.141	0.099	0.036	8.87	2.878
2	0.089	−0.333	0.317	51.13	0.223	0.117	0.059	14.59	3.66
3	0.101	−0.357	0.32	58.91	0.259	0.190	0.066	16.2	4.22
4	0.081	−0.286	0.276	52.53	0.16367	0.061	0.054	13.43	3.49
5	0.058	−0.234	0.212	48.64	0.069	0.098	0.040	8.976	2.746
6	0.014	−0.065	0.049	22.85	0.17433	0.05	0.009	2.299	0.773
7	0.143	−0.434	0.415	73.46	0.411	0.297	0.104	25.58	5.928

,

Table 2. Mean values of gesture features.

Gesture	Standard Deviation	Minimum	Maximum	Crossing the Zero Axis	Average Change in Amplitude	First Amplitude Jump	Mean Absolute Value	Wave Length	Wilson Amplitude
1	0.012	−0.476	0.028	20.87	0.032	0.021	0.009	2	0
2	0.011	−0.77	0.018	20.25	0.032	0.02	0.009	1.94	0
3	0.028	−0.675	0.057	32.62	0.079	0.024	0.019	4.53	0.625
4	0.014	−0.607	0.036	22.37	0.037	0.023	0.01	2.25	0.125
5	0.012	−0.51	0.026	17.62	0.033	0.02	0.009	1.93	0
6	0.007	−0.13	0.007	5.875	0.02	0.018	0.005	1.19	0
7	0.05	−0.746	0.137	54.87	0.145	0.024	0.035	8.63	1.875

and

Table 3. Maximum values of gesture features.

Gesture	Standard Deviation	Minimum	Maximum	Crossing the Zero Axis	Average Change in Amplitude	First Amplitude Jump	Mean Absolute Value	Wave Length	Wilson Amplitude
1	0.112	−0.04	0.5	76.37	0.278	0.208	0.072	17.78	6.375
2	0.211	−0.03	0.77	79.75	0.515	0.269	0.138	34.4	8.5
3	0.193	−0.08	0.635	82.87	0.48	0.447	0.125	30.75	8
4	0.177	−0.04	0.614	81	0.314	0.08	0.12	29.62	7.625
5	0.126	−0.048	0.496	77.5	0.1	0.215	0.079	19	6.5
6	0.026	−0.027	0.119	43.25	0.472	0.099	0.016	3.867	2.25
7	0.253	−0.144	0.716	90.87	0.735	0.687	0.189	46.26	10

, respectively.

PCA showed that gesture classes overlap strongly in the feature space. There are no clear dividing surfaces between classes to identify a separate group of gestures. In order to solve the motion classification of gestures, it is necessary to apply non-linear classification methods, which can be used to evaluate the non-linear distribution of the data. For this purpose, the random forest classifier is used in the project research.

Figure 3 shows the projections of the EMG signals to the two principle components P1 and P2. Different classes of gestures are represented by different colours. Each individual point represents one measurement of hand motion in the two-dimensional space of P1, and P2. Each individual movement measurement is represented by a separate circle, the colour of which depends on the gesture class (type). From the cloud of points, it can be seen that the different classes of gestures overlap strongly, and no clear dividing line can be drawn between them.

Figure 4 shows the classification result using the random forest classifier. The figure shows the confusion matrix, which contains the delayed true values and the predicted values of the classifier. If the classification result is evaluated as a number from 0 to 1. Multiplying the given number by 100% results in average expression of the result. The confusion matrix shows how well the classifier was able to accurately recognize a certain class of motions and with which class of motions the real one is confused.

Confusion matrix (Figure 4) shows the highest classification accuracy was obtained for the identification of the 6th gesture. In this case, the accuracy was 99%. The worst accuracy result was obtained for the classification of the 2nd gesture. In this case, a classification accuracy of 73% was achieved. The 7th gesture was classified with 95% accuracy. Meanwhile, gestures 4 and 5 were classified by 75% accuracy.

5. Discussion

Looking at the results, it is clear that using traditional rehabilitation approaches as a foundation, digitized rehabilitation treatment has the most potential to enhance recovery results. Patients must engage with the digital device in a way that comes naturally to them in order for computer-assisted treatments to be effective, and they must also obtain support from the computer system depending on how well they do throughout the rehabilitation session. Gesture recognition has been proposed as a potential remedy for applications of the human-machine interface to alleviate this problem, indicating that digital rehabilitation treatment offers the greatest promise as a complement to traditional rehabilitation procedures and could improve rehabilitation outcomes [29]. To be effective, computer-assisted treatments require patients to interact with the digital device in a way that seems natural to them, while also receiving support from the computer system depending on their performance throughout the rehabilitation session. Gesture recognition has been explored as a feasible approach for human-machine interaction applications to overcome this issue.

Gesture recognition based on EMG is a method that allows dynamic evaluation of muscle work in real time. EMG is focused on electrical signals from muscles that are controlled by the cerebral system and are generated during muscle contraction, the signal reflects physiological properties of the muscles; the EMG signal is the motor electrical activity of the muscles [58]. Nowadays, in rehabilitation, it is preferable to use surface EMG signals are recorded with non-invasive electrodes detected at the surface and to obtain information about the activation time or intensity of surface muscles. It is the simplest way to measure muscle activation patterns and understand the behaviour of the human body under normal and pathological conditions [59].

Scientists also identified the following advantages of surface EMG during rehabilitation: that the electrodes are easier to apply, the patient feels minimal discomfort or pain, skin stretching or allergy when using them, the signals are better reproduced and they are very suitable for various movement recognition and analysis during exercising or training. However, the surface electrodes, which, in most cases, cover a larger area, take more information from the adjacent muscles, which can affect the accurate measurement results, which is very important in physical training assessment, recovery effectiveness or movement recognition and analysis in pathological conditions, especially after stroke, brain trauma, or spinal cord injuries [60].

Recently, EMG-based hand gesture recognition systems have faced challenges such as categorisation encounters, poor classification accuracy, low generalisation ability, limited training data and lack of robustness, and long time to calibrate the system for new users [61].

The development of an existing robotic exoskeleton for upper limb and hand rehabilitation is already in progress. Currently, it is able to perform passive exercises or training during rehabilitation after a stroke or trauma. One of the gaps in the project is the absence of a convenient algorithm to enable exercises and training with active-assisted rehabilitation. The use of robotic assist devices is also increasing in the application of rehabilitation systems and is and will allow the possibility of “seeing”/allowing patient muscle performance directly, helping in decision-making before and after training due to difficulty and intensity of the exercise, determining muscle response to perform ergonomic tests and physical load tolerance, monitoring and analyzing the effectiveness of exercise/training programs and muscle recovery; also for patients, it helps to “find” and train muscles, and enables analyses and improvement of sports activities and training progress [62]. To monitor accurate muscle performance in real time under different pathologies or dysfunctions, achieving an accurate and easy method with low latency for gesture recognition remains a challenge in real life, especially for assistive computerized systems and rehabilitation.

In our research, gesture and related muscle activity were recorded with eight EMG sensors. Each contraction of the arm muscles during one of the seven movements was recorded by eight EMG sensors, and the EMG time signal was recorded for up to 5 s. The results show that the gesture classes overlap strongly in the feature space. There are no clear dividing surfaces between classes to identify a separate group of gestures. In order to solve the motion classification of gestures, it is necessary to apply Pareto optimised non-linear classification methods, which can be used to evaluate the non-linear distribution of the data. A comparison with other approaches is offered in

Table 4. A comparison with other related approaches.

Methods	Number of Motions	Number of Subjects	Accuracy
Ours	7	25	92%
Zhou et al. [63]	8	5	91.18%
Lu et al. [64]	6	8	84.1%
Meyers et al. [65]	12	7	77.l%
Lu et al. [66]	6	1	75%

.

6. Conclusions

This study presents an approach to robot-assisted hand motion therapy for stroke patients. The study used machine learning techniques to determine and describe hand motion patterns in healthy individuals. Electromyographic data were collected from the flexion and extension muscles of the forearm using surface electrodes. Feature selection was performed using the Pareto optimization algorithm. The time and frequency characteristics were used as parameters in machine learning algorithms to recognize seven hand gestures and track rehabilitation progress. Research showed that the system was capable of reconstructing hand/finger movement kinematics and simulating the behaviour of every motion pattern. The correlation between the computed joint trajectories and the monitored hand movement was 0.96 on average, and statistical research carried out in various machine learning setups revealed a 92% accuracy in measuring the precision of motion patterns. The experimental study was conducted to determine which statistical estimates of the temporal signal can be used as motion discriminating features and suitable for developing a classification algorithm. The study analyzed nearly 10,500 individual measurements of the specified gestures. The results showed that the highest classification accuracy was obtained for the identification of the 6th gesture, while the worst accuracy result was obtained for the classification of the second gesture. Gestures 4 and 5 were classified with 75% accuracy. In summary, this study presents a promising approach to robot-assisted hand-motion therapy for stroke patients, which could potentially monitor the progress of rehabilitation or other medical processes. More studies are required to confirm the efficacy of this approach and refine the system for practical use.

The study’s results showed a high accuracy in measuring hand motion patterns, but the efficacy of this approach must be confirmed in clinical settings with larger patient populations to establish its reliability and effectiveness. Future research should investigate developing personalized therapy plans and adoption of the algorithm to react to a larger variety of potential actions, all based on individual patient needs, rather than a generalized approach, allowing to potentially learn to recognize patterns automatically without the need for manual feature engineering, potentially improving classification accuracy. While the study evaluated the accuracy of the system in measuring hand motion patterns, the long-term effects of robot-assisted therapy on stroke patients’ functional recovery and quality of life require further investigation, including the integration of this approach with other therapies, such as occupational and physical therapy.