Robustly Effective Approaches on Motor Imagery-Based Brain Computer Interfaces

Abstract

Motor Imagery Brain Computer Interfaces (MI-BCIs) are systems that receive the users’ brain activity as an input signal in order to communicate between the brain and the interface or an action to be performed through the detection of the imagination of a movement. Brainwaves’ features are crucial for the performance of the interface to be increased. The robustness of these features must be ensured in order for the effectiveness to remain high in various subjects. The present work consists of a review, which includes scientific publications related to the use of robust feature extraction methods in Motor Imagery from 2017 until today. The research showed that the majority of the works focus on spatial features through Common Spatial Patterns (CSP) methods (44.26%). Based on the combination of accuracy percentages and K-values, which show the effectiveness of each approach, Wavelet Transform (WT) has shown higher robustness than CSP and PSD methods in the majority of the datasets used for comparison and also in the majority of the works included in the present review, although they had a lower usage percentage in the literature (16.65%). The research showed that there was an increase in 2019 of the detection of spatial features to increase the robustness of an approach, but the time-frequency features, or a combination of those, achieve better results with their increase starting from 2019 onwards. Additionally, Wavelet Transforms and their variants, in combination with deep learning, manage to achieve high percentages thus making a method robustly accurate.

1. Introduction

The brain is the most complex human organ and even today it has not been fully mapped. Everything a person does, thinks or imagines is a result of the communication between different parts of the brain. This communication is conducted through electrical pulses. The frequency depends on the state in which the individual is at the given time. The frequencies of the brain signals have been categorized as shown in

Table 1. Brainwaves.

Brainwaves	Hz/V
Delta ($\delta$)	≤4 Hz, 100 $\mathsf{\mu }$V
Theta ($\theta$)	4–8 Hz, <100 $\mathsf{\mu }$V
Alpha ($\alpha$)	8–13 Hz, <50 $\mathsf{\mu }$V
Beta ($\beta$)	13–30 Hz, <30 $\mathsf{\mu }$V
Gamma ($\gamma$)	≥30 Hz, ≤10 $\mathsf{\mu }$V

[1,2,3]:

The brain signals can be acquired, filtered, and processed in order for an action to be executed, such as the movement of a robotic part. Brain Computer Interfaces (BCIs) are systems by which the communication between the human brain and applications, which in some cases include the control of hardware, is achieved. The goal of these systems is to transmit the command from the brain to the final implementation through the input signals that constitute the brain signals. The basic idea on which BCI is based, is to allow the individual to communicate and manage operations without using muscles but only through brain activity [4]. These systems have been used in various fields such as communication [5], prosthetics [6], accident prevention [7], health monitoring [8], timely prediction of brain disorders [9] or even entertainment applications [10]. In addition, the continuous evolution of BCI systems has led to the implementation of this technology in training strategies, Internet of Things applications, cognitive skills analysis, and helping identify problems such as writing or drawing. More specifically, in the works [11,12,13] the authors present analyses based on eye motion experiments, which lead to approaches that can identify cognitive abilities, fixation parameters and factors affecting eye–hand coordination. The results of the above papers show how, through the BCI systems, the subject can measure his/her cognitive skills, complex tasks execution, or learning abilities, and be provided with early detection of a problem or to improve the above skills. Moreover, ref. [14] presents the correlation between executive function and algorithmic thinking, and [15] shows how various media contents affect brain waves in terms of the individual’s attention ability. As already mentioned above, BCI systems can interact with the subject’s environment. The authors of [16] designed a BCI system controlling an Internet of Things-based robot. As can be seen from all the above, BCI systems are constantly evolving and finding applications in a variety of different areas, from the communication between the individual and his/her environment to the physical activities of the individual himself/herself.

1.1. Acquisition Methods

The process by which brain signals are loaded as input signals can be divided into three categories [4,17,18]:

Non-invasive: This technique is based on electrodes that measure the potential generated by the human brain through electroencephalography (EEG) or take measurements of the magnetic field through magnetoencephalography (MEG). The sensors are located at specific points on the scalp’s surface, where it has been observed that the function of neurons is more intense than in most human functions, resulting in the sensors receiving a stronger signal.

Semi-invasive: With this technique, electrodes are located on the exposed surface of the brain, measuring the electrical activity coming from the cerebral cortex (electrocorticography (ECoG)). The electrodes can be placed on the outside of or under the dura mater.

Invasive: In this technique, sensors are placed into the brain. More specifically, micro-electrodes are located in the cortex, measuring the electrical activity of every single neuron.

Non-invasive techniques constitute the safest method for measuring brain electrical activity or magnetic field because the electrodes are placed on the surface of the scalp and they do not penetrate it. For the same reason, semi-invasive and invasive techniques are considered risky because the placement of the electrodes is performed with surgery and in the case of invasive techniques, where micro-electrodes are in contact with the cortex, the procedure is even riskier. Additionally, the cost of non-invasive equipment is low as it can be easily found on the market. Of course, the price varies depending on the type of equipment and depending on the implementation. Nevertheless, the cost is still very low compared to the other two techniques. Additionally, for the reasons mentioned above, the non-invasive technique is not as effective as the other techniques [19] and needs more preprocessing stages. Due to the low cost of the equipment and measurements of the neurons’ activity from the surface of the scalp, resulting in a significant increase in noise and a decrease in spatial resolution [17]. However, this technique in applications, in combination with the development of technology in the field of neurotechnology, has managed to achieve high classification performances.

1.2. Communication Approaches

Brain Computer Interfaces are also divided into categories based on how the brain sends commands and communicates with the application. Some of these approaches are based on the human senses. The event-related potentials (ERP) techniques are based on the responses of the individual when observing, separately or in groups, repetitive visual (SSVEP, P300), auditory (SSAEP) or somatosensory (SSSEP) stimuli, for which their oscillation is periodical. As the individual focuses on the stimulus, the frequencies of the brain’s electrical waves tend to match the frequencies of the oscillating visual stimuli. In this way, the command is given through the brain’s signals to the implementation in order for a function to be performed [20,21,22,23,24,25].

Additionally, there are approaches based on the individual’s imagination. Motor imagery (MI) is based on the idea that the individual consciously imagines that he/she is performing a movement [26]. Of course, this image must represent a movement that is possible to perform by a human, such as the movement of the arms or legs. Research has shown that the areas of the brain that are responsible for creating movements are associated with areas where images of movement are created [27,28]. Fundamentals for motor imagery are two psychical phenomena. Event-related de-synchronization (ERD) is the amplitude’s decrease of a signal rhythm, when the individual imagines or performs a movement. This decrease is observed in specific cortex regions related to movements or the imagination of it, while in other areas of the brain an increase in the amplitude is observed. This amplitude increase, in the signal’s rhythm, is the second phenomenon called event-related synchronization (ERS) [29].

1.3. Process

Figure 1 shows the main process of a BCI system which consists of six basic stages [30]. The first stage concerns the acquisition of the brain signals with one of the techniques analyzed in the previous section. As mentioned, the way in which these data will be acquired has a very important role. The majority of studies focus on non-invasive techniques and datasets, due to the easy installation of the equipment and the low cost. Preprocessing level consists of methods aimed at improving signal quality. Common steps in preprocessing level are subsampling, signal frequency filtering, channel scaling and selection, spatial filtering, and frequency decomposition [31]. At the feature extraction level, the fully preprocessed signals insert into an extraction algorithm in order for critical information about the brainwaves to be extracted and analyzed in time and/or frequency domains or with spatial filtering. Then this information is grouped into the feature vectors, which are categorized into classes by the classifiers [32]. The next stage is the green implementation in the application. This stage implements the features classes of the preprocessed brain signals. The final stage consists of the feedback that BCI systems receive [33,34]. Feedback has a great impact on the increased performance. Through this process, the mental strategy of the individuals is improved during the entry of their EEG signals into the BCI system [35].

1.4. Feature Domains

The feature extraction stage has a crucial role in the BCIs in order for the system to be as efficient and as robust as possible. Robustness shows if and how much the accuracy changes based on the different brain characteristics of each participant. A robust method will extract only the features which will keep the performance of the model stable. This means that the performance will not be affected across the different sessions and participants. Based on the analysis domain, features can be divided into four categories [36,37,38]:

Time-domain: The time-domain features (TDF) are calculated based on the raw brain signals in relation to the change of time [39]. These temporal features constitute the simplest form of information on EEG signals. Additionally, based on the time domain, fractal analysis for nonlinear features can be extracted.

Frequency-domain: Frequency-domain features (FDF) describe the changes of the signal in relation to the frequency. The analysis of the spectral features is based on the Power Spectral Density (PSD) with which useful correlations can be drawn.

High importance statistical features, such as standard deviation, correlation, contrast, minimum, maximum, root mean square and average of a specific part of the EEG signal, are extracted through the above domains.

Time-frequency domain: The features extracted through time-frequency analysis (TFDF) constitute the basis for extracting useful information and patterns. Through time-frequency analysis, changes in the frequency of EEG signals can be acquired over time.

Spatial domain: Through this domain the representation of the EEG in a spatial way is achieved. The features of this domain are extracted by the interaction of multiple channels and are not limited to the study of a single one. Additionally, spatiotemporal features describe activities of various parts of the brain that are related to an action [38].

Furthermore, Wang, P. and Jiang, A., in [40], have roughly separated the features in into three large categories based on their type:

• Discriminative features;
• Statistical features;
• Data-driven adaptive features.

Discriminative features include all those features extracted in time, spectral and spatial domains, including all the variants by the corresponding domain methods. Statistical category, except the statistical features, includes also the entropy. Finally, the data-driven adaptive category includes the features extracted from the Boltzmann machine, neural networks, or deep learning approaches.

2. Review’s Approach

2.1. Structure of the Work

The present review is organized as follows: The first section consists of the introduction in which acquisition and communication methods are described, along with the basic structure of MI-BCI and the feature domains. Section 2 presents the research methods used for the present review and the early statistics of the research as well. Section 3 presents the related works that have been carried out. Section 4 presents the results of the present review. In more detail, the most used methods are shown, and which domain of features the authors have focused on in order to achieve robustness, as well as the effectiveness of each method based on accuracy and robustness. It also shows the comparison of the most effective methods based on the datasets in which they were used. Section 5 contains a discussion of the results and Section 6 summarizes the final conclusion of this work. At this point it is worth noting that research and datasets have been based solely on electrical signals acquired through electroencephalography. The plan that was followed was after the selection and study of the works, was the grouping of works in terms of feature extraction methods and the feature domain that the authors relied on in order for a trend to emerge about which method and which domain are the most used and that the authors focus on for achieving robustness. Then, the approaches that had numerical results in terms of both accuracy and robustness were presented. Cohen’s Kappa value was selected to measure robustness. Next, the most efficient approaches were selected in terms of feature extraction methods and classifiers, based on the datasets to which they were applied. In this way, for each dataset, there was one approach that was the most effective among others implemented in the same folder. Finally, conclusions emerged as to whether robustness matches the trend of the most used feature extraction methods and feature domains and what the most efficient robust approach is and whether it exists for all the cases.

2.2. Research of Scientific Works

Establishing criteria for the selection of scientific papers was very important in order to analyze the information and draw conclusions in a documented way. The literature search was based on some scientific questions aimed at accurately extracting results in order for the final review to contain all the fundamental information:

• Which are the most used feature extraction methods in Motor Imagery BCIs?
• Which methods are robust?
• Which approaches have high robustness and high accuracy as well?

Scopus (www.scopus.com, accessed on 15 November 2021) was used for searching for scientific works. It is a valid and reliable database that allows you to search scientific papers. The words used for the revision refer to the title of each work, the abstract, and the keywords. A chronological filter was also used to display the papers of the last five years in order for the review to relate to the latest developments on the subject. More specifically, the search in Scopus was as follows:

TITLE-ABS-KEY ((feature AND extraction) AND (robust) AND (motor AND imagery)) AND PUBYEAR > 2016

The review process of the papers based on the following acceptance criteria:

• The paper describes a feature extraction method of the MI-BCI field;
• The method described in the paper aims to extract robust brain activity features;
• The presentation of the numerical results of the metrics is necessary.

After the initial review of the results, the need emerged to establish rejection criteria in order to remove works whose content deviated from the main objectives of the review:

• Toolboxes are described;
• The paper is not related to the BCI field;
• The paper does not describe a feature extraction method;
• The proceedings were not accessible.

The research was conducted on 15 November 2021 and the results, without the usage of rejection criteria, were 52 papers. Figure 2 shows the various stages of the methodology followed, based on the PRISMA (www.prisma-statement.org, accessed on 12 April 2022) flow diagram. At this point, it is worth noting that the number of 52 papers concerned the primary research based on the Scopus, to which the criteria analyzed above were applied. During the study of the results, the need for secondary research arose, through citation searching in Google Scholar (www.scholar.google.com, accessed on 11 April 2022). The reason for this secondary search was the expansion of knowledge for specific subjects and terms in the MI-BCI field and EEG. As a result, 45 additional works were raised and referenced but they are not included in the following numeric or comparative tables or graphs.

2.3. Research Early Statistics

Based on the results, a connection emerged between the countries of origin of the scientific papers. With the usage of VosViewer (www.vosviewer.com, accessed on 3 December 2021) a bibliographic coupling network is constructed, between the countries with the largest number of scientific papers. As Figure 3 shows, China, Japan, and the United States of America have more intense bibliographic coupling based on the number of papers and the total number of citations. Apart from that, China is the country with the highest number of papers and citations (Link Strength). Japan, the United States of America, Australia, the United Kingdom and India follow.

Based on Figure 4, it seems that there is an increase of publications in 2019. Almost 69% of the papers were published in 2019, describing a modified version of CSP (Common Spatial Patterns) methods or their variants. CSP is one of the most common and popular feature extraction methods in Motor Imagery which had an intense emergence in 2019.

3. Related Work

In recent years, reviews and surveys have been published about the performance comparison of feature extraction methods in BCI systems since the feature extraction method plays a crucial role in the performance of each approach. In [41] the authors present a detailed presentation of the performance of deep learning approaches and their feature extraction methods for MI-BCI applications, including information about datasets and the feature domains. In the survey [42], the advantages and the disadvantages of feature extraction methods for BCI systems have been analyzed. Furthermore, the review [43] presents a very detailed and analytical study of various feature extraction approaches for different fields such as Motor Imagery, epilepsy, attention, P300, etc. The present review aims to present the MI-BCI feature extraction methods along with the classifiers used in each work to achieve a robust performance. To achieve this goal, the metric Kappa was set to measure robustness, something that was not presented in the above works. More specifically, the above reviews show the accuracy of each approach and in some cases the Kappa value instead of the accuracy percentages. This fact shows that robustness has a secondary role and the first one belongs to the accuracy of each approach. This work is based on the robustness of each approach and as a secondary comparison method has set the combination’s effectiveness of accuracy and robustness.

4. Efficient and Robust Approaches

4.1. Feature Extraction Methods

Table 2. Results.

Methods	Citations	Percentages
CSP-based	[40,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61]	44.26%
PSD methods	[52,62,63,64,65,66,67,68,69]	21.42%
Wavelet Transform-based	[48,53,54,64,70,71,72]	16.65%
Deep learning	[73,74,75,76]	9.52%
Riemmannian	[58,59,77]	7.14%
Boltzmann	[68,75]	4.76%
LCD	[45,78]	4.76%
dFC	[79]	2.38%
MFCC	[67]	2.38%
log-BP	[48]	2.38%
TPF	[80]	2.38%
RFB	[81]	2.38%
HHT	[82]	2.38%
EMD	[63]	2.38%

shows the feature extraction methods used on MI-BCI applications identified in this review. As already mentioned above, almost half of the papers used more than one method. The combination of methods is intended to extract more robust features and increase the accuracy. The research showed that the most used feature extraction methods are the Common Spatial Patterns-based approaches (CSP), Power Spectral Density (PSD) methods and Wavelet Transform-based approaches.

In the category of CSP novel or modified approaches of methods are included, such as the 8th-order Butterworth bandpass-filters in CSP with Tikhonov regularization (FB-TRCSP), Frequency Bank CSP (FBCSP), Frequency Domain CSP (FDCSP), Regularized CSPs (RCSP) and DTimeWrapping-Based CSP. CSPs as a feature extraction method, expressed by a filtering algorithm extracting eigenvalues of two modes in the spatial domain. Through several EEG channels, this algorithm extracts the spatial components while increasing the difference of the variance between the two modes [44].

The Power Spectral Density category includes novel approaches such as Weighted Difference of Power Spectral Density (WDPSD) as well as Fast Fourier Transform (FFT), Welch, Welch periodograms and the adaptive autoregressive (AAR) method. PSD methods analyze EEG signal characteristics. The conversion of the EEG signal from the time domain to the frequency domain, to export the description of the signal’s distribution [83], has as a result the extraction of the event-related de-synchronization (ERD) features [42,84].

Finally, Wavelet Transform methods (WT) also include novel approaches such as Flexible Analytic Wavelet Transform (FAWT), Multivariate Empirical Wavelet Transform (EWT), Automatic Independent Component Analysis with Wavelet Transform (AICA-WT) and Successive Decomposition Index (SDI). Methods such as Discrete Wavelet Transform (DWT) and Continuous Wavelet Transform (CWT) are also included. Wavelet Transform constitutes a mathematical method identifying information from continuous data. It has a multi-resolution nature and the processing of non-stationary signals is achieved in the time-frequency domain [42].

As shown in Table 2, the majority of the works focus on CSP methods and their variants. In the work [85], the authors show that the Common Spatial Patterns is one of the most popular approaches in terms of effectiveness. The CSP algorithm was first introduced in 1991 [86] as a method to extract EEG signals’ abnormal components [87]. In 2000 it was presented as a feature extraction method for classification [88]. This means that CSP is a method that has been evolving for 30 years. It constitutes a method by which handmade features are extracted effectively with high accuracy [40,45]. In addition, its variants aim to solve common CSP problems such as noise sensitivity or overfitting when the sample size is small [44,62].

Fifty-eight percent of the works implemented with CSP methods or their variants were published in 2019. Furthermore, based on Table 2 and Figure 5, it is shown that the researchers aim to extract discriminative features from the spatial domain, leading to the trend being that the studies focus on spatial features to increase the performance and the robustness.

4.2. Feature Domains Distribution

Figure 6 shows the distribution of feature domains in the present review based on the research conducted. The majority of the works focus on spatial features. This is because most of the works focus on CSP or on a combination of CSP and other methods.

The distribution of the features based on the adopted categorization of Wang, P. and Jiang, A. is shown in Figure 5. It is obvious that the majority of the features were extracted manually based on the time, frequency, time-frequency and spatial domain with the appropriate feature extraction methods.

Many papers examined in the present research combine several methods and approaches, aiming to extract features from different domains. In the following percentages are included all the different feature extraction methods, domains and feature types were identified in the research, even if those were in the same work.

4.3. Efficient Approaches

For the present research, the effectiveness of each approach will be based on two basic metrics: classification accuracy and Cohen’s Kappa value. If the Kappa value has a high score, the approach does not appear to have a randomness index in its accuracy, but there is an agreement in the score of the accuracy [64].

4.3.1. Accuracy

The feature extraction process has a crucial role in the accuracy of the system [89].

Table 3. Approaches with the highest accuracy.

Feature Extraction	Classifier	Kappa	Avg. Accuracy	Dataset	Work
EMD-FFT	SVM	0.9244	95.89%	BCI III Dataset 1	[63]
RJSPCA	SVM-based	0.916	78%	BCI III Dataset 1	[46]
MEWT	FFNN	0.95	99.55%	BCI III IVa	[71]
CWT	DCNN	0.9869	99.35%	BCI III IVa	[90]
STFT	DCNN	0.9798	98.7%	BCI III IVa	[90]
SDI	FFNN	0.9693	97.46%	BCI III IVa	[64]
SDI	SVM	0.915	93.05%	BCI III IVa	[64]
FAWT	Subspace kNN	0.917	95.97%	BCI III IVa	[70]
CSP	SGRM	0.54	77.7%	BCI III IVa	[47]
MEWT	FFNN	0.95	99.52%	BCI III IVb	[71]
SDI	FFNN	0.978	97.5%	BCI III V	[64]
MEWT	FFNN	0.894	91.8%	BCI III V	[71]
CNN	LSTM	0.55	77.44%	BCI IV 2a	[73]
CSP/LCD	SRDA	0.73	79.7%	BCI IV 2a	[45]
CSP	SVM	0.92	96.4%	BCI IV 2b	[51]
CSP	Bootstrap	0.92	96.5%	BCI IV 2b	[51]
CNN	LSTM	0.544	65.88%	BCI IV 2b	[73]
CSP	SGRM	0.57	78.2%	BCI IV 2b	[47]

shows the feature extraction methods along with their classifiers combined to achieve the accuracy percentages and Kappa values of each approach in specific datasets. The average classification accuracy is extracted from the mean accuracy of the total number of subjects. In the following works, the metric for the robustness (Kappa) has been calculated by the authors of each paper. During the review of proceeding surveys, works with high accuracy percentages were identified, but without the calculated Kappa value. Therefore, it was not possible to measure the robustness of the corresponding approach, despite its high accuracy, and for this reason, they were not included in the following tables.

The majority of the works which had an experimental part and not only theoretical used a dataset from BCI Competition III and IV (www.bbci.de/competition/, accessed on 20 December 2021).

Most of the approaches were used on the dataset BCI Competition III IVa. In this motor imagery dataset EEG actions were recorded, corresponding to the movement of the right hand and foot. The records were extracted from five subjects, with 118 EEG channels in a 10/20 electrode configuration, at 1000 Hz sampling frequency. At this point it is worth noting that the researchers, in [51], used the BCI competition IV 2b dataset and the accuracy rate achieved was 96.4% and 96.5% with SVM and bootstrap respectively with a Kappa value equal to 0.92. This dataset includes two motor imagery classes, right/ left hand, exported from nine subjects. The sampling frequency was 250 Hz and for the EEG recording three bipolar channels were used (C₃, C_z, C₄) and three EOG (electrooculogram) signals. EOGs are responsible for the noise representation of eyes blinking. In the BCI IV 2a dataset, where the information was obtained from twenty-two EEG channels and three EOG from nine subjects, the work [45] achieved 79.7% accuracy and 0.73 in Kappa value for a 4-class problem. Additionally, in [64], the accuracy percentage of 99.52% with a Kappa value of more than 0.9 was achieved in the BCI III IVb dataset, which contains two classes (left hand, foot) of one single subject and continuous data and 97.5% in the BCI III V dataset containing three classes (left hand, right hand, word association) from three subjects with a Kappa value equal to 0.978. Finally, dataset 1 from the BCI III competition was used, which contains two classes (tongue, left pinky) from one subject, with 64 ECoG channels. With this dataset, ref. [63] achieved 95.89% accuracy with Kappa value 0.9244. Additionally, the paper [90] in BCI III IVa achieved 99.35% accuracy and a 0.9869 Kappa value by combining Continuous Wavelet Transform (CWT) with a Deep Convolutional Neural Network (DCNN) based AlexNet model. This dataset includes 118 EEG channels and information from five subjects.

Table 4. Datasets.

Dataset	Channels	Classes	Subjects
BCI III Dataset 1	64 ECoG	2	Continuous EEG
BCI III IVa	118 EEG	2	5
BCI III IVb	118 EEG	2	Continuous EEG
BCI III V	32 EEG	3	Continuous EEG
BCI IV 2a	22 EEG/3 EOG	4	9
BCI IV 2b	3 bipolar EEG/3 EOG	2	9

shows the basic characteristics of each dataset.

4.3.2. High Robustness

In order to extract conclusions about the robustness of each method, the metric of accuracy was not enough, as its value can vary in different sessions or participants and its high value may have arisen by chance or solely for a specific subject. For this reason, the value of the Cohen Kappa coefficient (K) was used as a factor by which the degree of robustness of each method would be determined. Kappa value is a metric that measures the inter-rater agreement for qualitative items of a model. This measurement is taken by the comparison of the performance the model has, through its accuracy, with the random accuracy that the model could have. This statistical measure is considered more robust than other metrics because it includes the agreement results, which occurred by chance [51,91]. Based on a confusion matrix the Kappa value is calculated by the following fraction:

This statistical measure is considered more robust than other metrics because it includes the agreement results, which occurred by chance:

$$Kappa\equiv \frac{2\ast (TN\ast TP-FN\ast FP)}{(TN+FN)(FN+TP)+(FP+TP)(TN+FP).}$$

In general, the Kappa value constitutes a metric, which shows the robustness of an approach. A robust score for the $Kappa$ value is more than 0.8 [51]. The higher the value of $Kappa$ over 0.8 the more agreement there is, hence the robustness. In case $Kappa$ = 1 then there is a perfect agreement and a fully robust model. Accuracy by itself cannot prove that an approach is robust because there is always an active risk that the high accuracy is random.

For this reason,

Table 5. Highest effectiveness per dataset.

Domain	Classes	Avg. Accuracy	K-Value	Dataset	Work
Frequency domain	2	95.89%	0.9244	BCI III Dataset 1	[63]
Time-Frequency domain	2	99.35%	0.987	BCI III IVa	[90]
Time-Frequency domain	2	99.52%	0.95	BCI III IVb	[71]
Multi-domain	3	97.5%	0.978	BCI III V	[64]
Spatial domain	4	79.7%	0.73	BCI IV 2a	[45]
Spatial domain	2	96.5%	0.92	BCI IV 2b	[51]

shows the studies which achieved the highest accuracy and Kappa values during the research, using the BCI III 1/IVa/IVb/V and BCI IV 2a/2b datasets. As already mentioned above, the accuracy percentages and the Kappa-values constitute the mean values of the subjects or the time/frequency intervals that were separated in each work. Through the averages, the comparison of the methods in terms of their performance and robustness becomes more objective than presenting only the maximum values of a single subject or a single interval.

Figure 7 shows the feature extraction methods which achieved the most robust performance in each of the different datasets used in the research. The blue bars show the accuracy percentages (left axis) while the grey line shows the kappa values (right axis). The dots in the grey line are located among the accuracy percentage of each approach.

The Kappa values in [64] are represented by a graph for the BCI III IVb dataset and the values cannot be extracted with precision. For this reason, the above study is missing from Figure 7.

5. Discussion

As a first conclusion, it is noticed that the datasets used were based on non-invasive methods, except the dataset with the ECoG channels. The researchers working in the field of MI focus on the increase of the performance and the robustness of non-invasive techniques due to the ease of channel placement, the low cost, the safety they give to the individual and the more general convenience they offer in extracting information from the brain.

Table 2 shows that the majority of feature extraction methods are based on Common Spatial Patterns, including their variants. Without a doubt, CSP is the most common feature extraction technique in MI-BCIs and many variants have emerged in order to improve its performance. The spatial filtering of CSP has been designed in order to provide answers for the two-classes problems, but variants for multi-class problems have also been designed. In terms of robustness, Table 5 shows that one of the works, which achieved a high classification accuracy (96.4% and 96.5%), is based on CSP with a Kappa value equal to 0.92. This value shows the increased robustness of this approach. On the other hand, the literature shows that the CSP methods are not always robust, mainly due to the presence of noise [62,92]. Additionally, the CSP methods do not consider the location of the electrodes in spatial terms [93] and the increase of this type of method during 2019 led to the increase of spatial domain features. PSD methods, which cover the second largest percentage of the works studied (22.5%), also have high classification accuracy percentages and high Kappa values in EGoG based datasets or when it is combined with deep learning methods. In general, CSP and PSD methods cover 70% of the total works, but the accuracy they achieved and their K-values are not so high as the wavelets were used in the majority of each different dataset. Furthermore, WT also achieved the most robust performance among all works included in the present review, based on Table 3. Based on Table 5, which includes the most effective approaches on each dataset, three out of the six highest effectiveness approaches, based on Figure 7, belong to Wavelets Transform (MEWT, CWT, SDI). At this point, it should be noted that CWT is used for the presentation of a one dimensional (1-D) EEG signal, to two-dimensional (2-D) images and then the feature extraction made in the DCNN based AlexNet model. The SDI approach to feature extraction [94] based on multi-domain EEG signals (time, frequency, time-frequency, and spatial), which was motivated by DWT [64], achieved high scores in accuracy and Kappa value terms for 2-class and 3-class problems.

As Figure 7 shows, the majority of the most effective approaches which achieved high accuracy and high Kappa values are combined with wavelets methods and deep learning. Comparing the approaches which achieved the highest scores in each dataset, WT methods were more effective on datasets based on EEG channels, while CSP methods on datasets combined bipolar EEG and EOG channels. PSD approaches achieved high scores in accuracy and Kappa value on the ECoG channels-based dataset. Based on the classes of each dataset, WT and CSP are capable of high robust accuracy rates in 3-class and 4-class, respectively.

From a complexity perspective for the most effective approaches, ref. [64] used the SDI features approach, which is based on DWT, combined with FFNN. In [71], FFNN was used, but MEWT was used as the feature extraction method. Both approaches combine variants of Wavelet Transform with FFNN. The authors of [64] mention that the SDI approach is less complex than the MEWT approach and also has lower computational cost. The average training time of a single trial was 1.27 ms for the SDI approach, while in the MEWT was 22.2 ms. The highest feature extraction time for each subject for SDI and MEWT approaches was 1.36 and 1.68 s, respectively. Since both approaches have the same authors, the hardware specifications were the same. More specifically, M-5Y10c CPU 0.8GHz CPU is used with 8GB of RAM. In [90], the method followed was the representation of the signals in images through STFT and CWT. Again, this method uses a variant of Wavelet Transform in order for high accuracy and robustness to be achieved. Then the images are imported into the DCNN based AlexNet model in order for the feature extraction and the classification to be performed. The total training time was almost 41 min. In terms of real-time implementation, the hardware requirements will be higher since deep learning methods are computational power and time-consuming. On the other hand CSP approaches, such as [45,51], are not very effective on real-time applications [71]. Additionally, ref. [63] used the ECoG dataset which means that the complexity is lower since in these semi-invasive techniques the noise is reduced.

6. Conclusions

This review explored the robust feature extraction methods in the field of Motor Imagery-based Brain Computer Interfaces, based on the published works of the last five years. The research showed that there was an increase in CSP-based methods in 2019 but 87.5% of the WT-based approaches were published from 2019 onward. This fact shows that the evolution trend of CSP methods is replaced gradually by Wavelet Transform. In order to develop a robust feature extraction method, researchers focused on discriminative features. More specifically, spatial and time-frequency domains were the reference point for the majority of works. The most used datasets were variants from BCI Competition III and V for non-invasive channels. Determining whether an approach is effectively robust was based on its accuracy rate and Kappa value. The approaches with the highest accuracy percentages and also the highest Kappa values were those with combined Wavelet Transform or its variants, and deep learning methods in the majority of datasets. The Wavelet Transform methods constitute the majority of the approaches which achieved the highest robust accuracy among the works which are included in the present review.

The research led to the conclusion that WT achieved a higher robust accuracy than the most common feature extraction methods, CSP and PSD, for three out of the six datasets used. PSD methods achieved a high performance for a 2-class problem in which the signals were from semi-invasive techniques (ECoG). Additionally, WT methods have high effectiveness on 2-class and 3-class problems on a dataset whose information was acquired from 118 EEG channels. Furthermore, CSP methods are robustly effective in the 4-class problem and the 2-class problem, as well as in bipolar EEG and EOG channels.

Based on the results, the conclusion that emerges is that the most robust technique differs depending on the number of classes of each problem and the way the signals were acquired. CSP methods achieved high robust accuracy in datasets in which EOG channels and bipolar EEG were used and, along with WT, they are very effective for 3-class and 4-class problems, respectively. WT methods are also robust and effective in 2-class problems in datasets in which the information is acquired by EEG channels exclusively. In addition, PSD has high robust accuracy on semi-invasive-based datasets.

With the rise of deep learning in artificial intelligence and also in Motor Imagery, the number of works on BCIs will be increased in the near future. As mentioned above, high accuracy percentages and robustness are based on deep learning approaches and wavelets. Interesting future research might be the implementation of a robustness deep learning-based feature extraction method in online applications in order to study what the deviations will be in terms of accuracy and robustness. Besides the robust feature extraction approach, this research will also include the hardware requirements since the deep learning methods face challenges in online applications as far as the hardware is concerned. It is also very important for the hardware requirements and also for the deep learning approach to take into account the time in which the Motor Imagery system will respond in order to work properly in real-time implementations.