Particle Swarm Optimisation for Emotion Recognition Systems: A Decade Review of the Literature

Abstract

Particle Swarm Optimisation (PSO) is a popular technique in the field of Swarm Intelligence (SI) that focuses on optimisation. Researchers have explored multiple algorithms and applications of PSO, including exciting new technologies, such as Emotion Recognition Systems (ERS), which enable computers or machines to understand human emotions. This paper aims to review previous studies related to PSO findings for ERS and identify modalities that can be used to achieve better results through PSO. To achieve a comprehensive understanding of previous studies, this paper will adopt a Systematic Literature Review (SLR) process to filter related studies and examine papers that contribute to the field of PSO in ERS. The paper’s primary objective is to provide better insights into previous studies on PSO algorithms and techniques, which can help future researchers develop more accurate and sustainable ERS technologies. By analysing previous studies over the past decade, the paper aims to identify gaps and limitations in the current research and suggest potential areas for future research. Overall, this paper’s contribution is twofold: first, it provides an overview of the use of PSO in ERS and its potential applications. Second, it offers insights into the contributions and limitations of previous studies and suggests avenues for future research. This can lead to the development of more effective and sustainable ERS technologies, with potential applications in a wide range of fields, including healthcare, gaming, and customer service.

1. Introduction

Artificial Intelligence (AI) is a system that can perceive its environment and take actions to maximise its chances of success [1]. It has become an essential tool for addressing real-world problems inspired by human-logical thinking. One approach to achieving AI is through the SI field, a subfield of AI that has gained increasing attention due to its ability to solve high-complexity problems within a specific time frame [2]. The SI concept was inspired by natural biological systems that adopt the collective behaviour of an organised group [2]. SI is inspired by natural biological systems that use the collective behaviour of an organised group to solve complex problems [3]. Swarms are vast numbers of simple homogenous agents that interact with their environment, exhibiting decentralised control of global behaviour [2]. Swarm-based techniques use nature-inspired algorithms to produce fast, robust, and cost-effective solutions to complex problems [3]. Social swarms in nature, such as honeybees, bird flocks, and ant colonies, exhibit collective behaviour that can be modelled and adapted to solve complex problems [2,3].

Inside the SI field are examples of algorithms, such as cuckoo search, flower pollination algorithm and particle swarm, optimisation that have been included to achieve SI results [2]. Such an example from a previous study for SI is the applications of SI towards air overpressure (AOp) [4] from blasting and causing damage to nearby civilians. Therefore, to predict AOp accurately, ref. [4] used the artificial neural network (ANN), and the prediction was trained using PSO to predict the AOp accuracy [4]. PSO is a swarm-based intelligence that exploits the concept of social behaviour algorithm with a potential solution to a given problem viewed as a particle similar to a flock of birds [3,5,6]. Each particle then combines its historical best locations and gradually approaches the objective function optimum solution [3]. The above-mentioned advantages show that PSO is a promising development for optimising solutions to real-world problems [7].

PSO, in computing terms, is also defined as a computational procedure to select the most effective element and optimised it based on the collection of accessible alternatives [8]. Either with increment or minimisation, it is the true operation of the simplest state while constantly screening the input elements associated with an allowed set of accessible alternatives [8]. In the field of structural engineering, PSO is one of the evolutionary computation (EC) techniques used by many researchers [9]. PSO was proposed by [5] in 1995, and it contains memory that allows each member of the swarm to remember and acquire while moving through the searching space [9]. PSO is inspired by social problem-solving and seeks to perform a parallel search for the optimal solution with many individual searches associated with particles and collaborative influencing by joining the best performance of all searches [10]. According to [10], PSO is considered to be a good choice for practical solutions to optimisation problems when the problem is high-dimensional, defined by multiple criteria and potentially conflicting constraints or complex combinatorial nature [3,11] and the results achieved are based on the particles moving as a swarm in the space of the defined problem searching for the best solution, with the likelihood of finding an optimal or near-optimal solution being high.

PSO is a versatile algorithm that has been applied to a wide range of optimisation problems, including power system management [10] and indoor comfort and energy consumption optimisation [11]. PSO is especially useful in practical solutions to optimisation problems that are high-dimensional, defined by multiple criteria and potentially conflicting constraints, or of complex combinatorial nature. For example, in the study by [11], an artificial bee colony algorithm was used in conjunction with PSO to optimise the control of electrical appliances to minimise operational costs, reduce energy consumption, and maximise comfort levels in indoor spaces. In general, it is considered to be a good choice for practical solutions to optimisation problems when the problem is high-dimensional, is defined by multiple criteria and potentially conflicting constraints, or is of complex combinatorial nature. The PSO algorithm has demonstrated efficiency in various optimization-related applications, and recent technological advancements have the potential to enhance its features by integrating PSO. One such example is the ERS. ERS is a sub-part of artificial intelligence that enables a system to learn and recognise human emotions through different data modalities.

Affective computing (AC) is a related field that focuses on developing computers that can understand human behaviour, as introduced by [12]. Over the last decade, researchers have identified the subpart of AI that can be enhanced further using the capabilities of AI machine learning. One of the main highlights that have been suggested is machines and computers identify or learn human emotions [13,14] to improve the interaction between humans and computers and increase potential future applications. Supported by the field of AC that was introduced by [12], in recent years, researchers identified specific applications, such as the ERS. ERS is an emerging technology that has gained attention and has become an important area of study due to the progress of technology. Commonly, ERS is an embedded technology based on AI that enables machines to recognise human emotions [15]. The ability of machines to identify human emotions has been a focus of researchers, and the development of ERS has resulted from this body of work. ERS is still in the developing stage, and researchers and engineers are working on producing more modalities to enable more real-world applications [16]. ERS has the potential to be a significant technology that enhances AI further in the real world. Indeed, ERS has the potential to revolutionise various sectors and bring about significant advancements, namely, towards manufacturing, healthcare, and education. A previous study suggested that the application of ERS across various industries is possible, given the innovations seen thus far [13]. In healthcare, ERS can be used to detect and/or monitor patients’ emotional states, manage mental health conditions, and provide personalised care. For example, ERS can be used to detect depression symptoms in patients and alert healthcare providers to intervene and provide timely care [17]. In the automotive industry, ERS can be utilised to improve driving safety by detecting and responding to the emotional state of drivers, such as detecting drowsiness or stress and alerting the driver to take necessary actions [18]. In the education sector, ERS can be used to monitor students’ emotional states and provide personalised feedback, tailored learning materials, and interventions to improve learning outcomes [19].

Overall, ERS has the potential to bring about significant improvements and advancements in various sectors, making it an important field of study for researchers and technologists. ERS using PSO involves the use of PSO as an optimisation method to enhance the performance of ERS in recognising human emotions. This approach aims to improve the accuracy and efficiency of ERS by optimising the selection and combination of modalities, feature extraction, and classification algorithms. Hence, an SLR is needed to find a better understanding of ERS using PSO.

Key Contributions

The major contributions of this paper are summarised below:

• The paper focuses on the PSO approach for ERS, highlighting its significance in metaheuristic algorithms and its potential for achieving improved results in ERS. This contribution enhances the understanding of the benefits of PSO, specifically in the context of ERS;
• The methodology employed in this study is SLR, which offers valuable insights into the field of study by narrowing down the focus to specific areas of interest. SLR has been widely used in previous studies to analyse research trends, identify expert opinions, and review the relevant literature. By utilising SLR, this paper provides a comprehensive review that enriches the understanding of the field;
• The findings of this study offer valuable insights into the development of ERS through the utilisation of the PSO algorithm. Specifically, the study discusses the successful accuracies achieved by the PSO algorithm in ERS and suggests future research directions that should be explored by experts in the fields of SI, AI, PSO, and ERS.

The paper is structured as follows: Section 2 examines earlier works on PSO, ERS, and SLR. This is a general literature review on the key topics. Section 3 presents the methodology employed in this study. Section 4 presents the findings and results based on the chosen methodology. Section 5 offers a discussion of the findings and results presented in Section 4. Section 6 addresses the limitations identified in previous studies. Finally, Section 7 provides the conclusion of the paper.

2. Related Works

This section will discuss the observed recent literature that is related to PSO and ERS. Furthermore, this section will include the SLR background works from previous researchers. This section is a general literature review on the key topics not limited to works identified from the SLR on PSO for ERS. The SLR on PSO for ERS is presented in the subsequent sections.

2.1. Particle Swarm Optimization

As mentioned earlier in the paper, PSO was created by [5] and used the technique that has been inspired by natural bird flocking, fish swarms, and bee colonies to optimise decision-making by nature [5] and transform into algorithms that enable real-life metaheuristic algorithms to find optimisation [20]. Since then, the PSO algorithm has gone through a lot of changes, and there were many extended versions have been proposed [21]. In recent years, various dynamic and adaptive variations based on the objectives of optimising decision-making have been forwarded [21]. With all the adaptations, variations, and innovations, some versions can be described as the ‘standard’ due to the superior outcomes of optimisation in the real-world [21].

PSO is likened to the simulation of bird behaviour in the algorithm [20]. This model illustrates how a swarm of birds fly around in search of food and shelter, and this behaviour is incorporated into the PSO algorithm. The movement of particles in the algorithm, analogous to the flock of birds, involves sharing the best position to efficiently identify optimal points for decision-making [3,5,21]. In a recent study conducted by [20], the researchers examined the behaviour of social animals using the artificial life theory to construct artificial life systems with cooperative behaviour through computer simulations. The study highlighted the importance of understanding five fundamental principles in this context. Those five principles were:

• Proximity: The swarm will carry out simple space and time computations;
• Quality: The swarm sense the quality change in the environment and responds to the changes;
• Diverse Response: The swarm should not limit its way to get the resources in a narrow scope;
• Stability: The swarm does not change its behaviour with every environmental change;
• Adaptability: The swarm can change its behaviour when the change is worthwhile.

In PSO, particles have the ability to update their positions in response to changes in the environment [20]. As the particles move and rotate in space, they aim to approach the optimal value by adjusting their positions relative to the best position [3]. Additionally, ref. [21] conducted experiments involving a small number of global optimisations based on human problems. These optimisations were conducted using different topologies, which refer to methods of organising particles to facilitate the flow of information within the population. This arrangement allows individual members of the population to be influenced by those that have achieved the best performance thus far [21]. Although PSO and many metaheuristic algorithms have been proposed in recent years, there are many terms to solve optimisation problems that can be identified further. As one example, the Pendulum Search Algorithm (PSA) [22] is a population-based algorithm of an optimisation problem; however, the findings of the study highlighted that PSA is outperforming PSO [22]. The concept of PSA is a physical phenomenon mimicking the harmonic motion of a pendulum to move the search agents for the optimal solution [22]. As suggested by [22], the application in the real-world PSA can be applied to the vaccine distribution optimisation problem.

Over the last few years, PSO has attracted the interest of many researchers and researchers addressed many inspired nature-based algorithms rather than just flocking of birds, bee colonies, ant colonies, and fish swarms. Metaheuristic approaches, and intelligent algorithms have been identified to provide the maximisation of gain and the minimisation of loss [11,22]. In a study by [23], a dynamic multi-swarm (DMS-PSO) based system has been proposed to select the most suitable attributes and assist the diagnosis of heart diseases in medical analysis. This study shows that a combination of fuzzy logic and DMS-PSO can offer more effective systems of medical diagnosis with improved system accuracy. From the experimental analysis, DMS-PSO indicates a relatively higher performance when compared with existing systems in the healthcare industries and manual diagnosis, and it is expected that in real-life applications, DMS-PSO can achieve more reliable results. Furthermore, ref. [24] showed an improved scheme of particle swarm algorithm (PSO) and Newtonian motion laws, labelled as centripetal accelerated particle swarm optimisation (CAPSO), has been proposed to accelerate the learning and convergence procedure of classifiers. The obtained results of the PSO-CAPSO identified nine medical disease diagnosis benchmarks indicating that the proposed method provides better results in terms of good convergence rate and classification accuracy.

In the PSO algorithm, each particle functions as an independent agent that explores the problem space based on its own experience and interactions with other particles, resembling the natural behaviour of animals [20]. Therefore, it is crucial to understand the suitability of the algorithm for specific applications. The PSO algorithm offers several advantages, including robustness, adaptability to different application environments, and strong distributed abilities that enable it to quickly converge to optimal values [20]. In recent years, advancements in computational algorithms have further enhanced the speed, quality, and robustness of the PSO algorithm. As a result, there are various potential applications that can benefit from utilising PSO. Findings on the previous studies [23,24,25], combining PSO with several other existing methods provides positive results. There are many applications in the real world, and the existing PSO may solve complex problems and enhance features in the SI and AI field. Related to the development of ERS, ERS can be one of the future-proof technology, and the identified modalities for ERS can benefit from the embedded PSO. Therefore, this study will look extensively at the findings of PSO in enabling ERS.

2.2. Emotion Recognition System

Over the last decade, researchers have identified the subpart of AI that can be enhanced further using the capabilities of machine learning. One of the main highlights that have been suggested is machines and computers identify or learn human emotions [26] to improve the interaction between human–computer and increase potential future applications for human–computer. Furthermore, for machines and computers to learn about human emotions, it needs a system that enables the required process to learn human emotion [13] and is specified as an emotion recognition system (ERS). ERS enables a system that accepts various modalities of data, learns, and allows the machine to recognise the emotion of human subjects.

Previous researchers specified ERS as originally in the field of Affective Computing (AC), a field that was brought forward by [12]. According to [12], AC is an ability of a computer that relates to, arises from, and understands human behaviour [12]. With that being said, AC led to further findings of emotion recognition, and it has been suggested by the previous researcher that AI learning can signifies for further findings of ERS as embedded technology in various applications [17]. Moreover, ERS has emerged as one of the attractive areas in the field of AC and AI with the promises of what the technology can achieve [26], such as developing robots that can interact and communicate with humans, equipped with functions for both understanding human emotions and expressing human emotions [26].

Outcomes and expectations from the previous studies show that ERS can be an important technology in the making based on the advantages offered to individuals, society, organisations, businesses, and industry via various platforms and applications. For example, ERS in healthcare [17], ERS in driving assistance in the car [18], ERS in the classroom [19] and ERS in smartwatches as the latest addition due to the modalities of smartwatches being the same ERS modalities. In existing research on ERS, researchers adopted AI algorithms, such as convolutional neural network (CNN) and deep neural network (DNN) for emotion recognition based on various data modalities, such as facial expression, voice intonation, heart signals (ECG), brain signal (EEG), and many others. It is important to look into the relationship between ERS and its significance in society and industry, specifically in the industrial revolution [26].

One of the main highlights that benefit from ERS is in healthcare industries [17], developed the modalities of ERS and suggested their applications in healthcare. Researchers used facial recognition combined with ECG to identify the six basic emotions [27]. Furthermore, ERS has been suggested to provide better health services quality assurance by facilitating decision-making throughout the COVID-19 pandemic by allowing safe monitoring and higher emotional awareness among the practitioners [26,27]. Furthermore, according to [28], social robots have risen to facilitate limitations due to COVID restrictions. Social robots work and interact with humans; therefore, features of emotion recognition technology might enhance the interaction between social robots and humans. From a marketing perspective from retail industries, ERS helped in advertising products. One of the main examples is text sentiment analysis. Ref. [29] suggested that text mining provides useful data in identifying emotional awareness through text. Text mining is a learning-based algorithm to describe characteristics of text, such as word expression based on human sentiments and emotions [29].

Previous research has focused extensively on the modalities and potential applications of ERS, but there has been limited study on the readiness of the technology for adoption. A recent paper by [30] examined the readiness of ERS from the perspective of university students to determine whether university students are aware of recent developments in ERS and whether national infrastructure should encourage the use of ERS. In a few years, ERS can be one of the significant technology that will be beneficial to industries and societies. Therefore, to contribute to the ERS development and benefits to the ERS practitioners, this study will be looking into one of the factors that enhance ERS modalities which is PSO.

2.3. Systematic Literature Review

SLR is a methodical and structured approach to identify, evaluate, and synthesise all relevant literature related to a specific research question or topic [31,32]. It provides a comprehensive overview of the current state of knowledge on a particular subject, helps identify research gaps, and informs future research directions. SLR is a valuable tool for researchers as it helps them avoid bias and ensures that all relevant studies are considered, leading to more accurate and reliable research outcomes. A previous study that adopted SLR [33,34,35] has shown significant results in identifying recent related studies and findings that help the researcher investigate the depth of the chosen field.

Specifically, this study aims to evaluate the utility of PSO in facilitating ERS development. To gain deeper insights into previous research, the methodology of SLR has been selected to determine the potential benefits of PSO for ERS practitioners. In a prior study conducted by [36], works on PSO were reviewed extensively using the SLR methodology. Specifically, the study utilised SLR focusing on PSO’s applicability in medical disease detection. The review explored works highlighting the feasibility of PSO in the healthcare industry and demonstrated its potential for disease classification, including conditions, such as heart disease. Furthermore, additional findings from [36] obtained through the SLR indicate that PSO techniques can be employed for the detection of liver disease, cancer, brain disease, and diabetes.

Other related works of PSO using SLR from [3] reviewed PSO and its potential applications based on the SLR method. Ref. [3] conducted an SLR on the potential applications of PSO algorithms. The review categorised PSO applications into healthcare, environmental, industrial, commercial, smart city, and general aspects. Technical characteristics of different PSO methods and applications were discussed in terms of accuracy and evaluation environment. The paper also proposed a case study to investigate the effectiveness of different PSO methods and applications. On the other hand, ref. [37] conducted an SLR of over two decades of PSO-related work. The study aimed to establish a general understanding of the trend of PSO and its related study to the applications, algorithms, and results from previous studies. The trend of publications and studies related to PSO for the twenty years was analysed in the paper.

The previous SLR conducted by [3] will greatly contribute to this study by identifying relevant papers on PSO, including algorithms and related terms. The findings presented by [3] establish a solid foundation for understanding the potential applications of PSO across various industries and highlight its significance. In recent years, there has been increased attention on ERS, with various modalities being explored, including physiological modalities (e.g., EEG and ECG) and psychological modalities (e.g., facial expression and speech recognition) [26]. Previous research has also identified potential applications of ERS in industries, such as healthcare [17], education [19], and entertainment [16,38]. PSO can serve as a valuable tool for engineers, scientists, and researchers in developing and understanding the PSO methodology for creating the necessary modalities for emotion recognition. Therefore, employing SLR as the chosen methodology in this study will provide valuable insights into the utilisation of PSO in ERS, particularly over the last decade, and the recent developments.

3. Methodology

In this study, the SLR methodology was chosen and is consistent with previous works [39,40,41,42]. SLR is considered an ideal method for obtaining a comprehensive summary of prevalent knowledge [41]. By using SLR, the researchers can identify and synthesise all relevant research that leads to the research gap, which helps to frame the research questions and objectives [41]. In 2009, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was introduced by an international group of systematic reviewers, methodologists, and journal editors. PRISMA is a guideline designed to assist authors in preparing a comprehensive report for SLR [43].

The guideline is divided into three stages, with the first being identification. This section highlights the necessary search elements, such as keywords or search strings, criteria, and databases used. For this study, the search strings included “Particle Swarm Optimization” and “Swarm Intelligence” combined with “Emotion Recognition System” using OR and AND Boolean operators [44]. The search was conducted using the Lens.org platform, which has been increasingly recognised by researchers for its benefits [44,45]. The platform was initially launched in 2000 by Cambia as Patent Lens, a non-profit organisation based in Australia. Since 2013, it has been known as The Lens and has expanded to become more than just a patent search platform. The search on the Lens.org platform yielded 86 publications within the date range of 2012–2022 (as shown in Figure 1). There were no duplicates identified, and all 86 publications were included in the screening process.

The next stage in the systematic literature review process is screening. Screening involves three stages: identification, retrieval, and eligibility assessment. In the first stage, 86 publications were screened for identification, and any papers that did not meet the study’s criteria or were written in a different language were checked. Since most of the papers were eligible for retrieval, no papers were excluded. The next stage of screening is retrieval, where papers are accessed in full text. However, due to financial restrictions, nine papers were unable to be accessed, leaving 77 reports available for the next stage. In the final stage of screening, papers were assessed for eligibility for the study. This involved removing papers that were not directly related to the scope specified for this study, such as general papers on PSO that were not directed towards ERS. From the initial 77 reports, 17 non-journal articles, and conference proceedings were excluded, and two papers that were not related to PSO and ERS.

The final stage involves selecting the reports or papers that meet the inclusion criteria and will be included in the review and study. The selected papers must contain relevant information, meet the criteria, and be within the scope of the study. In this case, a total of 58 reports were found to be eligible for inclusion in the review and study. The results obtained from the Lens.org search will be discussed in the following section.

4. Results

This section reports the descriptive analysis results of the included 58 records, focusing on the document type, years of publications, subject area, keywords related to the studies, and most cited related to the studies.

4.1. Document Type

The data that has been collected has been classified based on two categories journal articles and conference proceedings articles. These types of research works are selected for the quality control process normally in place before publication. Specifically, the works go through a stringent review process by subject matter experts to verify the quality of the work. Moreover, the journal article and conference proceedings article were chosen due to journal has a better structure while the conference is short and precise that has been used in the conference [45].

The data presented in

Table 1. Document Type.

Document Type	Number of Publications	% (N = 58)
Journal Article	55	94.83%
Conference Proceedings Article	2	3.45%
Conference Proceedings	1	1.72%
TOTAL	58	100

indicates that the majority of the publications retrieved from the search are journal articles, accounting for 94.83% (55 publications) of the total. The remaining publications are conference proceedings (3.45%, 2 publications) and conference proceedings articles (1.72%, 1 publication). The high percentage of journal articles suggests that these publications are well-constructed and have undergone rigorous peer review before being published, indicating their potential to contribute significantly to the study of PSO and ERS. Therefore, this paper will focus primarily on journal articles to provide a more comprehensive discussion of the topics at hand.

4.2. Year of Publications

This study conducted a systematic review of works published between 2012 and 2022, and the results indicate an overall increasing trend in the number of publications over the decade, with a peak in 2020 (as shown in Figure 2).

The trend observed in Figure 2 shows that there was an initial increase in the number of returned publications from 2012 to 2013, but the number remained stagnant between 2013 and 2016. The trend then picked up again from 2017 to 2020, which can be attributed to recent advancements in technology that have led to the development of new potential applications for PSO and ERS. The year 2019 saw the highest number of returned publications, with a total of 13. However, since 2020, the number of publications has decreased, likely due to a shift in focus from developing ideas and concepts to implementing them in real-world applications. This can be seen from the trend for ERS patent documents overtime over the same period (see Figure 2). Overall, these trends suggest that ERS, powered by PSO, is an emerging technology with increasing potential for day-to-day applications, driven by advancements in technology and the digital revolution globally.

4.3. Field of Research

The collected records were analysed to determine the field of research or subject matter, as shown in

Table 2. Subject Matter.

Field	Frequency	% (n = 135)
Computer Science	45	33.33%
Artificial Intelligence	31	22.96%
Machine Learning	12	8.89%
Pattern Recognition	10	7.41%
Feature Extraction	8	5.93%
Feature Selection	7	5.20%
Algorithm	6	4.44%
Psychology	5	3.70%
Engineering	4	2.96%
Human–computer Interaction	4	2.96%
Biometrics	3	2.22%
TOTAL		100%

. The analysis included journal articles and conference proceeding articles and aimed to identify the main research fields for works on PSO and ERS, and the number of articles according to each field of research. It is important to note that commonly a single article is often tagged to more than one research field. Computer science was found to be the highest-ranking area of study, with 45 papers or 33.33% of the total frequency. Artificial intelligence was ranked second highest, with 31 papers or 22.96%. This means that for the majority of the 58 papers included in the review, the area of research was computer science. Furthermore, it is worth noting that all the fields of research generated from the analytics on Lens.org are related, such as machine learning, feature extraction and selection, algorithms, and human–computer interactions, which can be considered part of artificial intelligence. Therefore, all the papers remain in the same context of the field of research, with researchers focusing on developing the PSO model and developing ERS modalities with applications that benefit ERS from the PSO model.

4.4. Most Cited

Next, the retrieved records that were included in the studies were analysed in terms of citation. According to the analytics from Lens.org, the 58 publications collectively garnered 1858 scholarly citations. Table 3 shows the top 10 most cited works in the collection. Ranked 1st, the paper by [46] with 234 citations. The paper proposed using PSO-based using the concept of micro genetic algorithm (Micro-GA) to identify emotion recognition based on facial expression since the facial expression is also one of the modalities for ERS.

From Table 3, it can be observed that PSO has been recognised as part of machine intelligence that can further develop ERS. Most of the paper from Table 3 shows the relationship between PSO and main ERS modalities, such as the ECG and EEG. PSO remain the main concept, such example, a paper by [47] used EEG in deep learning extraction, such as feature extraction and feature selection from PSO, to categorise EEG. Furthermore, ref. [48] discussed using PSO applications in ECG self-supervised to detect and decide states of emotions category.

Table 3. Top 10 Most Cited.

Authors	Title	Results	Challenges	Year	Cites
[46]	A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition	Integrated with an SVM-based ensemble, mGA-PSO has the best average accuracy within database evaluation and cross-domain evaluation.	Further improvement is needed using the firefly algorithm and cuckoo searches to equip the current algorithm to deal with real world.	2016	234
[49]	EEG-Based Brain–computer Interfaces Using Motor-Imagery: Techniques and Challenges	Feature selection and feature extraction techniques using PSO achieve 90.4% accuracy with strong directional search and population-based search.	EEG-based on MI were fraught with signal processing and needed further investigation techniques for the feature extraction and feature selection.	2019	211
[50]	Diagnostic Classification of Intrinsic Functional Connectivity Highlights Somatosensory, Default Mode, and Visual Regions in Autism	Accuracy remained modest overall for PSO-SVM. It is applicable when there are more features than observation and applicable for both binary and multi-category tasks classification.	In a real-world situation, machine learning can be performed at a limited level permitted, and the PSO-SVM used required external validation datasets.	2015	125
[47]	Classification of Arrhythmia by Using Deep Learning with 2-D ECG Spectral Image Representation	The 2-D images classify eight kinds of arrhythmia and achieved 97.91% sensitivity, 99.61% specificity, 99.111% accuracy, and 98.59% precision value.	The study uses only a single lead ECG, and the effect of multiple lead ECGs is needed.	2020	91
[51]	Support Vector Machines to Detect Physiological Patterns for EEG and EMG-Based Human–computer Interaction: A Review	SVM is one of the most versatile classifiers for EEG and is particularly suitable for online human–computer interaction (HCI).	There were limited searches and results for electrophysiological signal-based to enhance EEG and SVM.	2017	76
[52]	ECG Beat Classification Using A Cost-Sensitive Classifier	A cost-sensitive for SVM proposed to ECG as a modified classifier. The error rate of 2.8% with no rejection and less than 1.2% for minimal classifier cost.	Model classifiers are limited and needed for more optimised sensitivity in the classification-cost.	2013	67
[53]	Filtering Techniques for Channel Selection in Motor Imagery EEG Applications: A Survey	Summarised the filtering techniques applied using MI-EEG in real-world applications.	Various parameters, time, complexity, and accuracy are needed for further investigation of MI-EEG.	2019	65
[54]	Cooperative Social Robots to Accompany Groups of People	The prediction Anticipation Model (PAM) is used and able to cooperate in real-life situations and identify normal human behaviour.	The PAM model intends to work with various groups of people within the lab, and there is needed for a particle filter sampling process for the social force model.	2012	64
[48]	Self-Supervised ECG Representation Learning for Emotion Recognition	The ECG-based proposed approach improves classification performances compared to fully supervised solutions.	The proposed approach of ECG-based may perform poorly in subject-independent emotion recognition.	2022	59
[55]	Pattern Mining Approaches Used in Sensor-Based Biometric Recognition: A Review	Pattern-mining approaches challenge the biometric issues the state-of-the-art relies on the precise sensors type and application domain.	The validation data set provides an unbiased evaluation of a model adapted to the training data set during the tuning of model hyperparameters. The evaluation becomes much more biased because the validation data set is included in the configuration of the model.	2019	52

Table 3. Top 10 Most Cited.

Authors	Title	Results	Challenges	Year	Cites
[46]	A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition	Integrated with an SVM-based ensemble, mGA-PSO has the best average accuracy within database evaluation and cross-domain evaluation.	Further improvement is needed using the firefly algorithm and cuckoo searches to equip the current algorithm to deal with real world.	2016	234
[49]	EEG-Based Brain–computer Interfaces Using Motor-Imagery: Techniques and Challenges	Feature selection and feature extraction techniques using PSO achieve 90.4% accuracy with strong directional search and population-based search.	EEG-based on MI were fraught with signal processing and needed further investigation techniques for the feature extraction and feature selection.	2019	211
[50]	Diagnostic Classification of Intrinsic Functional Connectivity Highlights Somatosensory, Default Mode, and Visual Regions in Autism	Accuracy remained modest overall for PSO-SVM. It is applicable when there are more features than observation and applicable for both binary and multi-category tasks classification.	In a real-world situation, machine learning can be performed at a limited level permitted, and the PSO-SVM used required external validation datasets.	2015	125
[47]	Classification of Arrhythmia by Using Deep Learning with 2-D ECG Spectral Image Representation	The 2-D images classify eight kinds of arrhythmia and achieved 97.91% sensitivity, 99.61% specificity, 99.111% accuracy, and 98.59% precision value.	The study uses only a single lead ECG, and the effect of multiple lead ECGs is needed.	2020	91
[51]	Support Vector Machines to Detect Physiological Patterns for EEG and EMG-Based Human–computer Interaction: A Review	SVM is one of the most versatile classifiers for EEG and is particularly suitable for online human–computer interaction (HCI).	There were limited searches and results for electrophysiological signal-based to enhance EEG and SVM.	2017	76
[52]	ECG Beat Classification Using A Cost-Sensitive Classifier	A cost-sensitive for SVM proposed to ECG as a modified classifier. The error rate of 2.8% with no rejection and less than 1.2% for minimal classifier cost.	Model classifiers are limited and needed for more optimised sensitivity in the classification-cost.	2013	67
[53]	Filtering Techniques for Channel Selection in Motor Imagery EEG Applications: A Survey	Summarised the filtering techniques applied using MI-EEG in real-world applications.	Various parameters, time, complexity, and accuracy are needed for further investigation of MI-EEG.	2019	65
[54]	Cooperative Social Robots to Accompany Groups of People	The prediction Anticipation Model (PAM) is used and able to cooperate in real-life situations and identify normal human behaviour.	The PAM model intends to work with various groups of people within the lab, and there is needed for a particle filter sampling process for the social force model.	2012	64
[48]	Self-Supervised ECG Representation Learning for Emotion Recognition	The ECG-based proposed approach improves classification performances compared to fully supervised solutions.	The proposed approach of ECG-based may perform poorly in subject-independent emotion recognition.	2022	59
[55]	Pattern Mining Approaches Used in Sensor-Based Biometric Recognition: A Review	Pattern-mining approaches challenge the biometric issues the state-of-the-art relies on the precise sensors type and application domain.	The validation data set provides an unbiased evaluation of a model adapted to the training data set during the tuning of model hyperparameters. The evaluation becomes much more biased because the validation data set is included in the configuration of the model.	2019	52

5. Discussion

Researchers in the past have demonstrated the potential of using PSO algorithms in feature selection for ERS (Table 3). Given that PSO is a widely accepted technique for optimising results, the 58 included previous studies will be discussed in this section to examine how the PSO technique has been applied to achieve optimisation for ERS. Previous studies have explored various modalities for identifying human emotions in ERS. These modalities can be categorised into physiological and psychological modalities, and data mining, which analyses sentiments based on text. Physiological modalities include internal human responses, such as EEG, ECG, and photoplethysmography (PPG), while psychological modalities require external human responses. Additionally, physical modalities, such as facial expression, speech expression, and body movement, are also used in ERS. The following sections explore the studies according to the different modalities, then on directions for future research.

5.1. Physical Modalities

Physical modalities are one of the modalities to enable ERS. Most commonly known from previous research related to physical modalities are facial expression and speech recognition. Facial expression is based on the human facial movement, eyebrows, eyes, mouth, and the movement of the face [19], while speech is based on the voice and tone of a person that represents emotions and sentiments [38]. In the study by [46], the authors proposed a facial expression recognition system using the PSO technique with the concept of a micro-genetic algorithm (MGA) to optimise the real-time recognition of facial expressions. Facial expression is considered one of the popular modalities for emotion recognition systems with potential applications [46]. The proposed system consisted of three steps: feature extraction, feature optimisation, and emotion recognition. The MGA-PSO algorithm was used to identify the most significant features of different facial expressions. This algorithm incorporated personal expression and maintained a secondary swarm with a small population size of five to host the swarm leader and four follower particles with the highest/lowest correlations with the leader from the non-replaceable memory to increase global search capabilities [46,56,57]. The authors suggested that further improvements, such as using the firefly algorithm and cuckoo search, could be explored, and the multi-objective swarm evolutionary algorithm could also be added to deal with real-world optimisation. Next, the study by [58] focused on feature extraction and classification for facial expressions. They suggested the use of Local Binary Pattern (LBP) as a technique from PSO for facial expression. LBP is successful due to its speed and discrimination performance, especially for low-resolution images. The proposed approach was an improvement over a previous study and used an LBP histogram with different block sizes of a face image as feature vectors. The facial expression was further classified using Principal Component Analysis (PCA). The results of the study showed that using LBP improved the average accuracy by 22% compared to not using LBP. However, a limitation of the study was its generalisation to other datasets.

Another related study in the field of facial expression recognition is the work by [59], which proposed a Micro-Facial Expression based on Optimal Convolutional Neural Network (MFEOCNN) algorithm. The method involves pre-processing the input using geometric features, Histogram Oriented Gradient (HOG) features, and LBP, then optimising the features using the Modified Lion Optimization (MLO) algorithm. The final step involves using a convolutional neural network (CNN) to recognise facial expressions. The performance of the MFEOCNN algorithm was evaluated in terms of false measure and accuracy, and the results demonstrated that the proposed method outperformed existing methods in terms of recognition accuracy. Other physical modalities that have been found from the SLR are through speech recognition analysis. Ref. [60] proposed the use of metaheuristic analysis based on Ant Colony Optimization (ACO) algorithms, which have been employed in some previous studies. The research results suggest that using ACO could reduce complexity in speech recognition. The ACO approach involves constructing a graph model, then predicting the degree of the generated graph to satisfy the target degree sequence. The proposed method of ACO in speech recognition has advantages over complete graphs in terms of reduced complexity and more flexibility in the search space compared to the binary-connected graph model.

5.2. Physiological Modalities

Physiological modalities offer another approach to enabling ERS. Among these modalities, ECG and EEG have been the most studied. Physiological modalities can involve the stimulation of blood pressure, heart rate, skin sensors, and brain activity. Previous studies have shown that ECG and EEG can help understand human emotional states [17,18,61,62,63]. A previous study has been conducted on brain–computer interaction (BCI) systems that allow disabled individuals to control their environment using robots and manipulators [53]. The study focused on the use of EEG signals generated through Motor Imagery (MI), a method that produces signals related to motor movements. The study also explored channel selection for MI classification using a common spatial pattern (CSP) as it maximises the variance difference between the two classes. CSP was solved using an optimisation process that minimised the cost function, defined as the variance of the projected class of one signal while keeping the sum of the variance of both classes fixed. However, it has been noted in a previous study that the generalisation of EEG decoding methods is difficult as optimal channel selection is highly dependent on the application, features, criteria, and classifiers used [53]. They suggested that a deep and extensive analysis of each technique is necessary to design the optimal channel selection algorithm.

Another study has investigated EEG and physiological signal-based brain–computer interfaces that utilise MI data, which could potentially revolutionise technology [49]. The paper examines the selection of signal processing techniques using MI-Based EEG, with a particular focus on feature extraction and selection in such systems. The study also emphasises the use of PSO as a metaheuristic method, which showed a mean classification accuracy of 90.4% with strong directional search and population-based search strategies for both exploration and exploitation. The authors suggest that these methods could have numerous biomedical applications, including the replacement and restoration of central nervous system (CNS) functionality lost due to injury or disease. Lastly, a study was conducted on the classification of ECG using a two-dimensional convolutional neural network (CNN) model [47]. The proposed model was compared with other recent techniques for the automatic classification of arrhythmia and achieved an accuracy of 90%. The study concluded that deep CNNs could enhance the accuracy of diagnosis algorithms in modern machine-learning technologies and that 2-D images can be used to classify eight types of arrhythmia.

5.3. Data Mining Modalities

Text mining is another modality for ERS, which involves analysing human emotions through text using data mining techniques. This method is commonly used in sentiment analysis [29]. One study found in the reviewed literature used PSO for feature selection in text mining to classify emotions [64]. The study extended previous research on feature selection classifiers to analyse text mining. The results showed that the impact of PSO on the accuracy of the classifiers was minimal when PSO feature selection was used. The study suggested that future research should focus on more datasets to better understand the impact of PSO and other variants of PSO on ERS through feature selection in data mining [64].

5.4. Future Directions

Over the past decade, computer-vision-based techniques for human activity recognition have made significant progress and led to mature applications [65]. PSO algorithms have been proposed as effective due to their convergence and ease of implementation, but specifying certain parameters is necessary to maintain the solutions’ quality, particularly in enabling ERS [65]. Previous research has suggested potential PSO applications in healthcare, industry, commerce, and smart cities. Still, future research is needed to explore trend applications, such as image registration with 3D images from computer-assisted tomography (CAT) and MRI, as most previous research used 2D images [3,47]. Additionally, PSO can be utilised to develop optimal recommendations based on algorithms, but it is challenging to apply this approach to real-life scenarios that lack explicit feedback data, particularly when dealing with human emotions, which are not easily interpreted. Previous researchers have suggested for future researchers explore the enhancement of PSO through Neuro-Fuzzy Network (NFN) for Colour Image Processing (CIP) as this is seen as having strong potential for various commercial, healthcare, and environmental applications [3].

PSO is also reported as a robust and fast optimiser that can solve complex real-world problems; thus, for better outcomes, it is recommended for more research works be completed on adapting PSO for ERS [7]. In future research related to PSO for ERS, there are several potential areas of exploration. First, diverse search strategies, such as the firefly algorithm and cuckoo search, can be investigated to enhance the diversity of the swarm for subdimensions exploration, as suggested by [46]. Second, multi-objective evolutionary algorithms can be explored to equip the current algorithm to handle complex optimisation problems containing multiple criteria. Moreover, research towards BCI technology should focus on designing dependable systems with stable performance, adaptable to a wide range of users with varying mental states and environmental conditions, as highlighted by [49]. To create viable ERS products that have general appeal, intuitiveness, usability, reliability, and cost-effectiveness, the standard requirements for enabling PSO on ERS should also be considered in future works [3]. A study by [66] proposed the potential for investigating real-time movie-induced emotions, specifically to classify between happy and unhappy emotions. Previous studies also highlighted the limitations of PSO due to the complexity and memory consumption needed to stabilise the outputs [67].

Finally, the paper from [68] suggests that the quality of solutions generated by PSO is dependent on its parameters, indicating that the PSO parameter settings may not be optimal for every problem. Furthermore, PSO on the modalities of ERS may be susceptible to local optima when dealing with complex objective functions. Although many papers focus on the fundamental design of PSO systems for ERS optimisation, follow-up research is needed to establish the systems’ applicability in real-world settings. In addition, the literature retrieved for this study is based on Lens.org, although there were more recent papers that need to be explored from PSO and ERS. This study focused on ERS in modalities and PSO as optimisation for the ERS modalities; however, a variety of PSO methods can achieve more PSO applications and enable ERS results better. This will enable ERS researchers and practitioners to create more effective and practical applications of ERS.

6. Limitations

PSO is one of the more important and influential swarm intelligence algorithms, as seen from the discussion above. However, it is also important to highlight what could be considered as the problem that came with its popularity, namely, there have been several researchers that criticise or caution on the issue of some metaheuristics presented as a novel but have been challenged and shown to be similar to existing ones. Another highlighted problem is where some works were identified with issues, such as having centre bias.

The researchers highlight the worrying trend of authors repackaging existing optimisation ideas with a new metaphor. The main problem is such that the only thing new is the metaphor itself. Old wine in a new bottle, or “salami slicing” or “redundant publication,” can be a problem in research. This practice can lead to the dissemination of incomplete or misleading information, waste of resources, and contribute to the literature with a high volume of low-quality papers.

Researchers should always ensure that each publication provides new insights, adds value to the research question, and does not duplicate previous publications. When building on previous work, researchers should acknowledge and cite the source of information. This practice demonstrates scholarly integrity and helps to avoid plagiarism. Furthermore, citing previous studies can provide a context for the current research, demonstrate the novelty of the research question or methodology, and acknowledge the contribution of earlier work to the field.

Specific to this manuscript, with a systematic literature search, the authors were not able to entirely control what came back from the search queries. In other words, the authors conducted the SLR on the finalised included set of 58 works that met the specified SLR criteria (explained earlier). The discussions were formed based on the insights gained from reviewing the 58 papers. However, beyond the set of reviewed works, some works discussed some of the algorithms reviewed. Particularly, a number of the algorithms reviewed in the discussion section earlier had been identified by some researchers to have issues, namely, Salp swarm, lion optimisation, whale optimisation, cuckoo search, firefly algorithms, and flower pollination, each had been criticised by some researchers as being problematic. Nevertheless, the so call problematic metaheuristics often are very highly cited. Thus, this paper discusses this aspect as a limitation of this SLR findings.

Different Metaphors Existing Techniques

Recent research has highlighted that there are still objectives of optimisation that metaheuristics have yet to prove clear [69,70]. Researchers need to be aware of their differences to differentiate between metaphors, algorithms, and implementation [70]. For example, the algorithm proposed for cuckoo search [71,72] differs from the implementation by the authors. While the cuckoo search was one of the solutions to continuous optimisation problems, the cuckoo behaviour discussed in the cuckoo nest metaphor did not align with the algorithm and implementation [72,73].

The authors stressed that although metaphor-based algorithms inspired by natural behaviour have been cited in much of the literature, it is still unclear whether they introduce any novelty apart from new metaphors [74,75,76]. The ideas proposed in previous algorithms are often the same as the previously proposed algorithm, with only the metaphors used in the algorithms being different [70]. Furthermore, the majority of studies lack scientific motivation, which is often justified by the fact that the novelty of the metaphor-based algorithm is a lack of scientific rigour testing and comparing the proposed algorithm with other methods [76].

In the field of metaheuristic algorithms, some methods used exact algorithms over the last decades but were found to be inadequate or computationally too prohibitive [76]. A paper by [75] discussed available algorithms proposed over the last decade, including ant-lion optimisation, artificial bee colony algorithm, bee’s algorithm, bat-inspired algorithm, bird-swarm algorithm, and cuckoo search. The centre bias is becoming one of the major problems for Evolutionary Computing (EC) and the SI field. The 90 methods identified by [76] show that 47 methods utilise centre bias. The metaphor method is also identified as a problem of the current trend in EC and SI, as some methods do not have a centre bias and are identically similar to other methods [69,76].

According to [70], a metaheuristic problem can be useful in problem-independent algorithms and specific algorithms that are built to their specifications. The authors suggested that there may be enough methods for optimisation algorithms, and there is no need to introduce new metaphors that lead to the same outcome [75,77]. Previous papers introducing new metaphors of algorithms create confusion in the literature and hinder the basic understanding of existing metaphors meta-heuristically [70,71].

A recent study by [77] explained the three most famous PSO algorithms and contained highly cited [77], consisting of The Grey Wolf Optimization (GWO), Firefly Algorithm (FA), and Bat Algorithms (BA). These three highly cited PSO algorithms were not truly novel as the concept and metaphors introduced are similarly the same and can be seen as variants of existing PSO algorithms. Previous studies [77] show that many of these algorithms were equivalent and differed minimally, which led to a limited novelty approach for PSO as these three highly cited methods were similar methods and outcomes [77]. Therefore, in the last few years, refs. [69,70,77] proposed that the novelty of metaphor-based algorithms in the literature is just a reiteration of already well-known algorithms, and the novelty is based on the explanation of the terminology of a particular algorithm.

7. Conclusions

Over the years, technologies have continued to evolve, and their impact has been felt by individuals, organisations, and industries that have had to adapt to these changes. However, for any technology to be effectively applied, there is a need for a deeper understanding of its potential applications. ERS is a key technology that can enhance AI and enable the industrial revolution in terms of HCI. Therefore, it is essential to further understand and develop ERS to ensure its broader applicability. ERS is a significant technological advancement that can recognise human emotions and conditional states, as pointed out by [18,78]. To ensure its effectiveness, various modalities of ERS need to be enabled. Previous research has identified these modalities and the potential applications of ERS in different sectors and industries. To further investigate the modalities of ERS, PSO has been applied by researchers to better find the optimal solutions for developing these modalities.

PSO, as mentioned by [3], there is still a field from PSO that need to be covered, such as example, combining or hybridising PSO with a few novel optimisation techniques, such as Salp Swarm Algorithm (SSA), Whale-Optimization Algorithm (WOM), and Lion-Optimization Algorithm (LOA). Moreover, PSO has limitations in addressing the cloud memory, which would affect the stabilisation of the particles [59]. However, a recent study has highlighted that PSO has been identified as a potential method for developing the modalities of ERS due to its ability to optimise results for achieving the best outcome [10]. Thus, it is hoped that with this SLR study, future researchers can identify the best-proposed methods from previous studies and further explore the modalities of ERS.

In addition, future research is needed for metaphors with a sound basis that can bring useful and novel perspectives to optimisation problems [69,70]. The novelty of a specific algorithm has become a limitation for PSO itself, and the lack of scientific rigour in testing and comparing the proposed algorithm with other methods is an issue that needs to be addressed [77]. Nevertheless, if a different metaphor allows for better appreciation and applications of existing techniques, could it not be another form of research contribution?

Lastly, this SLR study provides valuable insights not only into existing work in the field but also from experts in related areas, such as HCI, BCI, and computer science. This information can contribute to a better understanding of PSO and ERS for future researchers, engineers, and practitioners in the field. Overall, the results from this SLR study can provide insightful information on ERS modalities that can lead to the development of impactful solutions for society.