Automated Facial Emotion Recognition Using the Pelican Optimization Algorithm with a Deep Convolutional Neural Network

Abstract

Facial emotion recognition (FER) stands as a pivotal artificial intelligence (AI)-driven technology that exploits the capabilities of computer-vision techniques for decoding and comprehending emotional expressions displayed on human faces. With the use of machine-learning (ML) models, specifically deep neural networks (DNN), FER empowers the automatic detection and classification of a broad spectrum of emotions, encompassing surprise, happiness, sadness, anger, and more. Challenges in FER include handling variations in lighting, poses, and facial expressions, as well as ensuring that the model generalizes well to various emotions and populations. This study introduces an automated facial emotion recognition using the pelican optimization algorithm with a deep convolutional neural network (AFER-POADCNN) model. The primary objective of the AFER-POADCNN model lies in the automatic recognition and classification of facial emotions. To accomplish this, the AFER-POADCNN model exploits the median-filtering (MF) approach to remove the noise present in it. Furthermore, the capsule-network (CapsNet) approach can be applied to the feature-extraction process, allowing the model to capture intricate facial expressions and nuances. To optimize the CapsNet model’s performance, hyperparameter tuning is undertaken with the aid of the pelican optimization algorithm (POA). This ensures that the model is finely tuned to detect a wide array of emotions and generalizes effectively across diverse populations and scenarios. Finally, the detection and classification of different kinds of facial emotions take place using a bidirectional long short-term memory (BiLSTM) network. The simulation analysis of the AFER-POADCNN system is tested on a benchmark FER dataset. The comparative result analysis showed the better performance of the AFER-POADCNN algorithm over existing models, with a maximum accuracy of 99.05%.

1. Introduction

Facial emotion recognition (FER) methods are mainly utilized to recognize facial expressions on the human face [1]. Numerous kinds of emotions occur, but some may not be superficial to the human eye [2]. Therefore, with the help of appropriate mechanisms, any kind of suggestions can aid in identifying the classification. In the FER field, there are different types of universal facial expressions, like neutral, happiness, surprise, fear, anger, sadness, and disgust [3]. From facial expressions, emotion extraction is an active study in mental health, psychiatry, and psychology nowadays [4]. The automatic emotion recognition from facial expressions comes with numerous usages, likely HCI (human–computer interaction), modern augmented reality, healthcare, smart living, and HRI (human–robot interaction) [5]. Most of the researchers are employing FER because it comes with many customs [6]. The procedure to build emotion-specific features is difficult because of several factors which appear from the nonlinear interaction among dissimilar evidence, multidimensional data, and the modalities that one can rapidly face with emotions in different scenarios [7].

There is an increasing demand for robust and accurate systems able to automatically recognize and classify human emotions from facial expressions. Emotions have a major role in human communication, affect decision-making processes, and have applications in diverse domains, from human–computer interaction to mental health assessment. Existing FER models often face challenges related to noise, feature extraction, and generalization. Recently, machine learning (ML) and deep networks have established an effective technique to avoid such restrictions by distinguishing the most multipart nonlinear features linked in multimodal data [8]. The two utmost-serious techniques in emotion detection are feature extraction and classification [9]. Some of the foremost feature classifiers used for superior classification exactness are artificial neural network, ML, and deep-learning (DL) systems [10]. The established feature-engineering and ML methods attempt to remove complicated as well as nonlinear patterns from the multivariate time-series data [11]. However, selecting an effective characteristic from many feature sets is very difficult, so the dimensionality-reduction method will be needed. The feature-extraction as well as selection process takes more time. For instance, when the dimensionality feature increases, the calculating feature overhead selection develops radically [12]. DL techniques, namely, the recurrent neural network (RNN), autoencoder (AE), and convolutional neural network (CNN), have improved in all the areas of computing, especially computer vision, natural language processing (NLP), audio-recognition machine translation, etc. [13]. Recently, DL methods have been employed to deliver high-level data abstraction to develop a flexible structure for emotion detection. In the DL method, DNNs are utilized to gather unique qualities from the high-level data illustration [14].

This study introduces an automated facial emotion recognition using the pelican optimization algorithm with a deep convolutional neural network (AFER-POADCNN) model. The major intention of the AFER-POADCNN model lies in the automated recognition and classification of facial emotions. To accomplish this, the AFER-POADCNN method exploits the median-filtering (MF) approach to remove the noise present in it. Furthermore, the capsule-network (CapsNet) approach can be applied to the feature-extraction process and the hyperparameter tuning of the CapsNet model is carried out by the POA. Finally, the detection and classification of different kinds of facial emotions take place using a bidirectional long short-term memory (BiLSTM) network. The performance analysis of the AFER-POADCNN technique is tested on benchmark FER databases. In short, the key contributions of the paper are summarized as follows.

• An AFER-POADCNN technique comprising MF-based preprocessing, a CapsNet feature extractor, POA-based hyperparameter tuning, and BiLSTM classification has been developed for FER. To the best of our knowledge, the AFER-POADCNN technique has never existed in the literature;
• The CapsNet model has been employed for feature extraction, allowing for the capture of intricate and nuanced facial expressions;
• The POA is presented to tune the hyperparameters of the capsule network, enhancing the model’s adaptability and generalization to different emotions and diverse populations;
• The BiLSTM model applied for emotion classification ensures the robust detection and categorization of various facial emotions.

2. Literature Review

Sarvakar et al. [15] proposed a neural network convolutionary (FERC) technique which uses two parts. At the initial stage, the image backdrop is removed, and then the face vector is removed. For the classification process, the expressional vector (EV) is employed. The two-layer CNN is constant, and the exponent and weight standards of the last perception layer may differ with each iteration. Then, the EV generation ensures the growth of issues before a novel background-removal procedure is utilized. Said and Barr [16] designed a face-sensitive CNN for human-emotion classification, which is collected from two phases. At the initial stage, the method is employed to identify faces in high-resolution images, and then the faces are trimmed for further processing. Next, the CNN is utilized to estimate facial expression, which is relayed on standard analytics. Then, it is implemented on pyramid images for processing scale invariance. In [17], the facial-expression-detection method has been designed. At an initial stage, an area of interest has been performed as face classification. Secondly, a DL-based CNN design is projected. In the third stage, some of the new data-augmentation methods have been applied.

Talaat [18] proposed a real-time emotion-identification method that employs three phases of emotion classification. The selection designs an enhanced DL method to identify facial emotions by utilizing the CNN. The projected emotion-recognition framework took up the benefit of employing the IoT and fog for reducing the delays for real-time classification, with a quick response time as well as providing location awareness. Chowdary et al. [19] mainly dealt with emotion detection by making use of the transfer-learning (TL) method. The well-defined networks of Mobile Net, Vgg19, Resnet50, and Inception V3 are utilized in the research. The pretrained ConvNets are deleted, and then entire connected layers are added that are more appropriate for the totality of the instructions. At last, the fresh additional networks are skilled to improve the weights. In [20], an automated framework algorithm is used for facial recognition by employing an FD-CNN, which is developed with four convolutional layers as well as two hiding networks for enhancing the accuracy. An extensive CK+ dataset is mainly employed, including facial images of dissimilar females and males with various expressions. For validating the projected technique, K-fold cross-validation is executed.

Sikkandar and Thiyagarajan [21] presented an improved cat swarm optimization (ICSO) method. The deep-CNN technique is employed for the extraction process. The ICSO is mainly designed to select optimum features. Using DCNN with the ICSO method enhances the retrieval performance; then, the ensemble classification algorithm uses a support vector machine (SVM) and neural network (NN) that are performed to classify facial expressions. Helaly et al. [22] developed a DCNN method based on an intelligent computer-vision system which is capable of identifying the facial emotions on human faces. In the first stage, the DCNN designed using the TL method is mainly introduced to build up an accurate FER system. Secondly, the research suggests the ResNet18 method.

In [23], a new end-to-end facial-microexpression-recognition architecture termed Deep3DCANN has been developed to combine these modules for active microexpression discovery. The first module of our design is a deep 3D-CNN that learns beneficial spatiotemporal features from a series of facial images. Kansizoglou et al. [24] offer a new model for online emotion detection that features audio as well as visual modalities, and then offers a receptive forecast when the system is sufficiently self-assured. The author developed two deep CNN techniques for removing emotional features; one model for each modality, and a DNN for their fusion. Li et al. [25] presented a unique self-supervised exclusive–inclusive interactive-learning (SEIIL) technique to simplify the discriminative multilabel FER in the wild that effectually grip-coupled manifold thoughts with incomplete unrestrained training data. Kansizoglou et al. [26] presented a new method that slowly maps, as well as learns, human personalities by considering and following a person’s emotional differences through communication. The developed network removes the facial landmarks of a subject, which are utilized to train a properly planned deep recurrent neural network (DRNN) framework.

3. The Proposed Model

In this study, we have designed a novel AFER-POADCNN model. The primary objective of the AFER-POADCNN model lies in the automated recognition and classification of facial emotions. To accomplish this, the AFER-POADCNN method includes different phases of operations, namely, MF-based preprocessing, the CapsNet model for feature extraction, the BiLSTM model for classification, and POA-based hyperparameter tuning. Figure 1 portrays the entire procedure of the AFER-POADCNN system.

3.1. Image Preprocessing

To remove the noise that exists in the input images, MF is used. It is a basic preprocessing method deployed in image processing to decrease noise and increase data quality [26]. Different from other smoothing techniques, MF methods excel at maintaining edges and fine details while efficiently mitigating impulse noise, like salt-and-pepper noise usually created in images. It works by changing all values of the pixels with the median value from a local neighborhood, making it specifically compatible with applications where noise reduction is important, without affecting the integrity of significant image features. MF is extensively utilized in domains like CV, remote sensing, and medical imaging for enhancing the data quality and robustness before additional visualization or analysis.

3.2. Feature Extraction

For deriving the feature vectors, the CapsNet model is applied. Capsule networks, also known as CapsNets, describe a new method for DL techniques, developed to address a few drawbacks of standard CNNs in tasks, namely image recognition [27]. Developed by Geoffrey Hinton and his team, CapsNets present capsules as the main building blocks. These capsules are smaller collections of neurons that function together to identify different parts of objects or visual patterns within an image. Dissimilar CNNs depend on max-pooling for extracting features, and CapsNets utilize dynamic-routing mechanisms to evaluate the spatial relationships among parts and their entire objects. This allows CapsNets to manage complex hierarchical connections among features that are specifically beneficial in conditions where object pose and orientation matter, like in understanding handwritten features or identifying objects in cluttered scenes. Another feature of CapsNets is the capability to manage variable-length pose vectors for all parts, permitting them to obtain rich information about the relative positions and orientations of object components. This makes CapsNets robust to various transformations, including scaling, rotation, and deformation, making them a compelling choice for tasks like image segmentation and object recognition in challenging real-world conditions. While CapsNets are still an evolving field of research, they hold significant promise in advancing the state-of-the-art in computer vision and pattern recognition. Figure 2 exemplifies the infrastructure of the CapsNet.

3.3. Hyperparameter Tuning

In this work, the POA optimally chooses the hyperparameter values of the CapsNet model. POA is a new swarm intelligence (SI)-based optimization algorithm, and pelicans are its population [28]. The swarm member implies a candidate solution. Mainly, the swarm members are randomly initialized based on the problem limit:

$$z_{i,j}=l_j+rand\cdot \left(u_j-l_j\right),j=\left[\mathrm{1,2},\dots ,m\right],i=\left[\mathrm{1,2},\dots ,N\right]$$

(1)

In Equation (1), the upper and lower bounds of the ${j\mathrm{t}\mathrm{h}}^{}$variable of the problem are $u_j$ and $l_j$, correspondingly. The amount of the ${j\mathrm{t}\mathrm{h}}^{}$ parameter defined by the ${i\mathrm{t}\mathrm{h}}^{}$ candidate solutions are $z_{i,j}$; the number of swarm members is $N$, an accidental amount between $(\mathrm{0,1}$) is $rand$, and the number of parameters in the solution space is $m$.

The pelican swarm’s associates can be described by the matrix. In the following matrix, the value in each row describes a candidate solution; moreover, the value in the column depicts the presented amount for the variable in the solution space.

$$Z=\left[\begin{array}{l}{Z}_1\\ ⋮\\ Z_i\\ ⋮\\ Z_N\end{array}\right]=\left[\begin{array}{lllll}{z}_{1,1}& \dots & z_{1,j}& \dots & z_{1,m}\\ ⋮& \ddots & z_{i,j}& & ⋮\\ z_{i,1}& \dots & ⋮& \dots & z_{i,m}\\ ⋮& & ⋮& \ddots & ⋮\\ z_{N,1}& \cdots & z_{N,j}& \dots & z_{N,m}\end{array}\right]$$

(2)

In Equation (2),$Z$ and $Z_i$ are the swarm matrix and ${i\mathrm{t}\mathrm{h}}^{}$ pelicans.

In this work, the cost function is calculated on any candidate solution. The cost-function vector ($F_i$) is used to define the attained amount for the cost function, as follows:

$$F={\left[\begin{array}{l}{F}_1\\ ⋮\\ F_i\\ ⋮\\ F_N\end{array}\right]}_{N\times 1}=\left[\begin{array}{l}F\left(Z_1\right)\\ ⋮\\ F\left(Z_i\right)\\ ⋮\\ F\left(Z_N\right)\end{array}\right]$$

(3)

1.

Exploration Stage (Moving direction of Bait)

First, pelican members used to identify the hunting region; then, they move towards that place. The solution area is scanned due to its simulation of the pelican strategy; also, it gives rise to the exploration capability of the POA to explore different areas of the solution space. Consider that the hunting position is randomly generated in the searching region; the exploration ability rises once it finds the solution space. This can be mathematically expressed as follows:

$$z_{i,j}^{p_1}=\left\{\begin{array}{l}{z}_{i,j}+rand.\left(p_j-I.z_{i,j}\right),F_p<F_i,\\ z_{i,j}+rand.\left(z_{i,j}-p_j\right),else,\end{array}\right.$$

(4)

In Equation (4), the new situation of ${i\mathrm{t}\mathrm{h}}^{}$ pelicans at the dimension is $z_{i,j}^{p_1}$, $I$ indicates a random integer; the prey location in the ${j\mathrm{t}\mathrm{h}}^{}$ dimension denotes the $p_j$. The parameter $I$ is arbitrarily chosen for any member and any iteration. If the quantity of $I$ is 2, then it increases in dislocation for a member; thus, the member conducts a new region of the problem. The exploration ability of these optimizers for the incorrect scanning of the problem is better than $\mathrm{t}\mathrm{h}\mathrm{e}$ $I$ parameter.

The new position for the pelican is attained, which provides the cost-function value. The algorithm could not move towards the nonoptimum region using this type of upgrade, which is called the effective-update process. This method is simulated by the subsequent formula:

$$Z_i=\left\{\begin{array}{l}{Z}_i^{p_1},F_i^{p_1}<F_i;\\ Z_i,else\end{array}\right.$$

(5)

In Equation (5), $Z_i^{p_1}$ shows the new position of ${i\mathrm{t}\mathrm{h}}^{}$ pelicans and $F_i^{p_1}$ represents its cost-function amount.

2.

Exploitation Stage (Winging on the Water Surface)

In this phase, the pelicans spread their wings to attain the water surface; therefore, the fish come out and they gather bait in their throat pouch. The statistical formula for the demeanor of pelicans during hunting is given below:

$$z_{i,j}^{p_2}=z_{i,j}+R\times \left(1-\frac{t}{T}\right)\times \left(2\times rand-1\right)\times z_{i,j}$$

(6)

In Equation (6), the new location of the ${i\mathrm{t}\mathrm{h}}^{}$ individuals at the $j\mathrm{th}$ parameter is $z_{i,j}^{p_2}$, and the neighborhood radius of $z_{i,j}$is $R\times \left(1-\frac{t}{T}\right)$, where the iteration number is $t$, $R$ equals 0.2, and the maximal iteration counter is $T$. $\mathrm{“}R\times \left(1-\frac{t}{T}\right)\mathrm{”}$ is the coefficient representing the radii of the surrounding of the swarm individual to the local exploration, and the proximity to any member to converge for a promising solution. The nearby area and swarm member with more precise and shorter movements can be examined, and the PO is able to converge the answer closer to the global optima. In this stage, an effective update is used for taking or refusing the new pelican position, which is formulated as follows:

$$Z_i=\left\{\begin{array}{l}{Z}_i^{p_2},F_i^{p_2}<F_i;\\ Z_i,else\end{array}\right.$$

(7)

In Equation (7), the novel situation of the ${i\mathrm{t}\mathrm{h}}^{}$ pelican is $Z_i^{p_2}$ and the cost-function amount is $F_i^{p_2}$.

3.

Repetition

Once the swarm individuals are updated, the optimal solution will be upgraded by the rate of the performance index and the new position of the swarm. The next iteration begins, and the different stages of the proposed PO, using the abovementioned formula, are repeated, finishing the whole performance. Eventually, the best solution candidate attained in the algorithm epoch is shown as a quasioptimal solution.

The POA algorithm derives an FF to obtain a high efficiency of classification. It determines a positive integer to represent the improved performance of the solution candidate. The decline of the classifier error rate is considered an FF.

$$fitness\left(x_i\right)=ClassifierErrorRate\left(x_i\right)$$

$$=\frac{numberofmisclassifiedsamples}{Totalnumberofsamples}\ast 100$$

(8)

3.4. Detection Using the BiLSTM Model

To detect the presence of emotions in distinct types, the BiLSTM model is applied. The input, forget, and output gates are the three gates of the LSTM unit [29]. The input gate defines what amount of the input data to remain in the existing state of the memory unit, the forget gate is a basic design for the LSTM to learn long-term dependency, and the output gate decides which state of the memory cell is transported to the hidden output state. The state of the memory unit at $t-1$ is $C_{t-1}$, the input of the unit at existing moment $t$ is $x_t,$and the output of the hidden state at prior time $t-1$is $h_{t-1}$, which are the three inputs of the LSTM neurons. The state of the memory cell at $t$ is $C_t$ and the output of the hidden layer (HL) at $t$ is $h_t$, which are the two outputs of the LSTM neuron:

$$f_t=\sigma \left(w_{fx}{x}_t+w_{fh}{h}_{t-1}+b_1\right)$$

(9)

$$I_t=\sigma \left(w_{\mathrm{i}\mathrm{x}}{x}_t+w_{ih}{h}_{t-1}+b_2\right)$$

(10)

$$Z_t=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left(w_{zx}{x}_t+w_{zh}{h}_{t-1}+b_3\right)$$

(11)

$$C_t=f_t\circ C_{t-1}+i_t\circ Z_t$$

(12)

$$O_t=\sigma \left(w_{ox}{x}_t+w_{oh}{h}_{t-1}+b_4\right)$$

(13)

$$h_t=O_t\circ \mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left(C_t\right)$$

(14)

From the equations, the sigmoid function is $\sigma$. The multiplication by elements is $“\circ ”$; $w$ and $b$ are the corresponding weight parameters and bias.

The prediction of the BiLSTM is based on time sequences and considers the negative and positive direction of prior data. The BiLSTM model comprises two layers of one-way LSTM, where HL in the positive time direction consists of prior data series and evaluates the present data sequence. HL in the reverse time direction is used to add the reverse data series in the calculation and read the future data series in the input. Next, the value defined by the two LSTM modules is feedforwarded into the output layer.

4. Results and Discussion

The proposed model is simulated using the Python 3.8.5 tool on a PC i5-8600k, GeForce 1050Ti 4 GB, 16 GB RAM, 250 GB SSD, and 1 TB HDD. The parameter settings are given as follows: learning rate: 0.01, dropout: 0.5, batch size: 5, epoch count: 50, and activation: ReLU. In this section, the FER outcome of the AFER-POADCNN algorithm is examined on the FER database [30], comprising 920 images and 8 classes, as illustrated in

Table 1. Details of the database.

Classes	No. of Samples
Anger	45
Contempt	18
Disgust	59
Fear	25
Happy	69
Neutral	593
Sad	28
Surprise	83
Total No. of Sample Images	920

.

Figure 3 demonstrates the confusion matrices attained by the AFER-POADCNN algorithm at 80:20 and 70:30 of the TR phase/TS phase. The outcome denotes the effective recognition and classification of all eight classes.

In

Table 2. FER outcome of the AFER-POADCNN algorithm at 80:20 of the TR phase/TS phase.

Classes	$\mathit{A}\mathit{c}\mathit{c}{\mathit{u}}_{\mathit{y}}$	$\mathit{S}\mathit{e}\mathit{n}{\mathit{s}}_{\mathit{y}}$	$\mathit{S}\mathit{p}\mathit{e}{\mathit{c}}_{\mathit{y}}$	${\mathit{F}}_{\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}}$	MCC
TR Phase (80%)
Anger	98.91	82.35	99.72	87.50	87.12
Contempt	98.51	46.67	99.58	56.00	56.45
Disgust	99.05	94.23	99.42	93.33	92.83
Fear	98.78	63.64	99.86	75.68	76.52
Happy	99.46	98.15	99.56	96.36	96.09
Neutral	97.42	99.36	94.01	98.00	94.43
Sad	99.59	91.30	99.86	93.33	93.15
Surprise	99.05	95.52	99.40	94.81	94.29
Average	98.85	83.90	98.93	86.88	86.36
TS Phase (20%)
Anger	98.37	72.73	100.00	84.21	84.55
Contempt	100.00	100.00	100.00	100.00	100.00
Disgust	98.91	100.00	98.87	87.50	87.69
Fear	99.46	66.67	100.00	80.00	81.43
Happy	98.91	93.33	99.41	93.33	92.74
Neutral	98.91	100.00	96.67	99.20	97.54
Sad	100.00	100.00	100.00	100.00	100.00
Surprise	97.83	87.50	98.81	87.50	86.31
Average	99.05	90.03	99.22	91.47	91.28

and Figure 4, the FER results of the AFER-POADCNN technique at 80:20 of the TR phase/TS phase are presented. The simulation values highlight that the AFER-POADCNN method properly recognized facial emotions accurately. With 80% of the TR phase, the AFER-POADCNN technique offers an average $acc{u}_y$ of 98.85%, $sen{s}_y$ of 83.90%, $spe{c}_y$ of 98.93%, $F_{score}$ of 86.88%, and an MCC of 86.36%. Additionally, with 20% of the TS phase, the AFER-POADCNN approach achieves an average $acc{u}_y$ of 99.05%, $sen{s}_y$ of 90.03%, $spe{c}_y$ of 99.22%, $F_{score}$ of 91.47%, and an MCC of 91.28%.

In

Table 3. FER outcome of AFER-POADCNN algorithm at 70:30 of the TR phase/TS phase.

Classes	$\mathit{A}\mathit{c}\mathit{c}{\mathit{u}}_{\mathit{y}}$	$\mathit{S}\mathit{e}\mathit{n}{\mathit{s}}_{\mathit{y}}$	$\mathit{S}\mathit{p}\mathit{e}{\mathit{c}}_{\mathit{y}}$	${\mathit{F}}_{\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}}$	MCC
TR Phase (70%)
Anger	99.22	96.88	99.35	92.54	92.23
Contempt	99.38	62.50	99.84	71.43	71.87
Disgust	98.91	87.18	99.67	90.67	90.17
Fear	99.07	68.42	100.00	81.25	82.32
Happy	99.38	95.83	99.66	95.83	95.50
Neutral	97.67	99.76	93.67	98.25	94.86
Sad	98.91	64.71	99.84	75.86	76.52
Surprise	98.76	93.10	99.32	93.10	92.42
Average	98.91	83.55	98.92	87.37	86.99
TS Phase (30%)
Anger	99.28	100.00	99.24	92.86	92.74
Contempt	99.28	80.00	100.00	88.89	89.11
Disgust	98.91	90.00	99.61	92.31	91.76
Fear	99.64	83.33	100.00	90.91	91.12
Happy	98.91	95.24	99.22	93.02	92.46
Neutral	97.46	98.82	95.28	97.96	94.64
Sad	98.91	81.82	99.62	85.71	85.26
Surprise	98.19	88.00	99.20	89.80	88.82
Average	98.82	89.65	99.02	91.43	90.74

and Figure 5, the FER outcome of the AFER-POADCNN approach at 70:30 of the TR phase/TS phase is presented. The outcome displayed that the AFER-POADCNN algorithm appropriately detected the facial emotions accurately. With 70% of the TR phase, the AFER-POADCNN system gains an average $acc{u}_y$ of 98.91%, $sen{s}_y$ of 83.55%, $spe{c}_y$ of 98.92%, $F_{score}$ of 87.37%, and an MCC of 86.99%. Furthermore, with 30% of the TS phase, the AFER-POADCNN approach reaches an average $acc{u}_y$ of 98.82%, $sen{s}_y$ of 89.65%, $spe{c}_y$ of 99.02%, $F_{score}$ of 91.43%, and an MCC of 90.74%.

To estimate the performance of the AFER-POADCNN algorithm at 80:20 of the TR phase/TS phase, TR and TS $acc{u}_y$ curves are defined, as illustrated in Figure 6. The TR and TS $acc{u}_y$ curves demonstrate the outcome of the AFER-POADCNN algorithm on various epochs. The figure offers meaningful details regarding the learning task and generalization capabilities of the AFER-POADCNN approach. With an enhancement in the epoch count, it is observed that the TR and TS $acc{u}_y$ curves are enhanced. It is still experimental that the AFER-POADCNN algorithm attains higher testing accuracy, which has the capability in identifying the patterns in the TR and TS data.

Figure 7 reveals the overall TR and TS loss values of the AFER-POADCNN algorithm at 80:20 of the TR phase/TS phase over epochs. The TR loss displays that the model loss is lesser over epochs. Primarily, the loss values are reduced as the model modifies the weight to minimize the prediction error on the TR and TS data. The loss curves demonstrate the extent to which the model fits the training data. It is detected that the TR and TS loss is steadily decreased, and represents that the AFER-POADCNN approach effectually learns the patterns displayed in the TR and TS data. It is also noticed that the AFER-POADCNN algorithm fine-tunes the parameters for decreasing the discrepancy between the prediction and the original training label.

The precision–recall (PR) outcome of the AFER-POADCNN approach at 80:20 of the TR phase/TS phase is represented by plotting the precision against the recall, as defined in Figure 8. The outcomes confirm that the AFER-POADCNN algorithm reaches higher PR performances under all classes. The outcome exhibits that the model learns to identify distinct classes. The AFER-POADCNN algorithm reaches improved solutions in the recognition of positive instances, with minimal false positives.

The ROC curves offered by the AFER-POADCNN algorithm at 80:20 of the TR phase/TS phase are illustrated in Figure 9, which has the capability of discriminating the class labels. The outcome implies appreciated insights into the trade-offs among the TPR and FPR rates, with various classifier thresholds and distinct counts of epochs. It defines the correct predictive outcome of the AFER-POADCNN approach on the classifier of various classes.

The comparison study of the AFER-POADCNN technique with recent systems is given in

Table 4. Comparative outcome of the AFER-POADCNN system with recent methods [31].

Methods	$\mathit{S}\mathit{e}\mathit{n}{\mathit{s}}_{\mathit{y}}$	$\mathit{S}\mathit{p}\mathit{e}{\mathit{c}}_{\mathit{y}}$	$\mathit{A}\mathit{c}\mathit{c}{\mathit{u}}_{\mathit{y}}$	${\mathit{F}}_{\mathit{s}\mathit{c}\mathit{o}\mathit{r}\mathit{e}}$
AFER-POADCNN	90.03	99.22	99.05	91.47
HGSO-DLFER	84.99	98.65	98.45	87.78
ResNet50	83.96	83.65	88.54	85.99
SVM	83.17	82.18	91.64	87.55
MobileNet	83.74	83.81	92.32	86.52
Inception-V3	80.23	84.06	93.74	73.82
CNN-VGG19	82.95	81.59	94.03	81.75

[31]. Figure 10 represents a comparative $acc{u}_y$ and $F_{score}$ examination of the AFER-POADCNN technique with recent models. The results imply that the AFER-POADCNN approach gains better results. Based on the $acc{u}_y$, the AFER-POADCNN technique offers a higher $acc{u}_y$of 99.05%, while the HGSO-DLFER, ResNet50, SVM, MobileNet, Inception-v3, and CNN-VGG19 models obtain reduced $acc{u}_y$ values of 98.45%, 88.54%, 91.64%, 92.32%, 93.74%, and 94.035%, respectively. Additionally, based on the $F_{score}$, the AFER-POADCNN system achieves an enhanced $F_{score}$of 91.47%, while the HGSO-DLFER, ResNet50, SVM, MobileNet, Inception-v3, and CNN-VGG19 approaches reach decreased $F_{score}$ values of 87.78%, 85.99%, 87.55%, 86.52%, 73.82%, and 81.75%, correspondingly.

Figure 11 signifies a comparative $sen{s}_y$ and $spe{c}_y$ inspection of the AFER-POADCNN system with recent algorithms. The outcomes inferred that the AFER-POADCNN algorithm attains optimum solutions. Based on $sen{s}_y$, the AFER-POADCNN algorithm achieves a superior $sen{s}_y$ of 90.03%, while the HGSO-DLFER, ResNet50, SVM, MobileNet, Inception-v3, and CNN-VGG19 algorithms achieve lesser $sen{s}_y$ values of 84.99%, 83.96%, 83.17%, 83.74%, 80.23%, and 82.95%, correspondingly. Furthermore, based on $spe{c}_y$, the AFER-POADCNN system offers an enhanced $spe{c}_y$of 99.22%, while the HGSO-DLFER, ResNet50, SVM, MobileNet, Inception-v3, and CNN-VGG19 systems gain minimal $spe{c}_y$ values of 98.65%, 83.65%, 82.18%, 83.81%, 84.06%, and 81.59%, correspondingly. These performances confirmed the higher outcome of the AFER-POADCNN algorithm.

5. Conclusions

In this study, we have designed a novel AFER-POADCNN model for automated and accurate FER. The primary objective of the AFER-POADCNN model lies in the automatic recognition and classification of facial emotions. To accomplish this, the AFER-POADCNN method comprises MF-based preprocessing, a CapsNet feature extractor, POA-based hyperparameter tuning, and BiLSTM-based classification for FER. Finally, the detection and classification of different kinds of facial emotions take place using the BiLSTM network. The simulation analysis of the AFER-POADCNN algorithm can be tested on benchmark FER databases. The stimulation values demonstrated the better performance of the AFER-POADCNN model over existing techniques, with a maximum accuracy of 99.05%. Further enhancements can include expanding the model’s robustness to varying lighting conditions, facial expressions, and head poses. Additionally, exploring real-time applications and the integration of multimodal data, such as audio and text analysis, can pave the way for a more comprehensive understanding of human emotions in a wide range of contexts. Future work can investigate the computational efficiency of the proposed model.