Various Deep Learning Algorithms in Computational Intelligence

Deep Learning (DL) is an essential topic of increasing interest in science, industry, and academia. Unlike traditional and machine learning methods, DL methods can process large volumes of unstructured data discovering intricate structures in large data sets. It is rapidly becoming a tool for modeling and solving complex and difficult problems in different fields of science and technology. For example, in medicine for breast cancer, COVID-19 detection and diabetes detection and prediction; in autonomous vehicles in various tasks such as perception, mapping and localization; in astronomy, for classifying and detecting stars and galaxies; in the design of future wireless networks, and others. Although DL has been successfully applied in many fields and there are some theoretical developments, there are still many challenging problems in theory and applications that need to be solved for improving these techniques. For instance, it is important to find new methods to train the massive number of parameters required by DL architectures, solve overfitting and transfer learning problems, and others.

This Special Issue was dedicated to contributing to the state-of-the-art progress on DL with theoretical, practical, and creative insights that provided vanguard solutions to challenging problems or could demonstrate competitive performance. The participation of the community in sending works to this Special Issue was high and important, and after a rigorous peer-reviewing process only fourteen papers were accepted to be published. Next, we mention in chronological publication order the accepted works.

The first contribution was the “Optimization of Convolutional Neural Networks Architectures Using PSO for Sign Language Recognition” paper by Jonathan Fregoso, Claudia I. Gonzalez, and Gabriela E. Martínez [1]. Here, the authors presented an approach to designing convolutional neural network architectures using the particle swarm optimization algorithm (PSO), where they face the challenge of finding the optimal parameters of convolutional neural networks, particularly the number of layers, the filter size, the number of convolutional filters, and the batch size. They evaluated two different approaches to perform the optimization. In the first one, the parameters obtained by PSO were kept under the same conditions in each convolutional layer, and the classification rate gave the objective function evaluated by PSO. In the second one, the PSO generated different parameters per layer. The objective function was composed of recognition rate in conjunction with the Akaike information criterion; the latter helps to find the best network performance but with the minimum parameters. Finally, they tested the optimized architectures with three study cases: sign language databases where the Mexican Sign Language alphabet was included, the American Sign Language MNIST, and the American Sign Language alphabet. The authors found that the proposed methodologies achieved favorable results with a recognition rate higher than 99%, showing competitive results compared to other state-of-the-art approaches.

The paper “Multicriteria Evaluation of Deep Neural Networks for Semantic Segmentation of Mammographies” by Yoshio Rubio and Oscar Montiel [2] studied the critical problem of breast segmentation for automatic and accurate analysis of mammograms to increments the probability of a correct diagnostic while reducing the computational cost, which was done through an extensive evaluation of deep learning architectures for semantic segmentation of mammograms, including segmentation metrics, memory requirements, and average inference time. They used different combinations of two-stage segmentation architectures composed of a feature extraction net (VGG16 and ResNet50) and a segmentation net (FCN-8, U-Net, and PSPNet). The author used examples from the mini–Mammographic Image Analysis Society (MIAS) database for the learning phase. The experimental results showed that the best net scored a Sørensen–Dice similarity coefficient of 99.37% for breast boundary segmentation and 95.45% for pectoral muscle segmentation.

In the paper “RainPredRNN: A New Approach for Precipitation Nowcasting with Weather Radar Echo Images Based on Deep Learning” by Do Ngoc Tuyen, Tran Manh Tuan, Xuan-Hien Le, Nguyen Thanh Tung, Tran Kim Chau, Pham Van Hai, Vassilis C. Gerogiannis, and Le Hoang Son [3], the authors proposed a novel approach named RainPredRNN, which combines the UNet segmentation model and the PredRNN_v2 deep learning model for precipitation nowcasting with weather radar echo images. They found that it is possible to reduce the number of operations of the RainPredRNN model by leveraging the abilities of the contracting–expansive path of the UNet model. Consequently, the result offers the benefit of reducing the processing time of the overall model while maintaining reasonable errors in the predicted images. They validated their proposed mode through experiments on real reflectivity fields collected from the Phadin weather radar station in Dien Bien province in Vietnam. Some credible quality metrics, such as the mean absolute error (MAE), the structural similarity index measure (SSIM), and the critical success index (CSI), were used for analyzing the performance of the model. It was certified that the proposed model had produced improved performance, about 0.43, 0.95, and 0.94 of MAE, SSIM, and CSI, respectively, with only 30% of training time compared to the other methods.

The paper “Cubical Homology-Based Machine Learning: An Application in Image Classification” by Seungho Choe and Sheela Ramanna [4] contributed with a cubical homology-based algorithm for extracting topological features from 2D images to generate their topological signatures; they propose a score metric to measure the significance of the subcomplex calculated from the persistence diagram (topological signatures). Additionally, they used gray-level co-occurrence matrix (GLCM) and contrast limited adapting histogram equalization (CLAHE) to obtain additional image features to improve the classification performance; finally, they discussed the results of our supervised learning experiments of eight well-known machine learning models trained on six different published image datasets using the extracted topological features.

In the study, “Towards Predictive Vietnamese Human Resource Migration by Machine Learning: A Case Study in Northeast Asian Countries”, the authors Nguyen Hong Giang, Tien-Thinh Nguyen, Chac Cau Tay, Le Anh Phuong, and Thanh-Tuan Dang [5] presented a forecast of Vietnamese labor migration using the kNN, RFR, and BPNN models. The data collected in the study included 29 observations from 1992 to 2020. With the support of the collected data, the paper analyzed the role of labor exports in Vietnam’s socio-economic development and compared this export labor with that of other Asian countries. After that, they compared the three algorithms based on the results of their statistical accuracy indicators. This research highlighted the likely future contexts of labor migration between Vietnam and East Asia, including Korea, the Republic of China (Taiwan), and Japan. Furthermore, this research could assist the government of Vietnam in enacting new regulations for Vietnamese migrant workers to boost the socio-economic situation.

In “Lexicon-Enhanced Multi-Task Convolutional Neural Network for Emotion Distribution Learning”, by Yuchang Dong and Xueqiang Zeng [6], the authors dealt with the problem of emotion analysis behind massive human behavior such as text, pictures, movies, and music. To address this problem, they proposed a text emotion distribution learning model based on a lexicon-enhanced multi-task convolutional neural network (LMT-CNN). The overall architecture of the LMT-CNN model has three major modules: semantic information, emotion knowledge, and multi-task prediction. They constructed the input of the multi-task prediction module from the outputs of the first two modules. Then, they predicted the final emotion distribution through a fully connected layer. They presented extensive comparative experiments on nine commonly used emotional text datasets, showing that the proposed LMT-CNN model is superior to the compared EDL methods for both emotion distribution prediction and emotion recognition tasks.

In “A Schelling Extended Model in Networks—Characterization of Ghettos in Washington D.C”, Diego Ortega and Elka Korutcheva [7] proposed a method to bridge the gap between Sociophysics models and the measure of segregation by geographic information systems (GIS) techniques. Their procedure relies on capturing the ghettos’ location with a segregation model and basic data from the city in situations where detailed data cannot be completely reliable. This is a novel approach to the segregation field. They started by using basic GIS information from the region of interest to define a network and then ran an extended Schelling model on this framework. Their enhanced version considered the economic contribution to segregation, including terms for the housing market and the financial gap between the population. The result was the location of ghettos on the network, which could be mapped into their corresponding city areas. Finally, the predicted ostracized regions were compared to the ones characterized as ghettos by spatial analysis (SA) and machine learning (ML) algorithms. For the real case study of Washington D.C., the obtained accuracy was 80 ± 7%.

In “Hybrid Deep Learning Algorithm for Forecasting SARS-CoV-2 Daily Infections and Death Cases” by Fehaid Alqahtani, Mostafa Abotaleb, Ammar Kadi, Tatiana Makarovskikh, Irina Potoroko, Khder Alakkari, and Amr Badr [8], the authors used a hybrid deep learning algorithm to predict new cases of infection which is crucial for authorities to get ready for early handling of the virus spread. The hybrid deep learning method was used to improve the parameters of long short-term memory (LSTM). To evaluate the effectiveness of the proposed methodology, the authors collected a dataset based on the recorded cases in the Russian Federation and Chelyabinsk region between 22 January 2020 and 23 August 2022. In addition, five regression models were included in the conducted experiments to show the effectiveness and superiority of their proposed approach. The achieved results showed that their proposal could reduce the mean square error (RMSE), relative root mean square error (RRMSE), mean absolute error (MAE), coefficient of determination (R Square), coefficient of correlation (R), and mean bias error (MBE) when compared with the five base models. The results confirmed the effectiveness, superiority, and significance of the proposed approach in predicting the infection cases of SARS-CoV-2.

In “Score-Guided Generative Adversarial Networks” by Minhyeok Lee and Junhee Seok [9], a generative adversarial network (GAN) that introduces an evaluator module using pre-trained networks was proposed, and it was called a score-guided GAN (ScoreGAN). The module was trained using an evaluation metric for GANs, i.e., the Inception score, as a rough guide for the training of the generator. Using another pre-trained network instead of the Inception network, ScoreGAN circumvented the overfitting of the Inception network such that the generated samples do not correspond to adversarial examples of the Inception network. In addition, they used evaluation metrics only in an auxiliary role to prevent overfitting. When evaluated using the CIFAR-10 dataset, ScoreGAN achieved an Inception score of 10.36 ± 0.15, corresponding to state-of-the-art performance. To generalize the effectiveness of ScoreGAN, the model was evaluated further using another dataset, CIFAR-100. ScoreGAN outperformed existing methods.

In the paper “Improved Method for Oriented Waste Detection”, Weizhi Yang, Yi Xie, and Peng Gao [10] investigated automated waste classification and image-related problems when using robotic arms. Here, the authors, to solve the problem of low-accuracy image detection caused by irregular placement angles of robotic arms, proposed an improved oriented waste-detection method based on YOLOv5 by optimizing the detection head of the YOLOv5 model. This method generated an oriented detection box for a waste object at any angle. Based on the proposed scheme, they further improved three aspects of the performance of YOLOv5 in the detection of waste objects: the angular loss function was derived based on dynamic smoothing to enhance the model’s angular prediction ability, the backbone network was optimized with enhanced shallow features and attention mechanisms, and the feature aggregation network was improved to enhance the effects of feature multi-scale fusion. The experimental results showed that the detection performance of the proposed method for waste targets was better than other deep learning methods. Its average accuracy and recall were 93.9% and 94.8%, 11.6% and 7.6% higher than the original network, respectively.

In “Two Novel Models for Traffic Sign Detection Based on YOLOv5s” by Wei Bai, Jingyi Zhao, Chenxu Dai, Haiyang Zhang, Li Zhao, Zhanlin Ji, and Ivan Ganchev [11], the problem of object detection and image recognition for unmanned driving technology was studied, particularly, the detection and recognition of traffic signs which are affected by diverse factors such as light, the presence of small objects, and complicated backgrounds. It is well known that traditional traffic sign detection technology does not produce satisfactory results when facing the above problems. To solve this problem, the authors proposed two novel traffic sign detection models called YOLOv5-DH and YOLOv5-TDHSA. Experiments conducted on two public datasets showed that both proposed models perform better than the original YOLOv5s model and three other state-of-the-art models.

In “Barrier Options and Greeks: Modeling with Neural Networks”, the authors, Nneka Umeorah, Phillip Mashele, Onyecherelam Agbaeze, and Jules Clement Mba [12], proposed a non-parametric technique of option valuation and hedging. Using the fully connected feed-forward neural network, they replicated the extended Black–Scholes pricing model for the exotic barrier options and their corresponding Greeks. Their methodology involved benchmarking experiments resulting in an optimal neural network hyperparameter that effectively prices the barrier options and facilitates their option Greeks extraction. They compared the results from the optimal NN model to those produced by other machine learning models, such as the random forest and the polynomial regression; the output highlighted the proposed methodology’s accuracy and efficiency for the pricing problem. The results showed that the artificial neural network could effectively and accurately learn the extended Black–Scholes model from a given simulated dataset. This concept can similarly be applied to the valuation of complex financial derivatives without analytical solutions.

In “Developing a Deep Learning-Based Defect Detection System for Ski Goggles Lenses” by Dinh-Thuan Dang, and Jing-Wein Wang [13], the authors presented a study that developed a deep learning-based defect detection system for ski goggles lenses. In the study, the first step was to design an image acquisition model that combines cameras and light sources; this step aimed to capture clear and high-resolution images on the entire surface of the lenses. Next, defect categories were identified, including scratches, watermarks, spotlight, stains, dust-line, and dust-spot. Then, they were labeled to create the ski goggles lens defect dataset. Finally, the defects were detected automatically by fine-tuning the mobile-friendly object detection model, the MobileNetV3 backbone used in a feature pyramid network (FPN), and the Faster-RCNN detector. The experiments demonstrate the effectiveness of defect detection at faster inference speeds. The defect detection accuracy achieved a mean average precision (mAP) of 55%. The work automatically integrated all steps, from capturing images to defect detection.

In “A Unified Learning Approach for Malicious Domain Name Detection” by Atif Ali Wagan, Qianmu Li, Zubair Zaland, Shah Marjan, Dadan Khan Bozdar, Aamir Hussain, Aamir Mehmood Mirza, and Mehmood Baryalai [14]. The authors presented a novel unified learning approach that uses both numerical and textual features of the domain name to classify whether a domain name pair is malicious or not. They conducted experiments on a benchmark domain names dataset consisting of 90,000 domain names. The experimental results show that the proposed approach performs significantly better than the six comparative methods in terms of accuracy, precision, recall, and F1-Score.

The diversity of fields of publications covers a wide range of interesting topics for researchers, scholars, and students interested in vanguard scientific contributions and practical applications to solve real-life problems.