Algorithms

Research

Jump to: Review

21 pages, 6730 KiB

Open AccessArticle

Simplified Routing Mechanism for Capsule Networks

by János Hollósi, Áron Ballagi and Claudiu Radu Pozna

Algorithms 2023, 16(7), 336; https://doi.org/10.3390/a16070336 - 13 Jul 2023

Cited by 5 | Viewed by 2093

Abstract

Classifying digital images using neural networks is one of the most fundamental tasks within the field of artificial intelligence. For a long time, convolutional neural networks have proven to be the most efficient solution for processing visual data, such as classification, detection, or [...] Read more.

Classifying digital images using neural networks is one of the most fundamental tasks within the field of artificial intelligence. For a long time, convolutional neural networks have proven to be the most efficient solution for processing visual data, such as classification, detection, or segmentation. The efficient operation of convolutional neural networks requires the use of data augmentation and a high number of feature maps to embed object transformations. Especially for large datasets, this approach is not very efficient. In 2017, Geoffrey Hinton and his research team introduced the theory of capsule networks. Capsule networks offer a solution to the problems of convolutional neural networks. In this approach, sufficient efficiency can be achieved without large-scale data augmentation. However, the training time for Hinton’s capsule network is much longer than for convolutional neural networks. We have examined the capsule networks and propose a modification in the routing mechanism to speed up the algorithm. This could reduce the training time of capsule networks by almost half in some cases. Moreover, our solution achieves performance improvements in the field of image classification. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

16 pages, 1370 KiB

Open AccessArticle

Optimization of the Compressive Measurement Matrix in a Massive MIMO System Exploiting LSTM Networks

by Saidur R. Pavel and Yimin D. Zhang

Algorithms 2023, 16(6), 261; https://doi.org/10.3390/a16060261 - 23 May 2023

Viewed by 1129

Abstract

Massive multiple-input multiple-output (MIMO) technology, which is characterized by the use of a large number of antennas, is a key enabler for the next-generation wireless communication and beyond. Despite its potential for high performance, implementing a massive MIMO system presents numerous technical challenges, [...] Read more.

Massive multiple-input multiple-output (MIMO) technology, which is characterized by the use of a large number of antennas, is a key enabler for the next-generation wireless communication and beyond. Despite its potential for high performance, implementing a massive MIMO system presents numerous technical challenges, including the high hardware complexity, cost, and power consumption that result from the large number of antennas and the associated front-end circuits. One solution to these challenges is the use of hybrid beamforming, which divides the transceiving process into both analog and digital domains. To perform hybrid beamforming efficiently, it is necessary to optimize the analog beamformer, referred to as the compressive measurement matrix (CMM) here, that allows the projection of high-dimensional signals into a low-dimensional manifold. Classical approaches to optimizing the CMM, however, are computationally intensive and time consuming, limiting their usefulness for real-time processing. In this paper, we propose a deep learning based approach to optimizing the CMM using long short-term memory (LSTM) networks. This approach offers high accuracy with low complexity, making it a promising solution for the real-time implementation of massive MIMO systems. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

13 pages, 695 KiB

Open AccessArticle

Recovering the Forcing Function in Systems with One Degree of Freedom Using ANN and Physics Information

by Shadab Anwar Shaikh, Harish Cherukuri and Taufiquar Khan

Algorithms 2023, 16(5), 250; https://doi.org/10.3390/a16050250 - 12 May 2023

Cited by 2 | Viewed by 2276

Abstract

In engineering design, oftentimes a system’s dynamic response is known or can be measured, but the source generating these responses is not known. The mathematical problem where the focus is on inferring the source terms of the governing equations from the set of [...] Read more.

In engineering design, oftentimes a system’s dynamic response is known or can be measured, but the source generating these responses is not known. The mathematical problem where the focus is on inferring the source terms of the governing equations from the set of observations is known as an inverse source problem (ISP). ISPs are traditionally solved by optimization techniques with regularization, but in the past few years, there has been a lot of interest in approaching these problems from a deep-learning viewpoint. In this paper, we propose a deep learning approach—infused with physics information—to recover the forcing function (source term) of systems with one degree of freedom from the response data. We test our architecture first to recover smooth forcing functions, and later functions involving abruptly changing gradient and jump discontinuities in the case of a linear system. Finally, we recover the harmonic, the sum of two harmonics, and the gaussian function, in the case of a non-linear system. The results obtained are promising and demonstrate the efficacy of this approach in recovering the forcing functions from the data. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

17 pages, 6707 KiB

Open AccessArticle

Machine-Learning-Based Model for Hurricane Storm Surge Forecasting in the Lower Laguna Madre

by Cesar Davila Hernandez, Jungseok Ho, Dongchul Kim and Abdoul Oubeidillah

Algorithms 2023, 16(5), 232; https://doi.org/10.3390/a16050232 - 28 Apr 2023

Cited by 6 | Viewed by 2860

Abstract

During every Atlantic hurricane season, storms represent a constant risk to Texan coastal communities and other communities along the Atlantic coast of the United States. A storm surge refers to the abnormal rise of sea water level due to hurricanes and storms; traditionally, [...] Read more.

During every Atlantic hurricane season, storms represent a constant risk to Texan coastal communities and other communities along the Atlantic coast of the United States. A storm surge refers to the abnormal rise of sea water level due to hurricanes and storms; traditionally, hurricane storm surge predictions are generated using complex numerical models that require high amounts of computing power to be run, which grow proportionally with the extent of the area covered by the model. In this work, a machine-learning-based storm surge forecasting model for the Lower Laguna Madre is implemented. The model considers gridded forecasted weather data on winds and atmospheric pressure over the Gulf of Mexico, as well as previous sea levels obtained from a Laguna Madre ocean circulation numerical model. Using architectures such as Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) combined, the resulting model is capable of identifying upcoming hurricanes and predicting storm surges, as well as normal conditions in several locations along the Lower Laguna Madre. Overall, the model is able to predict storm surge peaks with an average difference of 0.04 m when compared with a numerical model and an average RMSE of 0.08 for normal conditions and 0.09 for storm surge conditions. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

18 pages, 8283 KiB

Open AccessArticle

Detection of Plausibility and Error Reasons in Finite Element Simulations with Deep Learning Networks

by Sebastian Bickel, Stefan Goetz and Sandro Wartzack

Algorithms 2023, 16(4), 209; https://doi.org/10.3390/a16040209 - 13 Apr 2023

Cited by 1 | Viewed by 3000

Abstract

The field of application of data-driven product development is diverse and ranges from requirements through the early phases to the detailed design of the product. The goal is to consistently analyze data to support and improve individual steps in the development process. In [...] Read more.

The field of application of data-driven product development is diverse and ranges from requirements through the early phases to the detailed design of the product. The goal is to consistently analyze data to support and improve individual steps in the development process. In the context of this work, the focus is on the design and detailing phase, represented by the virtual testing of products through Finite Element (FE) simulations. However, due to the heterogeneous data of a simulation model, automatic use is a big challenge. A method is therefore presented that utilizes the entire stock of calculated simulations to predict the plausibility of new simulations. Correspondingly, a large amount of data is utilized to support less experienced users of FE software in the application. Thus, obvious errors in the simulation should be detected immediately with this procedure and unnecessary iterations are therefore avoided. Previous solutions were only able to perform a general plausibility classification, whereas the approach presented in this paper is intended to predict specific error sources in FE simulations. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

22 pages, 3428 KiB

Open AccessArticle

Unsupervised Cyclic Siamese Networks Automating Cell Imagery Analysis

by Dominik Stallmann and Barbara Hammer

Algorithms 2023, 16(4), 205; https://doi.org/10.3390/a16040205 - 12 Apr 2023

Viewed by 1874

Abstract

Novel neural network models that can handle complex tasks with fewer examples than before are being developed for a wide range of applications. In some fields, even the creation of a few labels is a laborious task and impractical, especially for data that [...] Read more.

Novel neural network models that can handle complex tasks with fewer examples than before are being developed for a wide range of applications. In some fields, even the creation of a few labels is a laborious task and impractical, especially for data that require more than a few seconds to generate each label. In the biotechnological domain, cell cultivation experiments are usually done by varying the circumstances of the experiments, seldom in such a way that hand-labeled data of one experiment cannot be used in others. In this field, exact cell counts are required for analysis, and even by modern standards, semi-supervised models typically need hundreds of labels to achieve acceptable accuracy on this task, while classical image processing yields unsatisfactory results. We research whether an unsupervised learning scheme is able to accomplish this task without manual labeling of the given data. We present a VAE-based Siamese architecture that is expanded in a cyclic fashion to allow the use of labeled synthetic data. In particular, we focus on generating pseudo-natural images from synthetic images for which the target variable is known to mimic the existence of labeled natural data. We show that this learning scheme provides reliable estimates for multiple microscopy technologies and for unseen data sets without manual labeling. We provide the source code as well as the data we use. The code package is open source and free to use (MIT licensed). Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

26 pages, 6638 KiB

Open AccessArticle

How to Open a Black Box Classifier for Tabular Data

by Bradley Walters, Sandra Ortega-Martorell, Ivan Olier and Paulo J. G. Lisboa

Algorithms 2023, 16(4), 181; https://doi.org/10.3390/a16040181 - 27 Mar 2023

Cited by 5 | Viewed by 2514

Abstract

A lack of transparency in machine learning models can limit their application. We show that analysis of variance (ANOVA) methods extract interpretable predictive models from them. This is possible because ANOVA decompositions represent multivariate functions as sums of functions of fewer variables. Retaining [...] Read more.

A lack of transparency in machine learning models can limit their application. We show that analysis of variance (ANOVA) methods extract interpretable predictive models from them. This is possible because ANOVA decompositions represent multivariate functions as sums of functions of fewer variables. Retaining the terms in the ANOVA summation involving functions of only one or two variables provides an efficient method to open black box classifiers. The proposed method builds generalised additive models (GAMs) by application of L1 regularised logistic regression to the component terms retained from the ANOVA decomposition of the logit function. The resulting GAMs are derived using two alternative measures, Dirac and Lebesgue. Both measures produce functions that are smooth and consistent. The term partial responses in structured models (PRiSM) describes the family of models that are derived from black box classifiers by application of ANOVA decompositions. We demonstrate their interpretability and performance for the multilayer perceptron, support vector machines and gradient-boosting machines applied to synthetic data and several real-world data sets, namely Pima Diabetes, German Credit Card, and Statlog Shuttle from the UCI repository. The GAMs are shown to be compliant with the basic principles of a formal framework for interpretability. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

14 pages, 1399 KiB

Open AccessArticle

Framework for Evaluating Potential Causes of Health Risk Factors Using Average Treatment Effect and Uplift Modelling

by Daniela Galatro, Rosario Trigo-Ferre, Allana Nakashook-Zettler, Vincenzo Costanzo-Alvarez, Melanie Jeffrey, Maria Jacome, Jason Bazylak and Cristina H. Amon

Algorithms 2023, 16(3), 166; https://doi.org/10.3390/a16030166 - 19 Mar 2023

Cited by 1 | Viewed by 2524

Abstract

Acute myeloid leukemia (AML) is a type of blood cancer that affects both adults and children. Benzene exposure has been reported to increase the risk of developing AML in children. The assessment of the potential relationship between environmental benzene exposure and childhood has [...] Read more.

Acute myeloid leukemia (AML) is a type of blood cancer that affects both adults and children. Benzene exposure has been reported to increase the risk of developing AML in children. The assessment of the potential relationship between environmental benzene exposure and childhood has been documented in the literature using odds ratios and/or risk ratios, with data fitted to unconditional logistic regression. A common feature of the studies involving relationships between environmental risk factors and health outcomes is the lack of proper analysis to evidence causation. Although statistical causal analysis is commonly used to determine causation by evaluating a distribution’s parameters, it is challenging to infer causation in complex systems from single correlation coefficients. Machine learning (ML) approaches, based on causal pattern recognition, can provide an accurate alternative to model counterfactual scenarios. In this work, we propose a framework using average treatment effect (ATE) and Uplift modeling to evidence causation when relating exposure to benzene indoors and outdoors to childhood AML, effectively predicting causation when exposed indoors to this contaminant. An analysis of the assumptions, cross-validation, sample size, and interaction between predictors are also provided, guiding future works looking at the universalization of this approach in predicting health outcomes. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

17 pages, 1357 KiB

Open AccessArticle

Nearest Neighbours Graph Variational AutoEncoder

by Lorenzo Arsini, Barbara Caccia, Andrea Ciardiello, Stefano Giagu and Carlo Mancini Terracciano

Algorithms 2023, 16(3), 143; https://doi.org/10.3390/a16030143 - 6 Mar 2023

Cited by 1 | Viewed by 2549

Abstract

Graphs are versatile structures for the representation of many real-world data. Deep Learning on graphs is currently able to solve a wide range of problems with excellent results. However, both the generation of graphs and the handling of large graphs still remain open [...] Read more.

Graphs are versatile structures for the representation of many real-world data. Deep Learning on graphs is currently able to solve a wide range of problems with excellent results. However, both the generation of graphs and the handling of large graphs still remain open challenges. This work aims to introduce techniques for generating large graphs and test the approach on a complex problem such as the calculation of dose distribution in oncological radiotherapy applications. To this end, we introduced a pooling technique (ReNN-Pool) capable of sampling nodes that are spatially uniform without computational requirements in both model training and inference. By construction, the ReNN-Pool also allows the definition of a symmetric un-pooling operation to recover the original dimensionality of the graphs. We also present a Variational AutoEncoder (VAE) for generating graphs, based on the defined pooling and un-pooling operations, which employs convolutional graph layers in both encoding and decoding phases. The performance of the model was tested on both the realistic use case of a cylindrical graph dataset for a radiotherapy application and the standard benchmark dataset sprite. Compared to other graph pooling techniques, ReNN-Pool proved to improve both performance and computational requirements. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

14 pages, 916 KiB

Open AccessArticle

Acoustic Echo Cancellation with the Normalized Sign-Error Least Mean Squares Algorithm and Deep Residual Echo Suppression

by Eran Shachar, Israel Cohen and Baruch Berdugo

Algorithms 2023, 16(3), 137; https://doi.org/10.3390/a16030137 - 3 Mar 2023

Cited by 4 | Viewed by 2327

Abstract

This paper presents an echo suppression system that combines a linear acoustic echo canceller (AEC) with a deep complex convolutional recurrent network (DCCRN) for residual echo suppression. The filter taps of the AEC are adjusted in subbands by using the normalized sign-error least [...] Read more.

This paper presents an echo suppression system that combines a linear acoustic echo canceller (AEC) with a deep complex convolutional recurrent network (DCCRN) for residual echo suppression. The filter taps of the AEC are adjusted in subbands by using the normalized sign-error least mean squares (NSLMS) algorithm. The NSLMS is compared with the commonly-used normalized least mean squares (NLMS), and the combination of each with the proposed deep residual echo suppression model is studied. The utilization of a pre-trained deep-learning speech denoising model as an alternative to a residual echo suppressor (RES) is also studied. The results showed that the performance of the NSLMS is superior to that of the NLMS in all settings. With the NSLMS output, the proposed RES achieved better performance than the larger pre-trained speech denoiser model. More notably, the denoiser performed considerably better on the NSLMS output than on the NLMS output, and the performance gap was greater than the respective gap when employing the RES, indicating that the residual echo in the NSLMS output was more akin to noise than speech. Therefore, when little data is available to train an RES, a pre-trained speech denoiser is a viable alternative when employing the NSLMS for the preceding linear AEC. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

18 pages, 7764 KiB

Open AccessArticle

Fourier Neural Operator Network for Fast Photoacoustic Wave Simulations

by Steven Guan, Ko-Tsung Hsu and Parag V. Chitnis

Algorithms 2023, 16(2), 124; https://doi.org/10.3390/a16020124 - 19 Feb 2023

Cited by 8 | Viewed by 4096

Abstract

Simulation tools for photoacoustic wave propagation have played a key role in advancing photoacoustic imaging by providing quantitative and qualitative insights into parameters affecting image quality. Classical methods for numerically solving the photoacoustic wave equation rely on a fine discretization of space and [...] Read more.

Simulation tools for photoacoustic wave propagation have played a key role in advancing photoacoustic imaging by providing quantitative and qualitative insights into parameters affecting image quality. Classical methods for numerically solving the photoacoustic wave equation rely on a fine discretization of space and can become computationally expensive for large computational grids. In this work, we applied Fourier Neural Operator (FNO) networks as a fast data-driven deep learning method for solving the 2D photoacoustic wave equation in a homogeneous medium. Comparisons between the FNO network and pseudo-spectral time domain approach were made for the forward and adjoint simulations. Results demonstrate that the FNO network generated comparable simulations with small errors and was orders of magnitude faster than the pseudo-spectral time domain methods (~26× faster on a 64 × 64 computational grid and ~15× faster on a 128 × 128 computational grid). Moreover, the FNO network was generalizable to the unseen out-of-domain test set with a root-mean-square error of 9.5 × 10⁻³ in Shepp–Logan, 1.5 × 10⁻² in synthetic vasculature, 1.1 × 10⁻² in tumor and 1.9 × 10⁻² in Mason-M phantoms on a 64 × 64 computational grid and a root mean squared of 6.9 ± 5.5 × 10⁻³ in the AWA2 dataset on a 128 × 128 computational grid. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

21 pages, 1379 KiB

Open AccessArticle

Development and Implementation of an ANN Based Flow Law for Numerical Simulations of Thermo-Mechanical Processes at High Temperatures in FEM Software

by Olivier Pantalé

Algorithms 2023, 16(1), 56; https://doi.org/10.3390/a16010056 - 13 Jan 2023

Cited by 7 | Viewed by 2666

Abstract

Numerical methods based on finite element (FE) have proven their efficiency for many years in the thermomechanical simulation of forming processes. Nevertheless, the application of these methods to new materials requires the identification and implementation of constitutive and flow laws within FE codes, [...] Read more.

Numerical methods based on finite element (FE) have proven their efficiency for many years in the thermomechanical simulation of forming processes. Nevertheless, the application of these methods to new materials requires the identification and implementation of constitutive and flow laws within FE codes, which sometimes pose problems, particularly because of the strongly non-linear character of the behavior of these materials. Computational techniques based on machine learning and artificial neural networks are becoming more and more important in the development of these models and help the FE codes to integrate more complex behavior. In this paper, we present the development, implementation and use of an artificial neural network (ANN) based flow law for a GrC15 alloy under high temperature thermomechanical solicitations. The flow law modeling by ANN shows a significant superiority in terms of model prediction quality compared to classical approaches based on widely used Johnson–Cook or Arrhenius models. Once the ANN parameters have been identified on the base of experiments, the implementation of this flow law in a finite element code shows promising results in terms of solution quality and respect of the material behavior. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

19 pages, 2827 KiB

Open AccessArticle

Investigating Novice Developers’ Code Commenting Trends Using Machine Learning Techniques

by Tahira Niazi, Teerath Das, Ghufran Ahmed, Syed Muhammad Waqas, Sumra Khan, Suleman Khan, Ahmed Abdelaziz Abdelatif and Shaukat Wasi

Algorithms 2023, 16(1), 53; https://doi.org/10.3390/a16010053 - 12 Jan 2023

Cited by 3 | Viewed by 3304

Abstract

Code comments are considered an efficient way to document the functionality of a particular block of code. Code commenting is a common practice among developers to explain the purpose of the code in order to improve code comprehension and readability. Researchers investigated the [...] Read more.

Code comments are considered an efficient way to document the functionality of a particular block of code. Code commenting is a common practice among developers to explain the purpose of the code in order to improve code comprehension and readability. Researchers investigated the effect of code comments on software development tasks and demonstrated the use of comments in several ways, including maintenance, reusability, bug detection, etc. Given the importance of code comments, it becomes vital for novice developers to brush up on their code commenting skills. In this study, we initially investigated what types of comments novice students document in their source code and further categorized those comments using a machine learning approach. The work involves the initial manual classification of code comments and then building a machine learning model to classify student code comments automatically. The findings of our study revealed that novice developers/students’ comments are mainly related to Literal (26.66%) and Insufficient (26.66%). Further, we proposed and extended the taxonomy of such source code comments by adding a few more categories, i.e., License (5.18%), Profile (4.80%), Irrelevant (4.80%), Commented Code (4.44%), Autogenerated (1.48%), and Improper (1.10%). Moreover, we assessed our approach with three different machine-learning classifiers. Our implementation of machine learning models found that Decision Tree resulted in the overall highest accuracy, i.e., 85%. This study helps in predicting the type of code comments for a novice developer using a machine learning approach that can be implemented to generate automated feedback for students, thus saving teachers time for manual one-on-one feedback, which is a time-consuming activity. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

14 pages, 6999 KiB

Open AccessArticle

Fourier Neural Operator for Fluid Flow in Small-Shape 2D Simulated Porous Media Dataset

by Abouzar Choubineh, Jie Chen, David A. Wood, Frans Coenen and Fei Ma

Algorithms 2023, 16(1), 24; https://doi.org/10.3390/a16010024 - 1 Jan 2023

Cited by 8 | Viewed by 4677

Abstract

Machine Learning (ML) and/or Deep Learning (DL) methods can be used to predict fluid flow in porous media, as a suitable replacement for classical numerical approaches. Such data-driven approaches attempt to learn mappings between finite-dimensional Euclidean spaces. A novel neural framework, named Fourier [...] Read more.

Machine Learning (ML) and/or Deep Learning (DL) methods can be used to predict fluid flow in porous media, as a suitable replacement for classical numerical approaches. Such data-driven approaches attempt to learn mappings between finite-dimensional Euclidean spaces. A novel neural framework, named Fourier Neural Operator (FNO), has been recently developed to act on infinite-dimensional spaces. A high proportion of the research available on the FNO has focused on problems with large-shape data. Furthermore, most published studies apply the FNO method to existing datasets. This paper applies and evaluates FNO to predict pressure distribution over a small, specified shape-data problem using 1700 Finite Element Method (FEM) generated samples, from heterogeneous permeability fields as the input. Considering FEM-calculated outputs as the true values, the configured FNO model provides superior prediction performance to that of a Convolutional Neural Network (CNN) in terms of statistical error assessment based on the coefficient of determination (R

^{2}

) and Mean Squared Error (MSE). Sensitivity analysis considering a range of FNO configurations reveals that the most accurate model is obtained using

m o d e s = 15

and

w i d t h = 100

. Graphically, the FNO model precisely follows the observed trend in each porous medium evaluated. There is potential to further improve the FNO’s performance by including physics constraints in its network configuration. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

15 pages, 1390 KiB

Open AccessArticle

Predicting Dissolution Kinetics of Tricalcium Silicate Using Deep Learning and Analytical Models

by Taihao Han, Sai Akshay Ponduru, Arianit Reka, Jie Huang, Gaurav Sant and Aditya Kumar

Algorithms 2023, 16(1), 7; https://doi.org/10.3390/a16010007 - 22 Dec 2022

Cited by 3 | Viewed by 2684

Abstract

The dissolution kinetics of Portland cement is a critical factor in controlling the hydration reaction and improving the performance of concrete. Tricalcium silicate (C₃S), the primary phase in Portland cement, is known to have complex dissolution mechanisms that involve multiple reactions [...] Read more.

The dissolution kinetics of Portland cement is a critical factor in controlling the hydration reaction and improving the performance of concrete. Tricalcium silicate (C₃S), the primary phase in Portland cement, is known to have complex dissolution mechanisms that involve multiple reactions and changes to particle surfaces. As a result, current analytical models are unable to accurately predict the dissolution kinetics of C₃S in various solvents when it is undersaturated with respect to the solvent. This paper employs the deep forest (DF) model to predict the dissolution rate of C₃S in the undersaturated solvent. The DF model takes into account several variables, including the measurement method (i.e., reactor connected to inductive coupled plasma spectrometer and flow chamber with vertical scanning interferometry), temperature, and physicochemical properties of solvents. Next, the DF model evaluates the influence of each variable on the dissolution rate of C₃S, and this information is used to develop a closed-form analytical model that can predict the dissolution rate of C₃S. The coefficients and constant of the analytical model are optimized in two scenarios: generic and alkaline solvents. The results show that both the DF and analytical models are able to produce reliable predictions of the dissolution rate of C₃S when it is undersaturated and far from equilibrium. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

19 pages, 406 KiB

Open AccessArticle

RoSummary: Control Tokens for Romanian News Summarization

by Mihai Alexandru Niculescu, Stefan Ruseti and Mihai Dascalu

Algorithms 2022, 15(12), 472; https://doi.org/10.3390/a15120472 - 11 Dec 2022

Cited by 5 | Viewed by 2499

Abstract

Significant progress has been achieved in text generation due to recent developments in neural architectures; nevertheless, this task remains challenging, especially for low-resource languages. This study is centered on developing a model for abstractive summarization in Romanian. A corresponding dataset for summarization is [...] Read more.

Significant progress has been achieved in text generation due to recent developments in neural architectures; nevertheless, this task remains challenging, especially for low-resource languages. This study is centered on developing a model for abstractive summarization in Romanian. A corresponding dataset for summarization is introduced, followed by multiple models based on the Romanian GPT-2, on top of which control tokens were considered to specify characteristics for the generated text, namely: counts of sentences and words, token ratio, and n-gram overlap. These are special tokens defined in the prompt received by the model to indicate traits for the text to be generated. The initial model without any control tokens was assessed using BERTScore (

F_{1}

= 73.43%) and ROUGE (ROUGE-L accuracy = 34.67%). Control tokens improved the overall BERTScore to 75.42% using <LexOverlap>, while the model was influenced more by the second token specified in the prompt when performing various combinations of tokens. Six raters performed human evaluations of 45 generated summaries with different models and decoding methods. The generated texts were all grammatically correct and consistent in most cases, while the evaluations were promising in terms of main idea coverage, details, and cohesion. Paraphrasing still requires improvements as the models mostly repeat information from the reference text. In addition, we showcase an exploratory analysis of the generated summaries using one or two specific control tokens. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

15 pages, 5714 KiB

Open AccessArticle

Leverage Boosting and Transformer on Text-Image Matching for Cheap Fakes Detection

by Tuan-Vinh La, Minh-Son Dao, Duy-Dong Le, Kim-Phung Thai, Quoc-Hung Nguyen and Thuy-Kieu Phan-Thi

Algorithms 2022, 15(11), 423; https://doi.org/10.3390/a15110423 - 10 Nov 2022

Cited by 9 | Viewed by 3141

Abstract

The explosive growth of the social media community has increased many kinds of misinformation and is attracting tremendous attention from the research community. One of the most prevalent ways of misleading news is cheapfakes. Cheapfakes utilize non-AI techniques such as unaltered images with [...] Read more.

The explosive growth of the social media community has increased many kinds of misinformation and is attracting tremendous attention from the research community. One of the most prevalent ways of misleading news is cheapfakes. Cheapfakes utilize non-AI techniques such as unaltered images with false context news to create false news, which makes it easy and “cheap” to create and leads to an abundant amount in the social media community. Moreover, the development of deep learning also opens and invents many domains relevant to news such as fake news detection, rumour detection, fact-checking, and verification of claimed images. Nevertheless, despite the impact on and harmfulness of cheapfakes for the social community and the real world, there is little research on detecting cheapfakes in the computer science domain. It is challenging to detect misused/false/out-of-context pairs of images and captions, even with human effort, because of the complex correlation between the attached image and the veracity of the caption content. Existing research focuses mostly on training and evaluating on given dataset, which makes the proposal limited in terms of categories, semantics and situations based on the characteristics of the dataset. In this paper, to address these issues, we aimed to leverage textual semantics understanding from the large corpus and integrated with different combinations of text-image matching and image captioning methods via ANN/Transformer boosting schema to classify a triple of (image, caption₁, caption₂) into OOC (out-of-context) and NOOC (no out-of-context) labels. We customized these combinations according to various exceptional cases that we observed during data analysis. We evaluate our approach using the dataset and evaluation metrics provided by the COSMOS baseline. Compared to other methods, including the baseline, our method achieves the highest Accuracy, Recall, and F1 scores. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

18 pages, 2875 KiB

Open AccessEditor’s ChoiceArticle

Image-to-Image Translation-Based Data Augmentation for Improving Crop/Weed Classification Models for Precision Agriculture Applications

by L. G. Divyanth, D. S. Guru, Peeyush Soni, Rajendra Machavaram, Mohammad Nadimi and Jitendra Paliwal

Algorithms 2022, 15(11), 401; https://doi.org/10.3390/a15110401 - 30 Oct 2022

Cited by 31 | Viewed by 8373

Abstract

Applications of deep-learning models in machine visions for crop/weed identification have remarkably upgraded the authenticity of precise weed management. However, compelling data are required to obtain the desired result from this highly data-driven operation. This study aims to curtail the effort needed to [...] Read more.

Applications of deep-learning models in machine visions for crop/weed identification have remarkably upgraded the authenticity of precise weed management. However, compelling data are required to obtain the desired result from this highly data-driven operation. This study aims to curtail the effort needed to prepare very large image datasets by creating artificial images of maize (Zea mays) and four common weeds (i.e., Charlock, Fat Hen, Shepherd’s Purse, and small-flowered Cranesbill) through conditional Generative Adversarial Networks (cGANs). The fidelity of these synthetic images was tested through t-distributed stochastic neighbor embedding (t-SNE) visualization plots of real and artificial images of each class. The reliability of this method as a data augmentation technique was validated through classification results based on the transfer learning of a pre-defined convolutional neural network (CNN) architecture—the AlexNet; the feature extraction method came from the deepest pooling layer of the same network. Machine learning models based on a support vector machine (SVM) and linear discriminant analysis (LDA) were trained using these feature vectors. The F1 scores of the transfer learning model increased from 0.97 to 0.99, when additionally supported by an artificial dataset. Similarly, in the case of the feature extraction technique, the classification F1-scores increased from 0.93 to 0.96 for SVM and from 0.94 to 0.96 for the LDA model. The results show that image augmentation using generative adversarial networks (GANs) can improve the performance of crop/weed classification models with the added advantage of reduced time and manpower. Furthermore, it has demonstrated that generative networks could be a great tool for deep-learning applications in agriculture. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

17 pages, 3288 KiB

Open AccessArticle

Lithium-Ion Battery Prognostics through Reinforcement Learning Based on Entropy Measures

by Alireza Namdari, Maryam Asad Samani and Tariq S. Durrani

Algorithms 2022, 15(11), 393; https://doi.org/10.3390/a15110393 - 24 Oct 2022

Cited by 27 | Viewed by 4378

Abstract

Lithium-ion is a progressive battery technology that has been used in vastly different electrical systems. Failure of the battery can lead to failure in the entire system where the battery is embedded and cause irreversible damage. To avoid probable damages, research is actively [...] Read more.

Lithium-ion is a progressive battery technology that has been used in vastly different electrical systems. Failure of the battery can lead to failure in the entire system where the battery is embedded and cause irreversible damage. To avoid probable damages, research is actively conducted, and data-driven methods are proposed, based on prognostics and health management (PHM) systems. PHM can use multiple time-scale data and stored information from battery capacities over several cycles to determine the battery state of health (SOH) and its remaining useful life (RUL). This results in battery safety, stability, reliability, and longer lifetime. In this paper, we propose different data-driven approaches to battery prognostics that rely on: Long Short-Term Memory (LSTM), Autoregressive Integrated Moving Average (ARIMA), and Reinforcement Learning (RL) based on the permutation entropy of battery voltage sequences at each cycle, since they take into account vital information from past data and result in high accuracy. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

19 pages, 613 KiB

Open AccessArticle

Convolutional Neural Networks: A Roundup and Benchmark of Their Pooling Layer Variants

by Nikolaos-Ioannis Galanis, Panagiotis Vafiadis, Kostas-Gkouram Mirzaev and George A. Papakostas

Algorithms 2022, 15(11), 391; https://doi.org/10.3390/a15110391 - 23 Oct 2022

Cited by 10 | Viewed by 3919

Abstract

One of the essential layers in most Convolutional Neural Networks (CNNs) is the pooling layer, which is placed right after the convolution layer, effectively downsampling the input and reducing the computational power required. Different pooling methods have been proposed over the years, each [...] Read more.

One of the essential layers in most Convolutional Neural Networks (CNNs) is the pooling layer, which is placed right after the convolution layer, effectively downsampling the input and reducing the computational power required. Different pooling methods have been proposed over the years, each with its own advantages and disadvantages, rendering them a better fit for different applications. We introduce a benchmark between many of these methods that highlights an optimal choice for different scenarios depending on each project’s individual needs, whether it is detail retention, performance, or overall computational speed requirements. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

Review

Jump to: Research

30 pages, 3724 KiB

Open AccessEditor’s ChoiceReview

Defect Detection Methods for Industrial Products Using Deep Learning Techniques: A Review

by Alireza Saberironaghi, Jing Ren and Moustafa El-Gindy

Algorithms 2023, 16(2), 95; https://doi.org/10.3390/a16020095 - 8 Feb 2023

Cited by 90 | Viewed by 26362

Abstract

Over the last few decades, detecting surface defects has attracted significant attention as a challenging task. There are specific classes of problems that can be solved using traditional image processing techniques. However, these techniques struggle with complex textures in backgrounds, noise, and differences [...] Read more.

Over the last few decades, detecting surface defects has attracted significant attention as a challenging task. There are specific classes of problems that can be solved using traditional image processing techniques. However, these techniques struggle with complex textures in backgrounds, noise, and differences in lighting conditions. As a solution to this problem, deep learning has recently emerged, motivated by two main factors: accessibility to computing power and the rapid digitization of society, which enables the creation of large databases of labeled samples. This review paper aims to briefly summarize and analyze the current state of research on detecting defects using machine learning methods. First, deep learning-based detection of surface defects on industrial products is discussed from three perspectives: supervised, semi-supervised, and unsupervised. Secondly, the current research status of deep learning defect detection methods for X-ray images is discussed. Finally, we summarize the most common challenges and their potential solutions in surface defect detection, such as unbalanced sample identification, limited sample size, and real-time processing. Full article

(This article belongs to the Special Issue Deep Learning Architecture and Applications)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Deep Learning Architecture and Applications

Share This Special Issue

Special Issue Editors

Special Issue Information

Benefits of Publishing in a Special Issue

Published Papers (21 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI