Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.3
2.6
Journals
Electronics
Special Issues
Regularization Techniques for Machine Learning and Their Applications
share announcement

Regularization Techniques for Machine Learning and Their Applications

A special issue of Electronics (ISSN 2079-9292). This special issue belongs to the section "Artificial Intelligence".

Deadline for manuscript submissions: closed (31 December 2021) | Viewed by 33120

Special Issue Editors

Prof. Dr. Theodore Kotsilieris

E-Mail Website
Guest Editor
Department of Business Administration, University of the Peloponnese, GR 241-00 Kalamata, Greece
Interests: mobile agents; WSN routing algorithms; medical informatics; artificial intelligence; E-learning
Special Issues, Collections and Topics in MDPI journals
Dr. Ioannis E. Livieris

E-Mail Website
Guest Editor
1. Department of Business Administration, University of the Peloponnese, GR 241-00 Kalamata, Greece
2. Department of Mathematics, University of Patras, GR 265-00 Patras, Greece
Interests: artificial neural networks; numerical analysis; computational mathematics; machine learning; algorithms; semi-supervised learning; ICT in education; data mining; deep learning
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Ioannis Anagnostopoulos

E-Mail Website
Guest Editor
Department of Computer Science and Biomedical Informatics, University of Thessaly, 351-00 Lamia, Greece
Interests: internet technologies; health information systems; data management in bioinformatics; semantic interoperability; linked data
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

We invite you to submit your latest research in the development of ensemble algorithms to this Special Issue, “Regularization Techniques for Machine Learning and Their Applications”.

Over the last decade, learning theory has led to the achievement of significant progress in the development of sophisticated algorithms and their theoretical foundations. The theory builds on concepts which exploit ideas and methodologies from mathematical areas, such as optimization theory. Regularization is probably the key to address the challenging problem of overfitting, which usually occurs in high-dimensional learning. Its primary goal is to make the machine learning algorithm “learn” and not “memorize” by penalizing the algorithm to reduce its generalization error in order to avoid the risk of overfitting. As a result, the variance of the model is significantly reduced, without substantial increase in its bias and without losing any important properties in the data. 

The main aim of this Special Issue is to present the recent advances related to all kinds of regularization methodologies and investigations of the impact of their application to a diversity of real-world problems.

Prof. Dr. Theodore Kotsilieris
Dr. Ioannis E. Livieris
Prof. Dr. Ioannis Anagnostopoulos
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Regularized neural networks
  • Dropout & Dropconnect techniques
  • Regularization for deep learning models
  • Weight-constrained neural networks
  • L-norm regularization
  • Adversarial learning
  • Penalty functions
  • Multitask learning
  • Pooling techniques
  • Model selection techniques
  • Matrix regularizers
  • Data augmentation
  • Early stopping strategies

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (7 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Editorial

Jump to: Research

3 pages, 167 KiB  
Editorial
Special Issue: Regularization Techniques for Machine Learning and Their Applications
by Theodore Kotsilieris, Ioannis Anagnostopoulos and Ioannis E. Livieris
Electronics 2022, 11(4), 521; https://doi.org/10.3390/electronics11040521 - 10 Feb 2022
Cited by 3 | Viewed by 2215
Abstract
Over the last decade, learning theory performed significant progress in the development of sophisticated algorithms and their theoretical foundations. The theory builds on concepts that exploit ideas and methodologies from mathematical areas such as optimization theory. Regularization is probably the key to address [...] Read more.
Over the last decade, learning theory performed significant progress in the development of sophisticated algorithms and their theoretical foundations. The theory builds on concepts that exploit ideas and methodologies from mathematical areas such as optimization theory. Regularization is probably the key to address the challenging problem of overfitting, which usually occurs in high-dimensional learning. Its primary goal is to make the machine learning algorithm “learn” and not “memorize” by penalizing the algorithm to reduce its generalization error in order to avoid the risk of overfitting. As a result, the variance of the model is significantly reduced, without substantial increase in its bias and without losing any important properties in the data. Full article
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)

Research

Jump to: Editorial

21 pages, 4566 KiB  
Article
A Regularized Procedure to Generate a Deep Learning Model for Topology Optimization of Electromagnetic Devices
by Mauro Tucci, Sami Barmada, Alessandro Formisano and Dimitri Thomopulos
Electronics 2021, 10(18), 2185; https://doi.org/10.3390/electronics10182185 - 7 Sep 2021
Cited by 14 | Viewed by 2495
Abstract
The use of behavioral models based on deep learning (DL) to accelerate electromagnetic field computations has recently been proposed to solve complex electromagnetic problems. Such problems usually require time-consuming numerical analysis, while DL allows achieving the topologically optimized design of electromagnetic devices using [...] Read more.
The use of behavioral models based on deep learning (DL) to accelerate electromagnetic field computations has recently been proposed to solve complex electromagnetic problems. Such problems usually require time-consuming numerical analysis, while DL allows achieving the topologically optimized design of electromagnetic devices using desktop class computers and reasonable computation times. An unparametrized bitmap representation of the geometries to be optimized, which is a highly desirable feature needed to discover completely new solutions, is perfectly managed by DL models. On the other hand, optimization algorithms do not easily cope with high dimensional input data, particularly because it is difficult to enforce the searched solutions as feasible and make them belong to expected manifolds. In this work, we propose the use of a variational autoencoder as a data regularization/augmentation tool in the context of topology optimization. The optimization was carried out using a gradient descent algorithm, and the DL neural network was used as a surrogate model to accelerate the resolution of single trial cases in the due course of optimization. The variational autoencoder and the surrogate model were simultaneously trained in a multi-model custom training loop that minimizes total loss—which is the combination of the two models’ losses. In this paper, using the TEAM 25 problem (a benchmark problem for the assessment of electromagnetic numerical field analysis) as a test bench, we will provide a comparison between the computational times and design quality for a “classical” approach and the DL-based approach. Preliminary results show that the variational autoencoder manages regularizing the resolution process and transforms a constrained optimization into an unconstrained one, improving both the quality of the final solution and the performance of the resolution process. Full article
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)
►▼ Show Figures

Figure 1

21 pages, 1474 KiB  
Article
A Regularization-Based Big Data Framework for Winter Precipitation Forecasting on Streaming Data
by Andreas Kanavos, Maria Trigka, Elias Dritsas, Gerasimos Vonitsanos and Phivos Mylonas
Electronics 2021, 10(16), 1872; https://doi.org/10.3390/electronics10161872 - 4 Aug 2021
Cited by 10 | Viewed by 2511
Abstract
In the current paper, we propose a machine learning forecasting model for the accurate prediction of qualitative weather information on winter precipitation types, utilized in Apache Spark Streaming distributed framework. The proposed model receives storage and processes data in real-time, in order to [...] Read more.
In the current paper, we propose a machine learning forecasting model for the accurate prediction of qualitative weather information on winter precipitation types, utilized in Apache Spark Streaming distributed framework. The proposed model receives storage and processes data in real-time, in order to extract useful knowledge from different sensors related to weather data. In following, the numerical weather prediction model aims at forecasting the weather type given three precipitation classes namely rain, freezing rain, and snow as recorded in the Automated Surface Observing System (ASOS) network. For depicting the effectiveness of our proposed schema, a regularization technique for feature selection so as to avoid overfitting is implemented. Several classification models covering three different categorization methods namely the Bayesian, decision trees, and meta/ensemble methods, have been investigated in a real dataset. The experimental analysis illustrates that the utilization of the regularization technique could offer a significant boost in forecasting performance. Full article
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)
►▼ Show Figures

Figure 1

23 pages, 24767 KiB  
Article
Application of Deep Neural Network to the Reconstruction of Two-Phase Material Imaging by Capacitively Coupled Electrical Resistance Tomography
by Zhuoran Chen, Gege Ma, Yandan Jiang, Baoliang Wang and Manuchehr Soleimani
Electronics 2021, 10(9), 1058; https://doi.org/10.3390/electronics10091058 - 29 Apr 2021
Cited by 17 | Viewed by 2557
Abstract
A convolutional neural network (CNN)-based image reconstruction algorithm for two-phase material imaging is presented and verified with experimental data from a capacitively coupled electrical resistance tomography (CCERT) sensor. As a contactless version of electrical resistance tomography (ERT), CCERT has advantages such as no [...] Read more.
A convolutional neural network (CNN)-based image reconstruction algorithm for two-phase material imaging is presented and verified with experimental data from a capacitively coupled electrical resistance tomography (CCERT) sensor. As a contactless version of electrical resistance tomography (ERT), CCERT has advantages such as no invasion, low cost, no radiation, and rapid response for two-phase material imaging. Besides that, CCERT avoids contact error of ERT by imaging from outside of the pipe. Forward modeling was implemented based on the practical circular array sensor, and the inverse image reconstruction was realized by a CNN-based supervised learning algorithm, as well as the well-known total variation (TV) regularization algorithm for comparison. The 2D, monochrome, 2500-pixel image was divided into 625 clusters, and each cluster was used individually to train its own CNN to solve the 16 classes classification problem. Inherent regularization for the assumption of binary materials enabled us to use a classification algorithm with CNN. The iterative TV regularization algorithm achieved a close state of the two-phase material reconstruction by its sparsity-based assumption. The supervised learning algorithm established the mathematical model that mapped the simulated resistance measurement to the pixel patterns of the clusters. The training process was carried out only using simulated measurement data, but simulated and experimental tests were both conducted to investigate the feasibility of applying a multi-layer CNN for CCERT imaging. The performance of the CNN algorithm on the simulated data is demonstrated, and the comparison between the results created by the TV-based algorithm and the proposed CNN algorithm with the real-world data is also provided. Full article
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)
►▼ Show Figures

Figure 1

16 pages, 7646 KiB  
Article
An Advanced CNN-LSTM Model for Cryptocurrency Forecasting
by Ioannis E. Livieris, Niki Kiriakidou, Stavros Stavroyiannis and Panagiotis Pintelas
Electronics 2021, 10(3), 287; https://doi.org/10.3390/electronics10030287 - 26 Jan 2021
Cited by 90 | Viewed by 15383
Abstract
Nowadays, cryptocurrencies are established and widely recognized as an alternative exchange currency method. They have infiltrated most financial transactions and as a result cryptocurrency trade is generally considered one of the most popular and promising types of profitable investments. Nevertheless, this constantly increasing [...] Read more.
Nowadays, cryptocurrencies are established and widely recognized as an alternative exchange currency method. They have infiltrated most financial transactions and as a result cryptocurrency trade is generally considered one of the most popular and promising types of profitable investments. Nevertheless, this constantly increasing financial market is characterized by significant volatility and strong price fluctuations over a short-time period therefore, the development of an accurate and reliable forecasting model is considered essential for portfolio management and optimization. In this research, we propose a multiple-input deep neural network model for the prediction of cryptocurrency price and movement. The proposed forecasting model utilizes as inputs different cryptocurrency data and handles them independently in order to exploit useful information from each cryptocurrency separately. An extensive empirical study was performed using three consecutive years of cryptocurrency data from three cryptocurrencies with the highest market capitalization i.e., Bitcoin (BTC), Etherium (ETH), and Ripple (XRP). The detailed experimental analysis revealed that the proposed model has the ability to efficiently exploit mixed cryptocurrency data, reduces overfitting and decreases the computational cost in comparison with traditional fully-connected deep neural networks. Full article
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)
►▼ Show Figures

Figure 1

24 pages, 1718 KiB  
Article
An Advanced Pruning Method in the Architecture of Extreme Learning Machines Using L1-Regularization and Bootstrapping
by Paulo Vitor de Campos Souza, Luiz Carlos Bambirra Torres, Gustavo Rodrigues Lacerda Silva, Antonio de Padua Braga and Edwin Lughofer
Electronics 2020, 9(5), 811; https://doi.org/10.3390/electronics9050811 - 15 May 2020
Cited by 14 | Viewed by 3371
Abstract
Extreme learning machines (ELMs) are efficient for classification, regression, and time series prediction, as well as being a clear solution to backpropagation structures to determine values in intermediate layers of the learning model. One of the problems that an ELM may face is [...] Read more.
Extreme learning machines (ELMs) are efficient for classification, regression, and time series prediction, as well as being a clear solution to backpropagation structures to determine values in intermediate layers of the learning model. One of the problems that an ELM may face is due to a large number of neurons in the hidden layer, making the expert model a specific data set. With a large number of neurons in the hidden layer, overfitting is more likely and thus unnecessary information can deterioriate the performance of the neural network. To solve this problem, a pruning method is proposed, called Pruning ELM Using Bootstrapped Lasso BR-ELM, which is based on regularization and resampling techniques, to select the most representative neurons for the model response. This method is based on an ensembled variant of Lasso (achieved through bootstrap replications) and aims to shrink the output weight parameters of the neurons to 0 as many and as much as possible. According to a subset of candidate regressors having significant coefficient values (greater than 0), it is possible to select the best neurons in the hidden layer of the ELM. Finally, pattern classification tests and benchmark regression tests of complex real-world problems are performed by comparing the proposed approach to other pruning models for ELMs. It can be seen that statistically BR-ELM can outperform several related state-of-the-art methods in terms of classification accuracies and model errors (while performing equally to Pruning-ELM P-ELM), and this with a significantly reduced number of finally selected neurons. Full article
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)
►▼ Show Figures

Figure 1

14 pages, 2457 KiB  
Article
sDeepFM: Multi-Scale Stacking Feature Interactions for Click-Through Rate Prediction
by Baohua Qiang, Yongquan Lu, Minghao Yang, Xianjun Chen, Jinlong Chen and Yawei Cao
Electronics 2020, 9(2), 350; https://doi.org/10.3390/electronics9020350 - 19 Feb 2020
Cited by 5 | Viewed by 2711
Abstract
For estimating the click-through rate of advertisements, there are some problems in that the features cannot be automatically constructed, or the features built are relatively simple, or the high-order combination features are difficult to learn under sparse data. To solve these problems, we [...] Read more.
For estimating the click-through rate of advertisements, there are some problems in that the features cannot be automatically constructed, or the features built are relatively simple, or the high-order combination features are difficult to learn under sparse data. To solve these problems, we propose a novel structure multi-scale stacking pooling (MSSP) to construct multi-scale features based on different receptive fields. The structure stacks multi-scale features bi-directionally from the angles of depth and width by constructing multiple observers with different angles and different fields of view, ensuring the diversity of extracted features. Furthermore, by learning the parameters through factorization, the structure can ensure high-order features being effectively learned in sparse data. We further combine the MSSP with the classical deep neural network (DNN) to form a unified model named sDeepFM. Experimental results on two real-world datasets show that the sDeepFM outperforms state-of-the-art models with respect to area under the curve (AUC) and log loss. Full article
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-7
Back to TopTop