Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.9
2.1
Journals
Entropy
Special Issues
Recent Advances in Entropy and Divergence Measures, with Applications in...
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Recent Advances in Entropy and Divergence Measures, with Applications in Statistics and Machine Learning

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (31 October 2023) | Viewed by 10992

Special Issue Editors

Dr. Elena Castilla

E-Mail Website
Guest Editor
Department of Applied Mathematics, Rey Juan Carlos University, Mostoles, 28933 Madrid, Spain
Interests: information theory; categorical data analysis; composite likelihood; logistic regression models; reliability analysis and robust statistics
Dr. Abhik Ghosh

E-Mail Website
Guest Editor
Interdisciplinary Statistical Research Unit, Indian Statistical Institute, Kolkata 700108, India
Interests: generalized entropy and divergences; minimum divergence inference; robust statistics; inter-relation between statistics and information theory; high-dimensional statistics; biostatistics

Special Issue Information

Dear Colleagues,

The past thirty years have seen increasingly rapid advances in the field of statistical information theory. Particularly, maximum-likelihood-based methods are being replaced by alternative procedures based on divergence measures, having improved robustness properties under data contamination with only little or no loss in asymptotic efficiency. This idea has recently been applied in different types of statistical models and data from several applied domains of research. Furthermore, the use of Kullback–Leibler divergence plays an essential role in the construction of model selection criteria. Alternative (robust) criteria have also recently been developed based on divergence measures. Moreover, information theory has also been seen to be crucial in the development of efficient statistical machine learning algorithms and methods.

This Special Issue presents new developments in the field of statistical information theory based on generalized entropy and divergence measures, as well as their applications in data analysis and machine learning. We welcome both novel methodological and application-focused research contributions that utilize suitable new or existing entropy or divergence measures. Some particularly illustrative areas of interest include (but are not limited to): robustness, survival analysis and reliability, regression models, model selection, high-dimensional data analyses, Bayesian information theory, machine learning, estimating information–theoretic quantities, and applications of information theory for studying social networks.

Dr. Elena Castilla
Dr. Abhik Ghosh
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • information theory
  • divergence measures
  • generalized entropy
  • Bayesian statistics
  • regression models
  • machine learning
  • reliability
  • survival data analyses
  • robustness
  • model selection

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (5 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

18 pages, 423 KiB  
Article
Some Theoretical Foundations of Bare-Simulation Optimization of Some Directed Distances between Fuzzy Sets Respectively Basic Belief Assignments
by Michel Broniatowski and Wolfgang Stummer
Entropy 2024, 26(4), 312; https://doi.org/10.3390/e26040312 - 1 Apr 2024
Viewed by 1080
Abstract
It is well known that in information theory—as well as in the adjacent fields of statistics, machine learning and artificial intelligence—it is essential to quantify the dissimilarity between objects of uncertain/imprecise/inexact/vague information; correspondingly, constrained optimization is of great importance, too. In view of [...] Read more.
It is well known that in information theory—as well as in the adjacent fields of statistics, machine learning and artificial intelligence—it is essential to quantify the dissimilarity between objects of uncertain/imprecise/inexact/vague information; correspondingly, constrained optimization is of great importance, too. In view of this, we define the dissimilarity-measure-natured generalized φ–divergences between fuzzy sets, ν–rung orthopair fuzzy sets, extended representation type ν–rung orthopair fuzzy sets as well as between those fuzzy set types and vectors. For those, we present how to tackle corresponding constrained minimization problems by appropriately applying our recently developed dimension-free bare (pure) simulation method. An analogous program is carried out by defining and optimizing generalized φ–divergences between (rescaled) basic belief assignments as well as between (rescaled) basic belief assignments and vectors. Full article
(This article belongs to the Special Issue Recent Advances in Entropy and Divergence Measures, with Applications in Statistics and Machine Learning)
13 pages, 793 KiB  
Article
Slope Entropy Characterisation: An Asymmetric Approach to Threshold Parameters Role Analysis
by Mahdy Kouka, David Cuesta-Frau and Vicent Moltó-Gallego
Entropy 2024, 26(1), 82; https://doi.org/10.3390/e26010082 - 18 Jan 2024
Viewed by 1382
Abstract
Slope Entropy (SlpEn) is a novel method recently proposed in the field of time series entropy estimation. In addition to the well-known embedded dimension parameter, m, used in other methods, it applies two additional thresholds, denoted as δ and γ, to [...] Read more.
Slope Entropy (SlpEn) is a novel method recently proposed in the field of time series entropy estimation. In addition to the well-known embedded dimension parameter, m, used in other methods, it applies two additional thresholds, denoted as δ and γ, to derive a symbolic representation of a data subsequence. The original paper introducing SlpEn provided some guidelines for recommended specific values of these two parameters, which have been successfully followed in subsequent studies. However, a deeper understanding of the role of these thresholds is necessary to explore the potential for further SlpEn optimisations. Some works have already addressed the role of δ, but in this paper, we extend this investigation to include the role of γ and explore the impact of using an asymmetric scheme to select threshold values. We conduct a comparative analysis between the standard SlpEn method as initially proposed and an optimised version obtained through a grid search to maximise signal classification performance based on SlpEn. The results confirm that the optimised version achieves higher time series classification accuracy, albeit at the cost of significantly increased computational complexity. Full article
(This article belongs to the Special Issue Recent Advances in Entropy and Divergence Measures, with Applications in Statistics and Machine Learning)
►▼ Show Figures

Figure 1

26 pages, 1507 KiB  
Article
Entropy Estimators for Markovian Sequences: A Comparative Analysis
by Juan De Gregorio, David Sánchez and Raúl Toral
Entropy 2024, 26(1), 79; https://doi.org/10.3390/e26010079 - 17 Jan 2024
Cited by 4 | Viewed by 1974
Abstract
Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy [...] Read more.
Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy estimators are designed for sequences of independent events and their performances vary depending on the system being studied and the available data size. In this work, we compare different entropy estimators and their performance when applied to Markovian sequences. Specifically, we analyze both binary Markovian sequences and Markovian systems in the undersampled regime. We calculate the bias, standard deviation, and mean squared error for some of the most widely employed estimators. We discuss the limitations of entropy estimation as a function of the transition probabilities of the Markov processes and the sample size. Overall, this paper provides a comprehensive comparison of entropy estimators and their performance in estimating entropy for systems with memory, which can be useful for researchers and practitioners in various fields. Full article
(This article belongs to the Special Issue Recent Advances in Entropy and Divergence Measures, with Applications in Statistics and Machine Learning)
►▼ Show Figures

Figure 1

16 pages, 467 KiB  
Article
Robust Minimum Divergence Estimation for the Multinomial Circular Logistic Regression Model
by Elena Castilla and Abhik Ghosh
Entropy 2023, 25(10), 1422; https://doi.org/10.3390/e25101422 - 7 Oct 2023
Viewed by 1031
Abstract
Circular data are extremely important in many different contexts of natural and social science, from forestry to sociology, among many others. Since the usual inference procedures based on the maximum likelihood principle are known to be extremely non-robust in the presence of possible [...] Read more.
Circular data are extremely important in many different contexts of natural and social science, from forestry to sociology, among many others. Since the usual inference procedures based on the maximum likelihood principle are known to be extremely non-robust in the presence of possible data contamination, in this paper, we develop robust estimators for the general class of multinomial circular logistic regression models involving multiple circular covariates. Particularly, we extend the popular density-power-divergence-based estimation approach for this particular set-up and study the asymptotic properties of the resulting estimators. The robustness of the proposed estimators is illustrated through extensive simulation studies and few important real data examples from forest science and meteorology. Full article
(This article belongs to the Special Issue Recent Advances in Entropy and Divergence Measures, with Applications in Statistics and Machine Learning)
►▼ Show Figures

Figure 1

18 pages, 3644 KiB  
Article
Spatiotemporal Transformer Neural Network for Time-Series Forecasting
by Yujie You, Le Zhang, Peng Tao, Suran Liu and Luonan Chen
Entropy 2022, 24(11), 1651; https://doi.org/10.3390/e24111651 - 14 Nov 2022
Cited by 9 | Viewed by 4153
Abstract
Predicting high-dimensional short-term time-series is a difficult task due to the lack of sufficient information and the curse of dimensionality. To overcome these problems, this study proposes a novel spatiotemporal transformer neural network (STNN) for efficient prediction of short-term time-series with three major [...] Read more.
Predicting high-dimensional short-term time-series is a difficult task due to the lack of sufficient information and the curse of dimensionality. To overcome these problems, this study proposes a novel spatiotemporal transformer neural network (STNN) for efficient prediction of short-term time-series with three major features. Firstly, the STNN can accurately and robustly predict a high-dimensional short-term time-series in a multi-step-ahead manner by exploiting high-dimensional/spatial information based on the spatiotemporal information (STI) transformation equation. Secondly, the continuous attention mechanism makes the prediction results more accurate than those of previous studies. Thirdly, we developed continuous spatial self-attention, temporal self-attention, and transformation attention mechanisms to create a bridge between effective spatial information and future temporal evolution information. Fourthly, we show that the STNN model can reconstruct the phase space of the dynamical system, which is explored in the time-series prediction. The experimental results demonstrate that the STNN significantly outperforms the existing methods on various benchmarks and real-world systems in the multi-step-ahead prediction of a short-term time-series. Full article
(This article belongs to the Special Issue Recent Advances in Entropy and Divergence Measures, with Applications in Statistics and Machine Learning)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-5
Back to TopTop