Entropy

49 pages, 10680 KiB

Open AccessArticle

Multivariate Time Series Information Bottleneck

by Denis Ullmann, Olga Taran and Slava Voloshynovskiy

Entropy 2023, 25(5), 831; https://doi.org/10.3390/e25050831 - 22 May 2023

Cited by 2 | Viewed by 2878

Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition into the trio of “trend, seasonality, noise”, by [...] Read more.

Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition into the trio of “trend, seasonality, noise”, by attempts to copy the functioning of human synapses, and more recently, by transformer models with self-attention on the temporal dimension. These models may find applications in finance and e-commerce, where any increase in performance of less than

1 %

has large monetary repercussions, they also have potential applications in natural language processing (NLP), medicine, and physics. To the best of our knowledge, the information bottleneck (IB) framework has not received significant attention in the context of TS or MTS analyses. One can demonstrate that a compression of the temporal dimension is key in the context of MTS. We propose a new approach with partial convolution, where a time sequence is encoded into a two-dimensional representation resembling images. Accordingly, we use the recent advances made in image extension to predict an unseen part of an image from a given one. We show that our model compares well with traditional TS models, has information–theoretical foundations, and can be easily extended to more dimensions than only time and space. An evaluation of our multiple time series–information bottleneck (MTS-IB) model proves its efficiency in electricity production, road traffic, and astronomical data representing solar activity, as recorded by NASA’s interface region imaging spectrograph (IRIS) satellite. Full article

(This article belongs to the Special Issue Information-Theoretic Methods in Deep Learning: Theory and Applications)

► Show Figures

Figure 1

49 pages, 4386 KiB

Open AccessFeature PaperArticle

Free Choice in Quantum Theory: A p-adic View

by Vladimir Anashin

Entropy 2023, 25(5), 830; https://doi.org/10.3390/e25050830 - 22 May 2023

Cited by 4 | Viewed by 1661

Abstract

In this paper, it is rigorously proven that since observational data (i.e., numerical values of physical quantities) are rational numbers only due to inevitably nonzero measurements errors, the conclusion about whether Nature at the smallest scales is discrete or continuous, random and chaotic, [...] Read more.

In this paper, it is rigorously proven that since observational data (i.e., numerical values of physical quantities) are rational numbers only due to inevitably nonzero measurements errors, the conclusion about whether Nature at the smallest scales is discrete or continuous, random and chaotic, or strictly deterministic, solely depends on experimentalist’s free choice of the metrics (real or p-adic) he chooses to process the observational data. The main mathematical tools are p-adic 1-Lipschitz maps (which therefore are continuous with respect to the p-adic metric). The maps are exactly the ones defined by sequential Mealy machines (rather than by cellular automata) and therefore are causal functions over discrete time. A wide class of the maps can naturally be expanded to continuous real functions, so the maps may serve as mathematical models of open physical systems both over discrete and over continuous time. For these models, wave functions are constructed, entropic uncertainty relation is proven, and no hidden parameters are assumed. The paper is motivated by the ideas of I. Volovich on p-adic mathematical physics, by G. ‘t Hooft’s cellular automaton interpretation of quantum mechanics, and to some extent, by recent papers on superdeterminism by J. Hance, S. Hossenfelder, and T. Palmer. Full article

(This article belongs to the Special Issue New Trends in Theoretical and Mathematical Physics)

► Show Figures

Figure 1

12 pages, 314 KiB

Open AccessArticle

Orthogonal Polynomials with Singularly Perturbed Freud Weights

by Chao Min and Liwei Wang

Entropy 2023, 25(5), 829; https://doi.org/10.3390/e25050829 - 22 May 2023

Cited by 2 | Viewed by 1188

Abstract

In this paper, we are concerned with polynomials that are orthogonal with respect to the singularly perturbed Freud weight functions. By using Chen and Ismail’s ladder operator approach, we derive the difference equations and differential-difference equations satisfied by the recurrence coefficients. We also [...] Read more.

In this paper, we are concerned with polynomials that are orthogonal with respect to the singularly perturbed Freud weight functions. By using Chen and Ismail’s ladder operator approach, we derive the difference equations and differential-difference equations satisfied by the recurrence coefficients. We also obtain the differential-difference equations and the second-order differential equations for the orthogonal polynomials, with the coefficients all expressed in terms of the recurrence coefficients. Full article

(This article belongs to the Special Issue Random Matrices: Theory and Applications)

35 pages, 1600 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Local Phase Transitions in a Model of Multiplex Networks with Heterogeneous Degrees and Inter-Layer Coupling

by Nedim Bayrakdar, Valerio Gemmetto and Diego Garlaschelli

Entropy 2023, 25(5), 828; https://doi.org/10.3390/e25050828 - 22 May 2023

Viewed by 1380

Abstract

Multilayer networks represent multiple types of connections between the same set of nodes. Clearly, a multilayer description of a system adds value only if the multiplex does not merely consist of independent layers. In real-world multiplexes, it is expected that the observed inter-layer [...] Read more.

Multilayer networks represent multiple types of connections between the same set of nodes. Clearly, a multilayer description of a system adds value only if the multiplex does not merely consist of independent layers. In real-world multiplexes, it is expected that the observed inter-layer overlap may result partly from spurious correlations arising from the heterogeneity of nodes, and partly from true inter-layer dependencies. It is therefore important to consider rigorous ways to disentangle these two effects. In this paper, we introduce an unbiased maximum entropy model of multiplexes with controllable intra-layer node degrees and controllable inter-layer overlap. The model can be mapped to a generalized Ising model, where the combination of node heterogeneity and inter-layer coupling leads to the possibility of local phase transitions. In particular, we find that node heterogeneity favors the splitting of critical points characterizing different pairs of nodes, leading to link-specific phase transitions that may, in turn, increase the overlap. By quantifying how the overlap can be increased by increasing either the intra-layer node heterogeneity (spurious correlation) or the strength of the inter-layer coupling (true correlation), the model allows us to disentangle the two effects. As an application, we show that the empirical overlap observed in the International Trade Multiplex genuinely requires a nonzero inter-layer coupling in its modeling, as it is not merely a spurious result of the correlation between node degrees across different layers. Full article

(This article belongs to the Special Issue Recent Trends and Developments in Econophysics)

► Show Figures

Figure 1

15 pages, 331 KiB

Open AccessArticle

A Kind of (t, n) Threshold Quantum Secret Sharing with Identity Authentication

by Depeng Meng, Zhihui Li, Shuangshuang Luo and Zhaowei Han

Entropy 2023, 25(5), 827; https://doi.org/10.3390/e25050827 - 22 May 2023

Cited by 4 | Viewed by 1395

Abstract

Quantum secret sharing (QSS) is an important branch of quantum cryptography. Identity authentication is a significant means to achieve information protection, which can effectively confirm the identity information of both communication parties. Due to the importance of information security, more and more communications [...] Read more.

Quantum secret sharing (QSS) is an important branch of quantum cryptography. Identity authentication is a significant means to achieve information protection, which can effectively confirm the identity information of both communication parties. Due to the importance of information security, more and more communications require identity authentication. We propose a d-level

(t, n)

threshold QSS scheme in which both sides of the communication use mutually unbiased bases for mutual identity authentication. In the secret recovery phase, the sharing of secrets that only the participant holds will not be disclosed or transmitted. Therefore, external eavesdroppers will not get any information about secrets at this phase. This protocol is more secure, effective, and practical. Security analysis shows that this scheme can effectively resist intercept–resend attacks, entangle–measure attacks, collusion attacks, and forgery attacks. Full article

(This article belongs to the Special Issue Advanced Technology in Quantum Cryptography)

► Show Figures

Figure 1

18 pages, 3785 KiB

Open AccessArticle

Infrared Image Caption Based on Object-Oriented Attention

by Junfeng Lv, Tian Hui, Yongfeng Zhi and Yuelei Xu

Entropy 2023, 25(5), 826; https://doi.org/10.3390/e25050826 - 22 May 2023

Cited by 4 | Viewed by 1737

Abstract

With the ongoing development of image technology, the deployment of various intelligent applications on embedded devices has attracted increased attention in the industry. One such application is automatic image captioning for infrared images, which involves converting images into text. This practical task is [...] Read more.

With the ongoing development of image technology, the deployment of various intelligent applications on embedded devices has attracted increased attention in the industry. One such application is automatic image captioning for infrared images, which involves converting images into text. This practical task is widely used in night security, as well as for understanding night scenes and other scenarios. However, due to the differences in image features and the complexity of semantic information, generating captions for infrared images remains a challenging task. From the perspective of deployment and application, to improve the correlation between descriptions and objects, we introduced the YOLOv6 and LSTM as encoder-decoder structure and proposed infrared image caption based on object-oriented attention. Firstly, to improve the domain adaptability of the detector, we optimized the pseudo-label learning process. Secondly, we proposed the object-oriented attention method to address the alignment problem between complex semantic information and embedded words. This method helps select the most crucial features of the object region and guides the caption model in generating words that are more relevant to the object. Our methods have shown good performance on the infrared image and can produce words explicitly associated with the object regions located by the detector. The robustness and effectiveness of the proposed methods were demonstrated through evaluation on various datasets, along with other state-of-the-art methods. Our approach achieved BLUE-4 scores of 31.6 and 41.2 on KAIST and Infrared City and Town datasets, respectively. Our approach provides a feasible solution for the deployment of embedded devices in industrial applications. Full article

(This article belongs to the Special Issue Pattern Recognition and Data Clustering in Information Theory)

► Show Figures

Figure 1

32 pages, 2311 KiB

Open AccessArticle

Approximating Functions with Approximate Privacy for Applications in Signal Estimation and Learning

by Naima Tasnim, Jafar Mohammadi, Anand D. Sarwate and Hafiz Imtiaz

Entropy 2023, 25(5), 825; https://doi.org/10.3390/e25050825 - 22 May 2023

Cited by 4 | Viewed by 1777

Abstract

Large corporations, government entities and institutions such as hospitals and census bureaus routinely collect our personal and sensitive information for providing services. A key technological challenge is designing algorithms for these services that provide useful results, while simultaneously maintaining the privacy of the [...] Read more.

Large corporations, government entities and institutions such as hospitals and census bureaus routinely collect our personal and sensitive information for providing services. A key technological challenge is designing algorithms for these services that provide useful results, while simultaneously maintaining the privacy of the individuals whose data are being shared. Differential privacy (DP) is a cryptographically motivated and mathematically rigorous approach for addressing this challenge. Under DP, a randomized algorithm provides privacy guarantees by approximating the desired functionality, leading to a privacy–utility trade-off. Strong (pure DP) privacy guarantees are often costly in terms of utility. Motivated by the need for a more efficient mechanism with better privacy–utility trade-off, we propose Gaussian FM, an improvement to the functional mechanism (FM) that offers higher utility at the expense of a weakened (approximate) DP guarantee. We analytically show that the proposed Gaussian FM algorithm can offer orders of magnitude smaller noise compared to the existing FM algorithms. We further extend our Gaussian FM algorithm to decentralized-data settings by incorporating the

CAPE

protocol and propose

capeFM

. Our method can offer the same level of utility as its centralized counterparts for a range of parameter choices. We empirically show that our proposed algorithms outperform existing state-of-the-art approaches on synthetic and real datasets. Full article

(This article belongs to the Special Issue Information-Theoretic Privacy in Retrieval, Computing, and Learning)

► Show Figures

Figure 1

21 pages, 453 KiB

Open AccessArticle

Winning a CHSH Game without Entangled Particles in a Finite Number of Biased Rounds: How Much Luck Is Needed?

by Christoph Gallus, Pawel Blasiak and Emmanuel M. Pothos

Entropy 2023, 25(5), 824; https://doi.org/10.3390/e25050824 - 21 May 2023

Cited by 1 | Viewed by 2091

Abstract

Quantum games, such as the CHSH game, are used to illustrate the puzzle and power of entanglement. These games are played over many rounds and in each round, the participants, Alice and Bob, each receive a question bit to which they each have [...] Read more.

Quantum games, such as the CHSH game, are used to illustrate the puzzle and power of entanglement. These games are played over many rounds and in each round, the participants, Alice and Bob, each receive a question bit to which they each have to give an answer bit, without being able to communicate during the game. When all possible classical answering strategies are analyzed, it is found that Alice and Bob cannot win more than

75 %

of the rounds. A higher percentage of wins arguably requires an exploitable bias in the random generation of the question bits or access to “non-local“ resources, such as entangled pairs of particles. However, in an actual game, the number of rounds has to be finite and question regimes may come up with unequal likelihood, so there is always a possibility that Alice and Bob win by pure luck. This statistical possibility has to be transparently analyzed for practical applications such as the detection of eavesdropping in quantum communication. Similarly, when Bell tests are used in macroscopic situations to investigate the connection strength between system components and the validity of proposed causal models, the available data are limited and the possible combinations of question bits (measurement settings) may not be controlled to occur with equal likelihood. In the present work, we give a fully self-contained proof for a bound on the probability to win a CHSH game by pure luck without making the usual assumption of only small biases in the random number generators. We also show bounds for the case of unequal probabilities based on results from McDiarmid and Combes and numerically illustrate certain exploitable biases. Full article

(This article belongs to the Special Issue Quantum Probability and Randomness IV)

► Show Figures

Figure 1

12 pages, 329 KiB

Open AccessArticle

Entropy of Financial Time Series Due to the Shock of War

by Ewa A. Drzazga-Szczȩśniak, Piotr Szczepanik, Adam Z. Kaczmarek and Dominik Szczȩśniak

Entropy 2023, 25(5), 823; https://doi.org/10.3390/e25050823 - 21 May 2023

Cited by 6 | Viewed by 1829

Abstract

The concept of entropy is not uniquely relevant to the statistical mechanics but, among others, it can play pivotal role in the analysis of a time series, particularly the stock market data. In this area, sudden events are especially interesting as they describe [...] Read more.

The concept of entropy is not uniquely relevant to the statistical mechanics but, among others, it can play pivotal role in the analysis of a time series, particularly the stock market data. In this area, sudden events are especially interesting as they describe abrupt data changes with potentially long-lasting effects. Here, we investigate the impact of such events on the entropy of financial time series. As a case study, we assume data of the Polish stock market, in the context of its main cumulative index, and discuss it for the finite time periods before and after outbreak of the 2022 Russian invasion of Ukraine. This analysis allows us to validate the entropy-based methodology in assessing changes in the market volatility, as driven by the extreme external factors. We show that some qualitative features of such market variations can be well captured in terms of the entropy. In particular, the discussed measure appears to highlight differences between data of the two considered timeframes in agreement with the character of their empirical distributions, which is not always the case in terms of the conventional standard deviation. Moreover, the entropy of cumulative index averages, qualitatively, the entropies of composing assets, suggesting capability for describing interdependencies between them. The entropy is also found to exhibit signatures of the upcoming extreme events. To this end, the role of recent war in shaping the current economic situation is briefly discussed. Full article

(This article belongs to the Special Issue Recent Trends and Developments in Econophysics)

► Show Figures

Figure 1

17 pages, 838 KiB

Open AccessArticle

Attribute-Based Verifiable Conditional Proxy Re-Encryption Scheme

by Yongli Tang, Minglu Jin, Hui Meng, Li Yang and Chengfu Zheng

Entropy 2023, 25(5), 822; https://doi.org/10.3390/e25050822 - 19 May 2023

Cited by 3 | Viewed by 1883

Abstract

There are mostly semi-honest agents in cloud computing, so agents may perform unreliable calculations during the actual execution process. In this paper, an attribute-based verifiable conditional proxy re-encryption (AB-VCPRE) scheme using a homomorphic signature is proposed to solve the problem that the current [...] Read more.

There are mostly semi-honest agents in cloud computing, so agents may perform unreliable calculations during the actual execution process. In this paper, an attribute-based verifiable conditional proxy re-encryption (AB-VCPRE) scheme using a homomorphic signature is proposed to solve the problem that the current attribute-based conditional proxy re-encryption (AB-CPRE) algorithm cannot detect the illegal behavior of the agent. The scheme implements robustness, that is the re-encryption ciphertext, can be verified by the verification server, showing that the received ciphertext is correctly converted by the agent from the original ciphertext, thus, meaning that illegal activities of agents can be effectively detected. In addition, the article demonstrates the reliability of the constructed AB-VCPRE scheme validation in the standard model, and proves that the scheme satisfies CPA security in the selective security model based on the learning with errors (LWE) assumption. Full article

(This article belongs to the Special Issue Information Security and Privacy: From IoT to IoV)

► Show Figures

Figure 1

15 pages, 2586 KiB

Open AccessArticle

TSFN: A Novel Malicious Traffic Classification Method Using BERT and LSTM

by Zhaolei Shi, Nurbol Luktarhan, Yangyang Song and Huixin Yin

Entropy 2023, 25(5), 821; https://doi.org/10.3390/e25050821 - 19 May 2023

Cited by 8 | Viewed by 3164

Abstract

Traffic classification is the first step in network anomaly detection and is essential to network security. However, existing malicious traffic classification methods have several limitations; for example, statistical-based methods are vulnerable to hand-designed features, and deep learning-based methods are vulnerable to the balance [...] Read more.

Traffic classification is the first step in network anomaly detection and is essential to network security. However, existing malicious traffic classification methods have several limitations; for example, statistical-based methods are vulnerable to hand-designed features, and deep learning-based methods are vulnerable to the balance and adequacy of data sets. In addition, the existing BERT-based malicious traffic classification methods only focus on the global features of traffic and ignore the time-series features of traffic. To address these problems, we propose a BERT-based Time-Series Feature Network (TSFN) model in this paper. The first is a Packet encoder module built by the BERT model, which completes the capture of global features of the traffic using the attention mechanism. The second is a temporal feature extraction module built by the LSTM model, which captures the time-series features of the traffic. Then, the global and time-series features of the malicious traffic are incorporated together as the final feature representation, which can better represent the malicious traffic. The experimental results show that the proposed approach can effectively improve the accuracy of malicious traffic classification on the publicly available USTC-TFC dataset, reaching an F1 value of 99.50%. This shows that the time-series features in malicious traffic can help improve the accuracy of malicious traffic classification. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

12 pages, 480 KiB

Open AccessArticle

TTANAD: Test-Time Augmentation for Network Anomaly Detection

by Seffi Cohen, Niv Goldshlager, Bracha Shapira and Lior Rokach

Entropy 2023, 25(5), 820; https://doi.org/10.3390/e25050820 - 19 May 2023

Viewed by 1949

Abstract

Machine learning-based Network Intrusion Detection Systems (NIDS) are designed to protect networks by identifying anomalous behaviors or improper uses. In recent years, advanced attacks, such as those mimicking legitimate traffic, have been developed to avoid alerting such systems. Previous works mainly focused on [...] Read more.

Machine learning-based Network Intrusion Detection Systems (NIDS) are designed to protect networks by identifying anomalous behaviors or improper uses. In recent years, advanced attacks, such as those mimicking legitimate traffic, have been developed to avoid alerting such systems. Previous works mainly focused on improving the anomaly detector itself, whereas in this paper, we introduce a novel method, Test-Time Augmentation for Network Anomaly Detection (TTANAD), which utilizes test-time augmentation to enhance anomaly detection from the data side. TTANAD leverages the temporal characteristics of traffic data and produces temporal test-time augmentations on the monitored traffic data. This method aims to create additional points of view when examining network traffic during inference, making it suitable for a variety of anomaly detector algorithms. Our experimental results demonstrate that TTANAD outperforms the baseline in all benchmark datasets and with all examined anomaly detection algorithms, according to the Area Under the Receiver Operating Characteristic (AUC) metric. Full article

(This article belongs to the Special Issue Signal and Information Processing in Networks)

► Show Figures

Figure 1

13 pages, 1435 KiB

Open AccessArticle

Modeling Exact Frequency-Energy Distribution for Quakes by a Probabilistic Cellular Automaton

by Mariusz Białecki, Mateusz Gałka, Arpan Bagchi and Jacek Gulgowski

Entropy 2023, 25(5), 819; https://doi.org/10.3390/e25050819 - 19 May 2023

Cited by 1 | Viewed by 1234

Abstract

We develop the notion of Random Domino Automaton, a simple probabilistic cellular automaton model for earthquake statistics, in order to provide a mechanistic basis for the interrelation of Gutenberg–Richter law and Omori law with the waiting time distribution for earthquakes. In this work, [...] Read more.

We develop the notion of Random Domino Automaton, a simple probabilistic cellular automaton model for earthquake statistics, in order to provide a mechanistic basis for the interrelation of Gutenberg–Richter law and Omori law with the waiting time distribution for earthquakes. In this work, we provide a general algebraic solution to the inverse problem for the model and apply the proposed procedure to seismic data recorded in the Legnica-Głogów Copper District in Poland, which demonstrate the adequacy of the method. The solution of the inverse problem enables adjustment of the model to localization-dependent seismic properties manifested by deviations from Gutenberg–Richter law. Full article

(This article belongs to the Special Issue Complexity and Nonlinear Dynamics in the Geosciences: Methods and Applications)

► Show Figures

Figure 1

18 pages, 16091 KiB

Open AccessArticle

Study of Generalized Chaotic Synchronization Method Incorporating Error-Feedback Coefficients

by Yanan Xing, Wenjie Dong, Jian Zeng, Pengteng Guo, Jing Zhang and Qun Ding

Entropy 2023, 25(5), 818; https://doi.org/10.3390/e25050818 - 18 May 2023

Cited by 3 | Viewed by 1522

Abstract

In this paper, taking the generalized synchronization problem of discrete chaotic systems as a starting point, a generalized synchronization method incorporating error-feedback coefficients into the controller based on the generalized chaos synchronization theory and stability theorem for nonlinear systems is proposed. Two discrete [...] Read more.

In this paper, taking the generalized synchronization problem of discrete chaotic systems as a starting point, a generalized synchronization method incorporating error-feedback coefficients into the controller based on the generalized chaos synchronization theory and stability theorem for nonlinear systems is proposed. Two discrete chaotic systems with different dimensions are constructed in this paper, the dynamics of the proposed systems are analyzed, and finally, the phase diagrams, Lyapunov exponent diagrams, and bifurcation diagrams of these are shown and described. The experimental results show that the design of the adaptive generalized synchronization system is achievable in cases in which the error-feedback coefficient satisfies certain conditions. Finally, a chaotic hiding image encryption transmission system based on a generalized synchronization approach is proposed, in which an error-feedback coefficient is introduced into the controller. Full article

(This article belongs to the Topic Advances in Nonlinear Dynamics: Methods and Applications)

► Show Figures

Figure 1

20 pages, 4794 KiB

Open AccessArticle

Swarming Transition in Super-Diffusive Self-Propelled Particles

by Morteza Nattagh Najafi, Rafe Md. Abu Zayed and Seyed Amin Nabavizadeh

Entropy 2023, 25(5), 817; https://doi.org/10.3390/e25050817 - 18 May 2023

Viewed by 1548

Abstract

A super-diffusive Vicsek model is introduced in this paper that incorporates Levy flights with exponent

α

. The inclusion of this feature leads to an increase in the fluctuations of the order parameter, ultimately resulting in the disorder phase becoming more dominant as [...] Read more.

A super-diffusive Vicsek model is introduced in this paper that incorporates Levy flights with exponent

α

. The inclusion of this feature leads to an increase in the fluctuations of the order parameter, ultimately resulting in the disorder phase becoming more dominant as

α

increases. The study finds that for

α

values close to two, the order–disorder transition is of the first order, while for small enough values of

α

, it shows degrees of similarities with the second-order phase transitions. The article formulates a mean field theory based on the growth of the swarmed clusters that accounts for the decrease in the transition point as

α

increases. The simulation results show that the order parameter exponent

β

, correlation length exponent

ν

, and susceptibility exponent

γ

remain constant when

α

is altered, satisfying a hyperscaling relation. The same happens for the mass fractal dimension, information dimension, and correlation dimension when

α

is far from two. The study reveals that the fractal dimension of the external perimeter of connected self-similar clusters conforms to the fractal dimension of Fortuin–Kasteleyn clusters of the two-dimensional

Q = 2

Potts (Ising) model. The critical exponents linked to the distribution function of global observables vary when

α

changes. Full article

(This article belongs to the Section Statistical Physics)

► Show Figures

Figure 1

16 pages, 2953 KiB

Open AccessArticle

On the Possibility of Reproducing Utsu’s Law for Earthquakes with a Spring-Block SOC Model

by Alfredo Salinas-Martínez, Jennifer Perez-Oregon, Ana María Aguilar-Molina, Alejandro Muñoz-Diosdado and Fernando Angulo-Brown

Entropy 2023, 25(5), 816; https://doi.org/10.3390/e25050816 - 18 May 2023

Cited by 4 | Viewed by 1299

Abstract

The Olami, Feder and Christensen (OFC) spring-block model has proven to be a powerful tool for analyzing and comparing synthetic and real earthquakes. This work proposes the possible reproduction of Utsu’s law for earthquakes in the OFC model. Based on our previous works, [...] Read more.

The Olami, Feder and Christensen (OFC) spring-block model has proven to be a powerful tool for analyzing and comparing synthetic and real earthquakes. This work proposes the possible reproduction of Utsu’s law for earthquakes in the OFC model. Based on our previous works, several simulations characterizing real seismic regions were performed. We located the maximum earthquake in these regions and applied Utsu’s formulae to identify a possible aftershock area and made comparisons between synthetic and real earthquakes. The research compares several equations to calculate the aftershock area and proposes a new one with the available data. Subsequently, the team performed new simulations and chose a mainshock to analyze the behavior of the surrounding events, so as to identify whether they could be catalogued as aftershocks and relate them to the aftershock area previously determined using the formula proposed. Additionally, the spatial location of those events was considered in order to classify them as aftershocks. Finally, we plot the epicenters of the mainshock, and the possible aftershocks comprised in the calculated area resembling the original work of Utsu. Having analyzed the results, it is likely to say that Utsu’s law is reproducible using a spring-block model with a self-organized criticality (SOC) model. Full article

(This article belongs to the Special Issue Complexity and Nonlinear Dynamics in the Geosciences: Methods and Applications)

► Show Figures

Figure 1

13 pages, 1762 KiB

Open AccessArticle

Self-Regulated Symmetry Breaking Model for Stem Cell Differentiation

by Madelynn McElroy, Kaylie Green and Nikolaos K. Voulgarakis

Entropy 2023, 25(5), 815; https://doi.org/10.3390/e25050815 - 18 May 2023

Cited by 1 | Viewed by 1867

Abstract

In conventional disorder–order phase transitions, a system shifts from a highly symmetric state, where all states are equally accessible (disorder) to a less symmetric state with a limited number of available states (order). This transition may occur by varying a control parameter that [...] Read more.

In conventional disorder–order phase transitions, a system shifts from a highly symmetric state, where all states are equally accessible (disorder) to a less symmetric state with a limited number of available states (order). This transition may occur by varying a control parameter that represents the intrinsic noise of the system. It has been suggested that stem cell differentiation can be considered as a sequence of such symmetry-breaking events. Pluripotent stem cells, with their capacity to develop into any specialized cell type, are considered highly symmetric systems. In contrast, differentiated cells have lower symmetry, as they can only carry out a limited number of functions. For this hypothesis to be valid, differentiation should emerge collectively in stem cell populations. Additionally, such populations must have the ability to self-regulate intrinsic noise and navigate through a critical point where spontaneous symmetry breaking (differentiation) occurs. This study presents a mean-field model for stem cell populations that considers the interplay of cell–cell cooperativity, cell-to-cell variability, and finite-size effects. By introducing a feedback mechanism to control intrinsic noise, the model can self-tune through different bifurcation points, facilitating spontaneous symmetry breaking. Standard stability analysis showed that the system can potentially differentiate into several cell types mathematically expressed as stable nodes and limit cycles. The existence of a Hopf bifurcation in our model is discussed in light of stem cell differentiation. Full article

(This article belongs to the Collection Disorder and Biological Physics)

► Show Figures

Figure 1

15 pages, 648 KiB

Open AccessArticle

Properties of Spherically Symmetric Black Holes in the Generalized Brans–Dicke Modified Gravitational Theory

by Mou Xu, Jianbo Lu, Shining Yang and Hongnan Jiang

Entropy 2023, 25(5), 814; https://doi.org/10.3390/e25050814 - 18 May 2023

Cited by 2 | Viewed by 1514

Abstract

The many problems faced by the theory of general relativity (GR) have always motivated us to explore the modified theory of GR. Considering the importance of studying the black hole (BH) entropy and its correction in gravity physics, we study the correction of [...] Read more.

The many problems faced by the theory of general relativity (GR) have always motivated us to explore the modified theory of GR. Considering the importance of studying the black hole (BH) entropy and its correction in gravity physics, we study the correction of thermodynamic entropy for a kind of spherically symmetric black hole under the generalized Brans–Dicke (GBD) theory of modified gravity. We derive and calculate the entropy and heat capacity. It is found that when the value of event horizon radius

r_{+}

is small, the effect of the entropy-correction term on the entropy is very obvious, while for larger values

r_{+}

, the contribution of the correction term on entropy can be almost ignored. In addition, we can observe that as the radius of the event horizon increases, the heat capacity of BH in GBD theory will change from a negative value to a positive value, indicating that there is a phase transition in black holes. Given that studying the structure of geodesic lines is important for exploring the physical characteristics of a strong gravitational field, we also investigate the stability of particles’ circular orbits in static spherically symmetric BHs within the framework of GBD theory. Concretely, we analyze the dependence of the innermost stable circular orbit on model parameters. In addition, the geodesic deviation equation is also applied to investigate the stable circular orbit of particles in GBD theory. The conditions for the stability of the BH solution and the limited range of radial coordinates required to achieve stable circular orbit motion are given. Finally, we show the locations of stable circular orbits, and obtain the angular velocity, specific energy, and angular momentum of the particles which move in circular orbits. Full article

(This article belongs to the Special Issue Advances in Black Hole Thermodynamics)

► Show Figures

Figure 1

12 pages, 325 KiB

Open AccessFeature PaperArticle

Forward and Backward Recalling Sequences in Spatial and Verbal Memory Tasks: What Do We Measure?

by Jeanette Melin, Laura Göschel, Peter Hagell, Albert Westergren, Agnes Flöel and Leslie Pendrill

Entropy 2023, 25(5), 813; https://doi.org/10.3390/e25050813 - 18 May 2023

Cited by 1 | Viewed by 1686

Abstract

There are different views in the literature about the number and inter-relationships of cognitive domains (such as memory and executive function) and a lack of understanding of the cognitive processes underlying these domains. In previous publications, we demonstrated a methodology for formulating and [...] Read more.

There are different views in the literature about the number and inter-relationships of cognitive domains (such as memory and executive function) and a lack of understanding of the cognitive processes underlying these domains. In previous publications, we demonstrated a methodology for formulating and testing cognitive constructs for visuo-spatial and verbal recall tasks, particularly for working memory task difficulty where entropy is found to play a major role. In the present paper, we applied those insights to a new set of such memory tasks, namely, backward recalling block tapping and digit sequences. Once again, we saw clear and strong entropy-based construct specification equations (CSEs) for task difficulty. In fact, the entropy contributions in the CSEs for the different tasks were of similar magnitudes (within the measurement uncertainties), which may indicate a shared factor in what is being measured with both forward and backward sequences, as well as visuo-spatial and verbal memory recalling tasks more generally. On the other hand, the analyses of dimensionality and the larger measurement uncertainties in the CSEs for the backward sequences suggest that caution is needed when attempting to unify a single unidimensional construct based on forward and backward sequences with visuo-spatial and verbal memory tasks. Full article

(This article belongs to the Special Issue Applications of Entropy in Health Care)

12 pages, 6428 KiB

Open AccessArticle

Evolutionary Method of Heterogeneous Combat Network Based on Link Prediction

by Shaoming Qiu, Fen Chen, Yahui Wang and Jiancheng Zhao

Entropy 2023, 25(5), 812; https://doi.org/10.3390/e25050812 - 17 May 2023

Viewed by 1361

Abstract

Currently, research on the evolution of heterogeneous combat networks (HCNs) mainly focuses on the modeling process, with little attention paid to the impact of changes in network topology on operational capabilities. Link prediction can provide a fair and unified comparison standard for network [...] Read more.

Currently, research on the evolution of heterogeneous combat networks (HCNs) mainly focuses on the modeling process, with little attention paid to the impact of changes in network topology on operational capabilities. Link prediction can provide a fair and unified comparison standard for network evolution mechanisms. This paper uses link prediction methods to study the evolution of HCNs. Firstly, according to the characteristics of HCNs, a link prediction index based on frequent subgraphs (LPFS) is proposed. LPFS have been demonstrated on a real combat network to be superior to 26 baseline methods. The main driving force of research on evolution is to improve the operational capabilities of combat networks. Adding the same number of nodes and edges, 100 iterative experiments demonstrate that the evolutionary method (HCNE) proposed in this paper outperforms random evolution and preferential evolution in improving the operational capabilities of combat networks. Furthermore, the new network generated after evolution is more consistent with the characteristics of a real network. Full article

(This article belongs to the Topic Complex Systems and Network Science)

► Show Figures

Figure 1

25 pages, 4135 KiB

Open AccessArticle

A Secure Scheme Based on a Hybrid of Classical-Quantum Communications Protocols for Managing Classical Blockchains

by Ang Liu, Xiu-Bo Chen, Shengwei Xu, Zhuo Wang, Zhengyang Li, Liwei Xu, Yanshuo Zhang and Ying Chen

Entropy 2023, 25(5), 811; https://doi.org/10.3390/e25050811 - 17 May 2023

Cited by 7 | Viewed by 1955

Abstract

Blockchain technology affords data integrity protection and building trust mechanisms in transactions for distributed networks, and, therefore, is seen as a promising revolutionary information technology. At the same time, the ongoing breakthrough in quantum computation technology contributes toward large-scale quantum computers, which might [...] Read more.

Blockchain technology affords data integrity protection and building trust mechanisms in transactions for distributed networks, and, therefore, is seen as a promising revolutionary information technology. At the same time, the ongoing breakthrough in quantum computation technology contributes toward large-scale quantum computers, which might attack classic cryptography, seriously threatening the classic cryptography security currently employed in the blockchain. As a better alternative, a quantum blockchain has high expectations of being immune to quantum computing attacks perpetrated by quantum adversaries. Although several works have been presented, the problems of impracticality and inefficiency in quantum blockchain systems remain prominent and need to be addressed. First, this paper develops a quantum-secure blockchain (QSB) scheme by introducing a consensus mechanism—quantum proof of authority (QPoA) and an identity-based quantum signature (IQS)—wherein QPoA is used for new block generation and IQS is used for transaction signing and verification. Second, QPoA is developed by adopting a quantum voting protocol to achieve secure and efficient decentralization for the blockchain system, and a quantum random number generator (QRNG) is deployed for randomized leader node election to protect the blockchain system from centralized attacks like distributed denial of service (DDoS). Compared to previous work, our scheme is more practical and efficient without sacrificing security, greatly contributing to better addressing the challenges in the quantum era. Extensive security analysis demonstrates that our scheme provides better protection against quantum computing attacks than classic blockchains. Overall, our scheme presents a feasible solution for blockchain systems against quantum computing attacks through a quantum strategy, contributing toward quantum-secured blockchain in the quantum era. Full article

(This article belongs to the Topic Secure Applications with Blockchain and Artificial Intelligence)

► Show Figures

Figure 1

18 pages, 15138 KiB

Open AccessArticle

Wasserstein Distance-Based Deep Leakage from Gradients

by Zifan Wang, Changgen Peng, Xing He and Weijie Tan

Entropy 2023, 25(5), 810; https://doi.org/10.3390/e25050810 - 17 May 2023

Cited by 1 | Viewed by 2209

Abstract

Federated learning protects the privacy information in the data set by sharing the average gradient. However, “Deep Leakage from Gradient” (DLG) algorithm as a gradient-based feature reconstruction attack can recover privacy training data using gradients shared in federated learning, resulting in private information [...] Read more.

Federated learning protects the privacy information in the data set by sharing the average gradient. However, “Deep Leakage from Gradient” (DLG) algorithm as a gradient-based feature reconstruction attack can recover privacy training data using gradients shared in federated learning, resulting in private information leakage. However, the algorithm has the disadvantages of slow model convergence and poor inverse generated images accuracy. To address these issues, a Wasserstein distance-based DLG method is proposed, named WDLG. The WDLG method uses Wasserstein distance as the training loss function achieved to improve the inverse image quality and the model convergence. The hard-to-calculate Wasserstein distance is converted to be calculated iteratively using the Lipschit condition and Kantorovich–Rubinstein duality. Theoretical analysis proves the differentiability and continuity of Wasserstein distance. Finally, experiment results show that the WDLG algorithm is superior to DLG in training speed and inversion image quality. At the same time, we prove through the experiments that differential privacy can be used for disturbance protection, which provides some ideas for the development of a deep learning framework to protect privacy. Full article

(This article belongs to the Special Issue Information Theory for Interpretable Machine Learning)

► Show Figures

Figure 1

14 pages, 2498 KiB

Open AccessArticle

Subdomain Adaptation Capsule Network for Partial Discharge Diagnosis in Gas-Insulated Switchgear

by Yanze Wu, Jing Yan, Zhuofan Xu, Guoqing Sui, Meirong Qi, Yingsan Geng and Jianhua Wang

Entropy 2023, 25(5), 809; https://doi.org/10.3390/e25050809 - 17 May 2023

Cited by 3 | Viewed by 1423

Abstract

Deep learning methods, especially convolutional neural networks (CNNs), have achieved good results in the partial discharge (PD) diagnosis of gas-insulated switchgear (GIS) in the laboratory. However, the relationship of features ignored in CNNs and the heavy dependance on the amount of sample data [...] Read more.

Deep learning methods, especially convolutional neural networks (CNNs), have achieved good results in the partial discharge (PD) diagnosis of gas-insulated switchgear (GIS) in the laboratory. However, the relationship of features ignored in CNNs and the heavy dependance on the amount of sample data make it difficult for the model developed in the laboratory to achieve high-precision, robust diagnosis of PD in the field. To solve these problems, a subdomain adaptation capsule network (SACN) is adopted for PD diagnosis in GIS. First, the feature information is effectively extracted by using a capsule network, which improves feature representation. Then, subdomain adaptation transfer learning is used to accomplish high diagnosis performance on the field data, which alleviates the confusion of different subdomains and matches the local distribution at the subdomain level. Experimental results demonstrate that the accuracy of the SACN in this study reaches 93.75% on the field data. The SACN has better performance than traditional deep learning methods, indicating that the SACN has potential application value in PD diagnosis of GIS. Full article

(This article belongs to the Special Issue Fault Diagnosis Methods Based on Information Theory or Machine Learning: From Theory to Application)

► Show Figures

Figure 1

19 pages, 7368 KiB

Open AccessArticle

MSIA-Net: A Lightweight Infrared Target Detection Network with Efficient Information Fusion

by Jimin Yu, Shun Li, Shangbo Zhou and Hui Wang

Entropy 2023, 25(5), 808; https://doi.org/10.3390/e25050808 - 17 May 2023

Cited by 6 | Viewed by 1745

Abstract

In order to solve the problems of infrared target detection (i.e., the large models and numerous parameters), a lightweight detection network, MSIA-Net, is proposed. Firstly, a feature extraction module named MSIA, which is based on asymmetric convolution, is proposed, and it can greatly [...] Read more.

In order to solve the problems of infrared target detection (i.e., the large models and numerous parameters), a lightweight detection network, MSIA-Net, is proposed. Firstly, a feature extraction module named MSIA, which is based on asymmetric convolution, is proposed, and it can greatly reduce the number of parameters and improve the detection performance by reusing information. In addition, we propose a down-sampling module named DPP to reduce the information loss caused by pooling down-sampling. Finally, we propose a feature fusion structure named LIR-FPN that can shorten the information transmission path and effectively reduce the noise in the process of feature fusion. In order to improve the ability of the network to focus on the target, we introduce coordinate attention (CA) into the LIR-FPN; this integrates the location information of the target into the channel so as to obtain more expressive feature information. Finally, a comparative experiment with other SOTA methods was completed on the FLIR on-board infrared image dataset, which proved the powerful detection performance of MSIA-Net. Full article

(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)

► Show Figures

Figure 1

15 pages, 2121 KiB

Open AccessArticle

Inferring a Causal Relationship between Environmental Factors and Respiratory Infections Using Convergent Cross-Mapping

by Daipeng Chen, Xiaodan Sun and Robert A. Cheke

Entropy 2023, 25(5), 807; https://doi.org/10.3390/e25050807 - 17 May 2023

Cited by 4 | Viewed by 2527

Abstract

The incidence of respiratory infections in the population is related to many factors, among which environmental factors such as air quality, temperature, and humidity have attracted much attention. In particular, air pollution has caused widespread discomfort and concern in developing countries. Although the [...] Read more.

The incidence of respiratory infections in the population is related to many factors, among which environmental factors such as air quality, temperature, and humidity have attracted much attention. In particular, air pollution has caused widespread discomfort and concern in developing countries. Although the correlation between respiratory infections and air pollution is well known, establishing causality between them remains elusive. In this study, by conducting theoretical analysis, we updated the procedure of performing the extended convergent cross-mapping (CCM, a method of causal inference) to infer the causality between periodic variables. Consistently, we validated this new procedure on the synthetic data generated by a mathematical model. For real data in Shaanxi province of China in the period of 1 January 2010 to 15 November 2016, we first confirmed that the refined method is applicable by investigating the periodicity of influenza-like illness cases, an air quality index, temperature, and humidity through wavelet analysis. We next illustrated that air quality (quantified by AQI), temperature, and humidity affect the daily influenza-like illness cases, and, in particular, the respiratory infection cases increased progressively with increased AQI with a time delay of 11 days. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

59 pages, 15006 KiB

Open AccessArticle

Causality Analysis with Information Geometry: A Comparison

by Heng Jie Choong, Eun-jin Kim and Fei He

Entropy 2023, 25(5), 806; https://doi.org/10.3390/e25050806 - 16 May 2023

Cited by 3 | Viewed by 2221

Abstract

The quantification of causality is vital for understanding various important phenomena in nature and laboratories, such as brain networks, environmental dynamics, and pathologies. The two most widely used methods for measuring causality are Granger Causality (GC) and Transfer Entropy (TE), which rely on [...] Read more.

The quantification of causality is vital for understanding various important phenomena in nature and laboratories, such as brain networks, environmental dynamics, and pathologies. The two most widely used methods for measuring causality are Granger Causality (GC) and Transfer Entropy (TE), which rely on measuring the improvement in the prediction of one process based on the knowledge of another process at an earlier time. However, they have their own limitations, e.g., in applications to nonlinear, non-stationary data, or non-parametric models. In this study, we propose an alternative approach to quantify causality through information geometry that overcomes such limitations. Specifically, based on the information rate that measures the rate of change of the time-dependent distribution, we develop a model-free approach called information rate causality that captures the occurrence of the causality based on the change in the distribution of one process caused by another. This measurement is suitable for analyzing numerically generated non-stationary, nonlinear data. The latter are generated by simulating different types of discrete autoregressive models which contain linear and nonlinear interactions in unidirectional and bidirectional time-series signals. Our results show that information rate causalitycan capture the coupling of both linear and nonlinear data better than GC and TE in the several examples explored in the paper. Full article

(This article belongs to the Special Issue Causality and Complex Systems)

► Show Figures

Figure 1

18 pages, 1517 KiB

Open AccessArticle

Dynamical Analysis of Hyper-ILSR Rumor Propagation Model with Saturation Incidence Rate

by Xuehui Mei, Ziyu Zhang and Haijun Jiang

Entropy 2023, 25(5), 805; https://doi.org/10.3390/e25050805 - 16 May 2023

Cited by 6 | Viewed by 1638

Abstract

With the development of the Internet, it is more convenient for people to obtain information, which also facilitates the spread of rumors. It is imperative to study the mechanisms of rumor transmission to control the spread of rumors. The process of rumor propagation [...] Read more.

With the development of the Internet, it is more convenient for people to obtain information, which also facilitates the spread of rumors. It is imperative to study the mechanisms of rumor transmission to control the spread of rumors. The process of rumor propagation is often affected by the interaction of multiple nodes. To reflect higher-order interactions in rumor-spreading, hypergraph theories are introduced in a Hyper-ILSR (Hyper-Ignorant–Lurker–Spreader–Recover) rumor-spreading model with saturation incidence rate in this study. Firstly, the definition of hypergraph and hyperdegree is introduced to explain the construction of the model. Secondly, the existence of the threshold and equilibrium of the Hyper-ILSR model is revealed by discussing the model, which is used to judge the final state of rumor propagation. Next, the stability of equilibrium is studied by Lyapunov functions. Moreover, optimal control is put forward to suppress rumor propagation. Finally, the differences between the Hyper-ILSR model and the general ILSR model are shown in numerical simulations. Full article

(This article belongs to the Section Complexity)

► Show Figures

Figure 1

10 pages, 369 KiB

Open AccessArticle

Radial Basis Function Finite Difference Method Based on Oseen Iteration for Solving Two-Dimensional Navier–Stokes Equations

by Liru Mu and Xinlong Feng

Entropy 2023, 25(5), 804; https://doi.org/10.3390/e25050804 - 16 May 2023

Cited by 2 | Viewed by 1358

Abstract

In this paper, the radial basis function finite difference method is used to solve two-dimensional steady incompressible Navier–Stokes equations. First, the radial basis function finite difference method with polynomial is used to discretize the spatial operator. Then, the Oseen iterative scheme is used [...] Read more.

In this paper, the radial basis function finite difference method is used to solve two-dimensional steady incompressible Navier–Stokes equations. First, the radial basis function finite difference method with polynomial is used to discretize the spatial operator. Then, the Oseen iterative scheme is used to deal with the nonlinear term, constructing the discrete scheme for Navier–Stokes equation based on the finite difference method of the radial basis function. This method does not require complete matrix reorganization in each nonlinear iteration, which simplifies the calculation process and obtains high-precision numerical solutions. Finally, several numerical examples are obtained to verify the convergence and effectiveness of the radial basis function finite difference method based on Oseen Iteration. Full article

(This article belongs to the Collection Foundations of Statistical Mechanics)

► Show Figures

Figure 1

31 pages, 402 KiB

Open AccessArticle

Process and Time

by William Sulis

Entropy 2023, 25(5), 803; https://doi.org/10.3390/e25050803 - 15 May 2023

Cited by 3 | Viewed by 3996

Abstract

In regards to the nature of time, it has become commonplace to hear physicists state that time does not exist and that the perception of time passing and of events occurring in time is an illusion. In this paper, I argue that physics [...] Read more.

In regards to the nature of time, it has become commonplace to hear physicists state that time does not exist and that the perception of time passing and of events occurring in time is an illusion. In this paper, I argue that physics is actually agnostic on the question of the nature of time. The standard arguments against its existence all suffer from implicit biases and hidden assumptions, rendering many of them circular in nature. An alternative viewpoint to that of Newtonian materialism is the process view of Whitehead. I will show that the process perspective supports the reality of becoming, of happening, and of change. At the fundamental level, time is an expression of the action of process generating the elements of reality. Metrical space–time is an emergent aspect of relations between process-generated entities. Such a view is compatible with existing physics. The situation of time in physics is reminiscent of that of the continuum hypothesis in mathematical logic. It may be an independent assumption, not provable within physics proper (though it may someday be amenable to experimental exploration). Full article

(This article belongs to the Special Issue Quantum Information and Probability: From Foundations to Engineering)

32 pages, 4255 KiB

Open AccessReview

Reviewing Evolution of Learning Functions and Semantic Information Measures for Understanding Deep Learning

by Chenguang Lu

Entropy 2023, 25(5), 802; https://doi.org/10.3390/e25050802 - 15 May 2023

Cited by 2 | Viewed by 2045

Abstract

A new trend in deep learning, represented by Mutual Information Neural Estimation (MINE) and Information Noise Contrast Estimation (InfoNCE), is emerging. In this trend, similarity functions and Estimated Mutual Information (EMI) are used as learning and objective functions. Coincidentally, EMI is essentially the [...] Read more.

A new trend in deep learning, represented by Mutual Information Neural Estimation (MINE) and Information Noise Contrast Estimation (InfoNCE), is emerging. In this trend, similarity functions and Estimated Mutual Information (EMI) are used as learning and objective functions. Coincidentally, EMI is essentially the same as Semantic Mutual Information (SeMI) proposed by the author 30 years ago. This paper first reviews the evolutionary histories of semantic information measures and learning functions. Then, it briefly introduces the author’s semantic information G theory with the rate-fidelity function R(G) (G denotes SeMI, and R(G) extends R(D)) and its applications to multi-label learning, the maximum Mutual Information (MI) classification, and mixture models. Then it discusses how we should understand the relationship between SeMI and Shannon’s MI, two generalized entropies (fuzzy entropy and coverage entropy), Autoencoders, Gibbs distributions, and partition functions from the perspective of the R(G) function or the G theory. An important conclusion is that mixture models and Restricted Boltzmann Machines converge because SeMI is maximized, and Shannon’s MI is minimized, making information efficiency G/R close to 1. A potential opportunity is to simplify deep learning by using Gaussian channel mixture models for pre-training deep neural networks’ latent layers without considering gradients. It also discusses how the SeMI measure is used as the reward function (reflecting purposiveness) for reinforcement learning. The G theory helps interpret deep learning but is far from enough. Combining semantic information theory and deep learning will accelerate their development. Full article

(This article belongs to the Special Issue Entropy: The Cornerstone of Machine Learning)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Entropy, Volume 25, Issue 5 (May 2023) – 129 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI