Entropy

11 pages, 1082 KiB

Open AccessArticle

Fiducial Inference in Linear Mixed-Effects Models

by Jie Yang, Xinmin Li, Hongwei Gao and Chenchen Zou

Entropy 2025, 27(2), 161; https://doi.org/10.3390/e27020161 - 3 Feb 2025

We develop a novel framework for fiducial inference in linear mixed-effects (LME) models, with the standard deviation of random effects reformulated as coefficients. The exact fiducial density is derived as the equilibrium measure of a reversible Markov chain over the parameter space. The [...] Read more.

We develop a novel framework for fiducial inference in linear mixed-effects (LME) models, with the standard deviation of random effects reformulated as coefficients. The exact fiducial density is derived as the equilibrium measure of a reversible Markov chain over the parameter space. The density is equivalent in form to a Bayesian LME with noninformative prior, while the underlying fiducial structure adds new benefits to unify the inference of random effects and all other parameters in a neat and simultaneous way. Our fiducial LME needs no additional tests or statistics for zero variance and is more suitable for small sample sizes. In simulation and empirical analysis, our confidence intervals (CIs) are comparable to those based on Bayesian and likelihood profiling methods. And our inference for the variance of random effects has competitive power with the likelihood ratio test. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

15 pages, 16805 KiB

Open AccessArticle

Performance Improvement for Discretely Modulated Continuous-Variable Measurement-Device-Independent Quantum Key Distribution with Imbalanced Modulation

by Zehui Liu, Jiandong Bai, Fengchao Li, Yijun Li, Yan Tian and Wenyuan Liu

Entropy 2025, 27(2), 160; https://doi.org/10.3390/e27020160 - 3 Feb 2025

Abstract

The modulation mode at the transmitters plays a crucial role in the continuous-variable measurement-device-independent quantum key distribution (CV-MDI-QKD) protocol. However, in practical applications, differences in the modulation schemes between two transmitters can inevitably impact protocol performance, particularly when using discrete modulation with four-state [...] Read more.

The modulation mode at the transmitters plays a crucial role in the continuous-variable measurement-device-independent quantum key distribution (CV-MDI-QKD) protocol. However, in practical applications, differences in the modulation schemes between two transmitters can inevitably impact protocol performance, particularly when using discrete modulation with four-state or eight-state formats. This work primarily investigates the effect of imbalanced modulation at the transmitters on the security of the CV-MDI-QKD protocol under both symmetric and asymmetric distance scenarios. By employing imbalanced discrete modulation maps and numerical convex optimization techniques, the proposed CV-MDI-QKD protocol achieves a notably higher secret key rate and outperforms existing protocols in terms of maximum transmission distance. Specifically, simulation results demonstrate that the secret key rate and maximum transmission distance are boosted by approximately 77.77% and 24.3%, respectively, compared to the original protocol. This novel and simplified modulation method can be seamlessly implemented in existing experimental setups without requiring equipment modifications. Furthermore, it provides a practical approach to enhancing protocol performance and enabling cost-effective applications in secure quantum communication networks under real-world environments. Full article

(This article belongs to the Section Quantum Information)

► Show Figures

Figure 1

11 pages, 3819 KiB

Open AccessArticle

Improved CNN Prediction Based Reversible Data Hiding for Images

by Yingqiang Qiu, Wanli Peng and Xiaodan Lin

Entropy 2025, 27(2), 159; https://doi.org/10.3390/e27020159 - 3 Feb 2025

Abstract

This paper proposes a reversible data hiding (RDH) scheme for images with an improved convolutional neural network (CNN) predictor (ICNNP) that consists of three modules for feature extraction, pixel prediction, and complexity prediction, respectively. Due to predicting the complexity of each pixel with [...] Read more.

This paper proposes a reversible data hiding (RDH) scheme for images with an improved convolutional neural network (CNN) predictor (ICNNP) that consists of three modules for feature extraction, pixel prediction, and complexity prediction, respectively. Due to predicting the complexity of each pixel with the ICNNP during the embedding process, the proposed scheme can achieve superior performance compared to a CNNP-based scheme. Specifically, an input image is first split into two sub-images, i.e., a “Circle” sub-image and a “Square” sub-image. Meanwhile, each sub-image is applied to predict another one with the ICNNP. Then, the prediction errors of pixels are sorted based on the predicted pixel complexities. In light of this, some sorted prediction errors with less complexity are selected to be efficiently applied for low-distortion data embedding with a traditional histogram-shifting technique. Experimental results show that the proposed ICNNP can achieve better rate-distortion performance than the CNNP, demonstrating its effectiveness. Full article

(This article belongs to the Special Issue Information Hiding and Secret Sharing for New Carriers and Their Security Evaluation Methods)

► Show Figures

Figure 1

13 pages, 5321 KiB

Open AccessArticle

Solutions of a Class of Switch Dynamical Systems

by Marius-F. Danca

Entropy 2025, 27(2), 158; https://doi.org/10.3390/e27020158 - 2 Feb 2025

Abstract

In this paper, the solutions of a class of switch dynamical systems are investigated. The right-hand side of the underlying equations is discontinuous with respect to the state variable. The discontinuity is represented by jump discontinuous functions such as signum or Heaviside functions. [...] Read more.

In this paper, the solutions of a class of switch dynamical systems are investigated. The right-hand side of the underlying equations is discontinuous with respect to the state variable. The discontinuity is represented by jump discontinuous functions such as signum or Heaviside functions. In this paper, a novel approach of the solutions of this class of discontinuous equations is presented. The initial value problem is restated as a differential inclusion via Filippov’s regularization, after which, via the approximate selection results, the differential inclusion is transformed into a continuous, single-valued differential equation. Besides its existence, a sufficient uniqueness condition, the strengthened one-sided Lipschitz Condition, is also introduced. The important issue of the numerical integration of this class of equations is addressed, emphasizing by examples the errors that could appear if the discontinuity problem is neglected. The example of a mechanical system, a preloaded compliance system, is considered along with other examples. Full article

(This article belongs to the Special Issue Complex Dynamic System Modelling, Identification and Control, 2nd Edition)

22 pages, 722 KiB

Open AccessArticle

Levy Noise Affects Ornstein–Uhlenbeck Memory

by Iddo Eliazar

Entropy 2025, 27(2), 157; https://doi.org/10.3390/e27020157 - 2 Feb 2025

Abstract

This paper investigates the memory of the Ornstein–Uhlenbeck process (OUP) via three ratios of the OUP increments: signal-to-noise, noise-to-noise, and tail-to-tail. Intuition suggests the following points: (1) changing the noise that drives the OUP from Gauss to Levy will not affect the memory, [...] Read more.

This paper investigates the memory of the Ornstein–Uhlenbeck process (OUP) via three ratios of the OUP increments: signal-to-noise, noise-to-noise, and tail-to-tail. Intuition suggests the following points: (1) changing the noise that drives the OUP from Gauss to Levy will not affect the memory, as both noises share the common `independent increments’ property; (2) changing the auto-correlation of the OUP from exponential to slowly decaying will affect the memory, as the change yields a process with long-range correlations; and (3) with regard to Levy driving noise, the greater the noise fluctuations, the noisier the prediction of the OUP increments. This paper shows that intuition is plain wrong. Indeed, a detailed analysis establishes that for each of the three above-mentioned points, the very converse holds. Hence, Levy noise has a significant and counter-intuitive effect on Ornstein–Uhlenbeck memory. Full article

(This article belongs to the Collection Foundations of Statistical Mechanics)

31 pages, 1195 KiB

Open AccessArticle

Machine Learning Predictors for Min-Entropy Estimation

by Javier Blanco-Romero, Vicente Lorenzo, Florina Almenares Mendoza and Daniel Díaz-Sánchez

Entropy 2025, 27(2), 156; https://doi.org/10.3390/e27020156 - 2 Feb 2025

Abstract

This study investigates the application of machine learning predictors for the estimation of min-entropy in random number generators (RNGs), a key component in cryptographic applications where accurate entropy assessment is essential for cybersecurity. Our research indicates that these predictors, and indeed any predictor [...] Read more.

This study investigates the application of machine learning predictors for the estimation of min-entropy in random number generators (RNGs), a key component in cryptographic applications where accurate entropy assessment is essential for cybersecurity. Our research indicates that these predictors, and indeed any predictor that leverages sequence correlations, primarily estimate average min-entropy, a metric not extensively studied in this context. We explore the relationship between average min-entropy and the traditional min-entropy, focusing on their dependence on the number of target bits being predicted. Using data from generalized binary autoregressive models, a subset of Markov processes, we demonstrate that machine learning models (including a hybrid of convolutional and recurrent long short-term memory layers and the transformer-based GPT-2 model) outperform traditional NIST SP 800-90B predictors in certain scenarios. Our findings underscore the importance of considering the number of target bits in min-entropy assessment for RNGs and highlight the potential of machine learning approaches in enhancing entropy estimation techniques for improved cryptographic security. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

17 pages, 784 KiB

Open AccessArticle

Effects of Multiplicative Noise in Bistable Dynamical Systems

by Sara C. Quintanilha Valente, Rodrigo da Costa Lima Bruni, Zochil González Arenas and Daniel G. Barci

Entropy 2025, 27(2), 155; https://doi.org/10.3390/e27020155 - 2 Feb 2025

Abstract

This study explores the escape dynamics of bistable systems influenced by multiplicative noise, extending the classical Kramers rate formula to scenarios involving state-dependent diffusion in asymmetric potentials. Using a generalized stochastic calculus framework, we derive an analytical expression for the escape rate and [...] Read more.

This study explores the escape dynamics of bistable systems influenced by multiplicative noise, extending the classical Kramers rate formula to scenarios involving state-dependent diffusion in asymmetric potentials. Using a generalized stochastic calculus framework, we derive an analytical expression for the escape rate and corroborate it with numerical simulations. The results highlight the critical role of the equilibrium potential

U_{eq} (x)

, which incorporates noise intensity, stochastic prescription, and diffusion properties. We show how asymmetries and stochastic calculus prescriptions influence transition rates and equilibrium configurations. Using path integral techniques and weak noise approximations, we analyze the interplay between noise and potential asymmetry, uncovering phenomena such as barrier suppression and metastable state decay. The agreement between numerical and analytical results underscores the robustness of the proposed framework. This work provides a comprehensive foundation for studying noise-induced transitions in stochastic systems, offering insights into a broad range of applications in physics, chemistry, and biology. Full article

(This article belongs to the Special Issue Nonlinear Dynamics and Applications—In Honor of Professor Osvaldo Anibal Rosso's 70th Birthday)

15 pages, 4130 KiB

Open AccessArticle

Protocells Either Synchronize or Starve

by Marco Villani and Roberto Serra

Entropy 2025, 27(2), 154; https://doi.org/10.3390/e27020154 - 2 Feb 2025

Abstract

Two different processes take place in self-reproducing protocells, i.e., (i) cell reproduction by fission and (ii) duplication of the genetic material. One major problem is indeed that of assuring that the two processes take place at the same pace, i.e., that they synchronize, [...] Read more.

Two different processes take place in self-reproducing protocells, i.e., (i) cell reproduction by fission and (ii) duplication of the genetic material. One major problem is indeed that of assuring that the two processes take place at the same pace, i.e., that they synchronize, which is a necessary condition for sustainable growth. In previous theoretical works, using dynamical models, we had shown that such synchronization can spontaneously emerge, generation after generation, under a broad set of hypotheses about the architecture of the protocell, the nature of the self-replicating molecules, and the types of kinetic equations. However, an important class of cases (quadratic or higher-order self-replication) did not synchronize in the models we had used, but could actually lead to divergence of the concentration of replicators. We show here that this behavior is due to a simplification of the previous models, i.e., the “buffering” hypothesis, which assumes instantaneous equilibrium of the internal and external concentrations of those compounds which can cross the cell membrane. That divergence disappears if we make use of more realistic dynamical models, with finite transmembrane diffusion rates of the precursors of replicators. Full article

(This article belongs to the Section Entropy and Biology)

► Show Figures

Figure 1

23 pages, 912 KiB

Open AccessArticle

Extending the QMM Framework to the Strong and Weak Interactions

by Florian Neukart, Eike Marx and Valerii Vinokur

Entropy 2025, 27(2), 153; https://doi.org/10.3390/e27020153 - 2 Feb 2025

Abstract

We extend the Quantum Memory Matrix (QMM) framework, originally developed to reconcile quantum mechanics and general relativity by treating space–time as a dynamic information reservoir, to incorporate the full suite of Standard Model gauge interactions. In this discretized, Planck-scale formulation, each space–time cell [...] Read more.

We extend the Quantum Memory Matrix (QMM) framework, originally developed to reconcile quantum mechanics and general relativity by treating space–time as a dynamic information reservoir, to incorporate the full suite of Standard Model gauge interactions. In this discretized, Planck-scale formulation, each space–time cell possesses a finite-dimensional Hilbert space that acts as a local memory, or quantum imprint, for matter and gauge field configurations. We focus on embedding non-Abelian SU(3)

_{c}

(quantum chromodynamics) and SU(2)

_{L}

× U(1)

_{Y}

(electroweak interactions) into QMM by constructing gauge-invariant imprint operators for quarks, gluons, electroweak bosons, and the Higgs mechanism. This unified approach naturally enforces unitarity by allowing black hole horizons, or any high-curvature region, to store and later retrieve quantum information about color and electroweak charges, thereby preserving subtle non-thermal correlations in evaporation processes. Moreover, the discretized nature of QMM imposes a Planck-scale cutoff, potentially taming UV divergences and modifying running couplings at trans-Planckian energies. We outline major challenges, such as the precise formulation of non-Abelian imprint operators and the integration of QMM with loop quantum gravity, as well as possible observational strategies—ranging from rare decay channels to primordial black hole evaporation spectra—that could provide indirect probes of this discrete, memory-based view of quantum gravity and the Standard Model. Full article

(This article belongs to the Section Astrophysics, Cosmology, and Black Holes)

20 pages, 1356 KiB

Open AccessArticle

A Quantum Probability Approach to Improving Human–AI Decision Making

by Scott Humr, Mustafa Canan and Mustafa Demir

Entropy 2025, 27(2), 152; https://doi.org/10.3390/e27020152 - 2 Feb 2025

Abstract

Artificial intelligence is set to incorporate additional decision space that has traditionally been the purview of humans. However, AI systems that support decision making also entail the rationalization of AI outputs by humans. Yet, incongruencies between AI and human rationalization processes may introduce [...] Read more.

Artificial intelligence is set to incorporate additional decision space that has traditionally been the purview of humans. However, AI systems that support decision making also entail the rationalization of AI outputs by humans. Yet, incongruencies between AI and human rationalization processes may introduce uncertainties in human decision making, which require new conceptualizations to improve the predictability of these interactions. The application of quantum probability theory (QPT) to human cognition is on the ascent and warrants potential consideration to human–AI decision making to improve these outcomes. This perspective paper explores how QPT may be applied to human–AI interactions and contributes by integrating these concepts into human-in-the-loop decision making. To capture this and offer a more comprehensive conceptualization, we use human-in-the-loop constructs to explicate how recent applications of QPT can ameliorate the models of interaction by providing a novel way to capture these behaviors. Followed by a summary of the challenges posed by human-in-the-loop systems, we discuss newer theories that advance models of the cognitive system by using quantum probability formalisms. We conclude by outlining areas of promising future research in human–AI decision making in which the proposed methods may apply. Full article

(This article belongs to the Special Issue Quantum Information and Probability: From Foundations to Engineering III)

► Show Figures

Figure 1

22 pages, 332 KiB

Open AccessArticle

Algebraic Representations of Entropy and Fixed-Sign Information Quantities

by Keenan J. A. Down and Pedro A. M. Mediano

Entropy 2025, 27(2), 151; https://doi.org/10.3390/e27020151 - 1 Feb 2025

Abstract

Many information-theoretic quantities have corresponding representations in terms of sets. Many of these information quantities do not have a fixed sign—for example, the co-information can be both positive and negative. In previous work, we presented a signed measure space for entropy where the [...] Read more.

Many information-theoretic quantities have corresponding representations in terms of sets. Many of these information quantities do not have a fixed sign—for example, the co-information can be both positive and negative. In previous work, we presented a signed measure space for entropy where the smallest sets (called atoms) all have fixed signs. In the present work, we demonstrate that these atoms have natural algebraic behaviour which can be expressed in terms of ideals (characterised here as upper sets), and we show that this behaviour allows us to make bounding arguments and describe many fixed-sign information quantity expressions. As an application, we give an algebraic proof that the only completely synergistic system of three finite variables X, Y and

Z = f (X, Y)

is the XOR gate. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

23 pages, 1738 KiB

Open AccessArticle

Quantum Dynamics Framework with Quantum Tunneling Effect for Numerical Optimization

by Quan Tang and Peng Wang

Entropy 2025, 27(2), 150; https://doi.org/10.3390/e27020150 - 1 Feb 2025

Abstract

In recent years, optimization algorithms have developed rapidly, especially those which introduce quantum ideas, which perform excellently. Inspired by quantum thought, this paper proposes a quantum dynamics framework (QDF) which converts optimization problems into the problem of the constrained ground state of the [...] Read more.

In recent years, optimization algorithms have developed rapidly, especially those which introduce quantum ideas, which perform excellently. Inspired by quantum thought, this paper proposes a quantum dynamics framework (QDF) which converts optimization problems into the problem of the constrained ground state of the quantum system and analyzes optimization algorithms by simulating the dynamic evolution process of physical optimization systems in the ground state. Potential energy equivalence and Taylor expansion are performed on the objective function to obtain the basic iterative operations of optimization algorithms. Furthermore, a quantum dynamics framework based on the quantum tunneling effect (QDF-TE) is proposed which adopts dynamic multiple group collaborative sampling to improve the quantum tunneling effect of the QDF, thereby increasing the population diversity and improving algorithm performance. The probability distribution of solutions can be visually observed through the evolution of the wave function, which also indicates that the QDF-TE can strengthen the tunneling effect. The QDF-TE was evaluated on the CEC 2017 test suite and shown to be competitive with other heuristic optimization algorithms. The experimental results reveal the effectiveness of introducing a quantum mechanism to analyze and improve optimization algorithms. Full article

(This article belongs to the Special Issue Exploring the Horizon of Practical Utility in Near-Term Quantum Computing)

► Show Figures

Figure 1

28 pages, 3464 KiB

Open AccessArticle

Understanding the Impact of Evaluation Metrics in Kinetic Models for Consensus-Based Segmentation

by Raffaella Fiamma Cabini, Horacio Tettamanti and Mattia Zanella

Entropy 2025, 27(2), 149; https://doi.org/10.3390/e27020149 - 1 Feb 2025

Abstract

In this article, we extend a recently introduced kinetic model for consensus-based segmentation of images. In particular, we will interpret the set of pixels of a 2D image as an interacting particle system that evolves in time in view of a consensus-type process [...] Read more.

In this article, we extend a recently introduced kinetic model for consensus-based segmentation of images. In particular, we will interpret the set of pixels of a 2D image as an interacting particle system that evolves in time in view of a consensus-type process obtained by interactions between pixels and external noise. Thanks to a kinetic formulation of the introduced model, we derive the large time solution of the model. We will show that the parameters defining the segmentation task can be chosen from a plurality of loss functions that characterize the evaluation metrics. Full article

(This article belongs to the Section Multidisciplinary Applications)

24 pages, 582 KiB

Open AccessArticle

A Network Analysis of the Impact of the Coronavirus Pandemic on the US Economy: A Comparison of the Return and the Momentum Picture

by Janusz Miśkiewicz

Entropy 2025, 27(2), 148; https://doi.org/10.3390/e27020148 - 1 Feb 2025

Abstract

This study examines a cross-correlation analysis of companies included in the S&P 500 Index at three different intervals: before, during, and after the pandemic’s onset. The aim is to evaluate how the pandemic and related governmental actions have affected market structures and economic [...] Read more.

This study examines a cross-correlation analysis of companies included in the S&P 500 Index at three different intervals: before, during, and after the pandemic’s onset. The aim is to evaluate how the pandemic and related governmental actions have affected market structures and economic conditions. This paper introduces the notion of momentum time series, integrating return and volume data. We show that these momentum time series provide unique insights that differ from return time series, suggesting their potential utility in economic analysis. Our analysis employs the Manhattan and Mantegna distances to construct a threshold-based network, which we subsequently scrutinize. Lastly, we evaluate how the pandemic has influenced these outcomes. Full article

(This article belongs to the Special Issue Entropy, Econophysics, and Complexity)

16 pages, 947 KiB

Open AccessArticle

The Rosencrantz Coin: Predictability and Structure in Non-Ergodic Dynamics—From Recurrence Times to Temporal Horizons

by Dimitri Volchenkov

Entropy 2025, 27(2), 147; https://doi.org/10.3390/e27020147 - 1 Feb 2025

Abstract

We examine the Rosencrantz coin that can “stick” in states for extended periods. Non-ergodic dynamics is highlighted by logarithmically growing block lengths in sequences. Traditional entropy decomposition into predictable and unpredictable components fails due to the absence of stationary distributions. Instead, sequence structure [...] Read more.

We examine the Rosencrantz coin that can “stick” in states for extended periods. Non-ergodic dynamics is highlighted by logarithmically growing block lengths in sequences. Traditional entropy decomposition into predictable and unpredictable components fails due to the absence of stationary distributions. Instead, sequence structure is characterized by block probabilities and Stirling numbers of the second kind, peaking at block size

n / log n

. For large n, combinatorial growth dominates probability decay, creating a deterministic-like structure. This approach shifts the focus from predicting states to predicting temporal horizons, providing insights into systems beyond traditional equilibrium frameworks. Full article

(This article belongs to the Section Statistical Physics)

28 pages, 403 KiB

Open AccessArticle

Penalized Exponentially Tilted Likelihood for Growing Dimensional Models with Missing Data

by Xiaoming Sha, Puying Zhao and Niansheng Tang

Entropy 2025, 27(2), 146; https://doi.org/10.3390/e27020146 - 1 Feb 2025

Abstract

This paper develops a penalized exponentially tilted (ET) likelihood to simultaneously estimate unknown parameters and select variables for growing dimensional models with missing response at random. The inverse probability weighted approach is employed to compensate for missing information and to ensure the consistency [...] Read more.

This paper develops a penalized exponentially tilted (ET) likelihood to simultaneously estimate unknown parameters and select variables for growing dimensional models with missing response at random. The inverse probability weighted approach is employed to compensate for missing information and to ensure the consistency of parameter estimators. Based on the penalized ET likelihood, we construct an ET likelihood ratio statistic to test the contrast hypothesis of parameters. Under some wild conditions, we obtain the consistency, asymptotic properties, and oracle properties of parameter estimators and show that the constrained penalized ET likelihood ratio statistic for testing the contrast hypothesis possesses the Wilks’ property. Simulation studies are conducted to validate the finite sample performance of the proposed methodologies. Thyroid data taken from the First People’s Hospital of Yunnan Province is employed to illustrate the proposed methodologies. Full article

(This article belongs to the Special Issue Statistical Methods for Modeling High-Dimensional and Complex Data: Second Edition)

22 pages, 6162 KiB

Open AccessArticle

Recurrence Resonance and 1/f Noise in Neurons Under Quantum Conditions and Their Manifestations in Proteinoid Microspheres

by Yu Huang, Panagiotis Mougkogiannis, Andrew Adamatzky and Yukio Pegio Gunji

Entropy 2025, 27(2), 145; https://doi.org/10.3390/e27020145 - 1 Feb 2025

Abstract

Recurrence resonance (RR), in which external noise is utilized to enhance the behaviour of hidden attractors in a system, is a phenomenon often observed in biological systems and is expected to adjust between chaos and order to increase computational power. It is known [...] Read more.

Recurrence resonance (RR), in which external noise is utilized to enhance the behaviour of hidden attractors in a system, is a phenomenon often observed in biological systems and is expected to adjust between chaos and order to increase computational power. It is known that connections of neurons that are relatively dense make it possible to achieve RR and can be measured by global mutual information. Here, we used a Boltzmann machine to investigate how the manifestation of RR changes when the connection pattern between neurons is changed. When the connection strength pattern between neurons forms a partially sparse cluster structure revealing Boolean algebra or Quantum logic, an increase in mutual information and the formation of a maximum value are observed not only in the entire network but also in the subsystems of the network, making recurrence resonance detectable. It is also found that in a clustered connection distribution, the state time series of a single neuron shows

1 / f

noise. In proteinoid microspheres, clusters of amino acid compounds, the time series of localized potential changes emit pulses like neurons and transmit and receive information. Indeed, it is found that these also exhibit

1 / f

noise, and the results here also suggest RR. Full article

(This article belongs to the Special Issue Complexity and Evolution, 2nd Edition)

24 pages, 7864 KiB

Open AccessArticle

The Epistemic Uncertainty Gradient in Spaces of Random Projections

by Jeffrey F. Queißer, Jun Tani and Jochen J. Steil

Entropy 2025, 27(2), 144; https://doi.org/10.3390/e27020144 - 1 Feb 2025

Abstract

This work presents a novel approach to handling epistemic uncertainty estimates with motivation from Bayesian linear regression. We propose treating the model-dependent variance in the predictive distribution—commonly associated with epistemic uncertainty—as a model for the underlying data distribution. Using high-dimensional random feature transformations, [...] Read more.

This work presents a novel approach to handling epistemic uncertainty estimates with motivation from Bayesian linear regression. We propose treating the model-dependent variance in the predictive distribution—commonly associated with epistemic uncertainty—as a model for the underlying data distribution. Using high-dimensional random feature transformations, this approach allows for a computationally efficient, parameter-free representation of arbitrary data distributions. This allows assessing whether a query point lies within the distribution, which can also provide insights into outlier detection and generalization tasks. Furthermore, given an initial input, minimizing the uncertainty using gradient descent offers a new method of querying data points that are close to the initial input and belong to the distribution resembling the training data, much like auto-completion in associative networks. We extend the proposed method to applications such as local Gaussian approximations, input–output regression, and even a mechanism for unlearning of data. This reinterpretation of uncertainty, alongside the geometric insights it provides, offers an innovative and novel framework for addressing classical machine learning challenges. Full article

(This article belongs to the Special Issue The Mathematics of Structured Experience: Exploring Dynamics, Topology, and Complexity in the Brain)

20 pages, 1133 KiB

Open AccessArticle

As One and Many: Relating Individual and Emergent Group-Level Generative Models in Active Inference

by Peter Thestrup Waade, Christoffer Lundbak Olesen, Jonathan Ehrenreich Laursen, Samuel William Nehrer, Conor Heins, Karl Friston and Christoph Mathys

Entropy 2025, 27(2), 143; https://doi.org/10.3390/e27020143 - 1 Feb 2025

Abstract

Active inference under the Free Energy Principle has been proposed as an across-scales compatible framework for understanding and modelling behaviour and self-maintenance. Crucially, a collective of active inference agents can, if they maintain a group-level Markov blanket, constitute a larger group-level active inference [...] Read more.

Active inference under the Free Energy Principle has been proposed as an across-scales compatible framework for understanding and modelling behaviour and self-maintenance. Crucially, a collective of active inference agents can, if they maintain a group-level Markov blanket, constitute a larger group-level active inference agent with a generative model of its own. This potential for computational scale-free structures speaks to the application of active inference to self-organizing systems across spatiotemporal scales, from cells to human collectives. Due to the difficulty of reconstructing the generative model that explains the behaviour of emergent group-level agents, there has been little research on this kind of multi-scale active inference. Here, we propose a data-driven methodology for characterising the relation between the generative model of a group-level agent and the dynamics of its constituent individual agents. We apply methods from computational cognitive modelling and computational psychiatry, applicable for active inference as well as other types of modelling approaches. Using a simple Multi-Armed Bandit task as an example, we employ the new ActiveInference.jl library for Julia to simulate a collective of agents who are equipped with a Markov blanket. We use sampling-based parameter estimation to make inferences about the generative model of the group-level agent, and we show that there is a non-trivial relationship between the generative models of individual agents and the group-level agent they constitute, even in this simple setting. Finally, we point to a number of ways in which this methodology might be applied to better understand the relations between nested active inference agents across scales. Full article

(This article belongs to the Special Issue From Functional Imaging to Free Energy—Dedicated to Professor Karl Friston on the Occasion of His 65th Birthday)

► Show Figures

Figure 1

Journal Description

Entropy

Latest Articles

Journal Menu

Journal Browser

Highly Accessed Articles

Latest Books

E-Mail Alert

News

Topics

Conferences

Special Issues

Topical Collections

Further Information

Guidelines

MDPI Initiatives

Follow MDPI