Entropy

11 pages, 341 KiB

Open AccessFeature PaperArticle

Reconstructing Binary Signals from Local Histograms

by Jon Sporring and Sune Darkner

Entropy 2022, 24(3), 433; https://doi.org/10.3390/e24030433 - 21 Mar 2022

Cited by 2 | Viewed by 2695

In this paper, we considered the representation power of local overlapping histograms for discrete binary signals. We give an algorithm that is linear in signal size and factorial in window size for producing the set of signals, which share a sequence of densely [...] Read more.

In this paper, we considered the representation power of local overlapping histograms for discrete binary signals. We give an algorithm that is linear in signal size and factorial in window size for producing the set of signals, which share a sequence of densely overlapping histograms, and we state the values for the sizes of the number of unique signals for a given set of histograms, as well as give bounds on the number of metameric classes, where a metameric class is a set of signals larger than one, which has the same set of densely overlapping histograms. Full article

(This article belongs to the Special Issue Application of Entropy to Computer Vision and Medical Imaging)

► Show Figures

Figure 1

17 pages, 887 KiB

Open AccessArticle

Task’s Choice: Pruning-Based Feature Sharing (PBFS) for Multi-Task Learning

by Ying Chen, Jiong Yu, Yutong Zhao, Jiaying Chen and Xusheng Du

Entropy 2022, 24(3), 432; https://doi.org/10.3390/e24030432 - 21 Mar 2022

Cited by 7 | Viewed by 2824

Abstract

In most of the existing multi-task learning (MTL) models, multiple tasks’ public information is learned by sharing parameters across hidden layers, such as hard sharing, soft sharing, and hierarchical sharing. One promising approach is to introduce model pruning into information learning, such as [...] Read more.

In most of the existing multi-task learning (MTL) models, multiple tasks’ public information is learned by sharing parameters across hidden layers, such as hard sharing, soft sharing, and hierarchical sharing. One promising approach is to introduce model pruning into information learning, such as sparse sharing, which is regarded as being outstanding in knowledge transferring. However, the above method performs inefficiently in conflict tasks, with inadequate learning of tasks’ private information, or through suffering from negative transferring. In this paper, we propose a multi-task learning model (Pruning-Based Feature Sharing, PBFS) that merges a soft parameter sharing structure with model pruning and adds a prunable shared network among different task-specific subnets. In this way, each task can select parameters in a shared subnet, according to its requirements. Experiments are conducted on three benchmark public datasets and one synthetic dataset; the impact of the different subnets’ sparsity and tasks’ correlations to the model performance is analyzed. Results show that the proposed model’s information sharing strategy is helpful to transfer learning and superior to the several comparison models. Full article

(This article belongs to the Topic Machine and Deep Learning)

► Show Figures

Figure 1

12 pages, 1200 KiB

Open AccessArticle

An Enhanced Affine Projection Algorithm Based on the Adjustment of Input-Vector Number

by Jaewook Shin, Jeesu Kim, Tae-Kyoung Kim and Jinwoo Yoo

Entropy 2022, 24(3), 431; https://doi.org/10.3390/e24030431 - 20 Mar 2022

Cited by 1 | Viewed by 2177

Abstract

An enhanced affine projection algorithm (APA) is proposed to improve the filter performance in aspects of convergence rate and steady-state estimation error, since the adjustment of the input-vector number can be an effective way to increase the convergence rate and to decrease the [...] Read more.

An enhanced affine projection algorithm (APA) is proposed to improve the filter performance in aspects of convergence rate and steady-state estimation error, since the adjustment of the input-vector number can be an effective way to increase the convergence rate and to decrease the steady-state estimation error at the same time. In this proposed algorithm, the input-vector number of APA is adjusted reasonably at every iteration by comparing the averages of the accumulated squared errors. Although the conventional APA has the constraint that the input-vector number should be integer, the proposed APA relaxes that integer-constraint through a pseudo-fractional method. Since the input-vector number can be updated at every iteration more precisely based on the pseudo-fractional method, the filter performance of the proposed APA can be improved. According to our simulation results, it is demonstrated that the proposed APA has a smaller steady-state estimation error compared to the existing APA-type filters in various scenarios. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

31 pages, 904 KiB

Open AccessReview

GDP vs. LDP: A Survey from the Perspective of Information-Theoretic Channel

by Hai Liu, Changgen Peng, Youliang Tian, Shigong Long, Feng Tian and Zhenqiang Wu

Entropy 2022, 24(3), 430; https://doi.org/10.3390/e24030430 - 19 Mar 2022

Cited by 4 | Viewed by 2947

Abstract

The existing work has conducted in-depth research and analysis on global differential privacy (GDP) and local differential privacy (LDP) based on information theory. However, the data privacy preserving community does not systematically review and analyze GDP and LDP based on the information-theoretic channel [...] Read more.

The existing work has conducted in-depth research and analysis on global differential privacy (GDP) and local differential privacy (LDP) based on information theory. However, the data privacy preserving community does not systematically review and analyze GDP and LDP based on the information-theoretic channel model. To this end, we systematically reviewed GDP and LDP from the perspective of the information-theoretic channel in this survey. First, we presented the privacy threat model under information-theoretic channel. Second, we described and compared the information-theoretic channel models of GDP and LDP. Third, we summarized and analyzed definitions, privacy-utility metrics, properties, and mechanisms of GDP and LDP under their channel models. Finally, we discussed the open problems of GDP and LDP based on different types of information-theoretic channel models according to the above systematic review. Our main contribution provides a systematic survey of channel models, definitions, privacy-utility metrics, properties, and mechanisms for GDP and LDP from the perspective of information-theoretic channel and surveys the differential privacy synthetic data generation application using generative adversarial network and federated learning, respectively. Our work is helpful for systematically understanding the privacy threat model, definitions, privacy-utility metrics, properties, and mechanisms of GDP and LDP from the perspective of information-theoretic channel and promotes in-depth research and analysis of GDP and LDP based on different types of information-theoretic channel models. Full article

(This article belongs to the Special Issue Information Theoretical Security and Privacy)

9 pages, 299 KiB

Open AccessArticle

Generalized Householder Transformations

by Karl Svozil

Entropy 2022, 24(3), 429; https://doi.org/10.3390/e24030429 - 19 Mar 2022

Cited by 1 | Viewed by 2130

Abstract

The Householder transformation, allowing a rewrite of probabilities into expectations of dichotomic observables, is generalized in terms of its spectral decomposition. The dichotomy is modulated by allowing more than one negative eigenvalue or by abandoning binaries altogether, yielding generalized operator-valued arguments for contextuality. [...] Read more.

The Householder transformation, allowing a rewrite of probabilities into expectations of dichotomic observables, is generalized in terms of its spectral decomposition. The dichotomy is modulated by allowing more than one negative eigenvalue or by abandoning binaries altogether, yielding generalized operator-valued arguments for contextuality. We also discuss a form of contextuality by the variation of the functional relations of the operators, in particular by additivity. Full article

(This article belongs to the Special Issue Quantum Probability and Randomness III)

► Show Figures

Figure 1

12 pages, 1046 KiB

Open AccessArticle

Machine Learning Models and Statistical Complexity to Analyze the Effects of Posture on Cerebral Hemodynamics

by Max Chacón, Hector Rojas-Pescio, Sergio Peñaloza and Jean Landerretche

Entropy 2022, 24(3), 428; https://doi.org/10.3390/e24030428 - 19 Mar 2022

Cited by 4 | Viewed by 2642

Abstract

The mechanism of cerebral blood flow autoregulation can be of great importance in diagnosing and controlling a diversity of cerebrovascular pathologies such as vascular dementia, brain injury, and neurodegenerative diseases. To assess it, there are several methods that use changing postures, such as [...] Read more.

The mechanism of cerebral blood flow autoregulation can be of great importance in diagnosing and controlling a diversity of cerebrovascular pathologies such as vascular dementia, brain injury, and neurodegenerative diseases. To assess it, there are several methods that use changing postures, such as sit-stand or squat-stand maneuvers. However, the evaluation of the dynamic cerebral blood flow autoregulation (dCA) in these postures has not been adequately studied using more complex models, such as non-linear ones. Moreover, dCA can be considered part of a more complex mechanism called cerebral hemodynamics, where others (CO₂ reactivity and neurovascular-coupling) that affect cerebral blood flow (BF) are included. In this work, we analyzed postural influences using non-linear machine learning models of dCA and studied characteristics of cerebral hemodynamics under statistical complexity using eighteen young adult subjects, aged 27 ± 6.29 years, who took the systemic or arterial blood pressure (BP) and cerebral blood flow velocity (BFV) for five minutes in three different postures: stand, sit, and lay. With models of a Support Vector Machine (SVM) through time, we used an AutoRegulatory Index (ARI) to compare the dCA in different postures. Using wavelet entropy, we estimated the statistical complexity of BFV for three postures. Repeated measures ANOVA showed that only the complexity of lay-sit had significant differences. Full article

(This article belongs to the Topic Complex Systems and Artificial Intelligence)

► Show Figures

Figure 1

11 pages, 375 KiB

Open AccessArticle

Controversial Variable Node Selection-Based Adaptive Belief Propagation Decoding Algorithm Using Bit Flipping Check for JSCC Systems

by Hao Wang, Wei Zhang, Yizhe Jing, Yanyan Chang and Yanyan Liu

Entropy 2022, 24(3), 427; https://doi.org/10.3390/e24030427 - 19 Mar 2022

Cited by 1 | Viewed by 2202

Abstract

An end-to-end joint source–channel (JSC) encoding matrix and a JSC decoding scheme using the proposed bit flipping check (BFC) algorithm and controversial variable node selection-based adaptive belief propagation (CVNS-ABP) decoding algorithm are presented to improve the efficiency and reliability of the joint source–channel [...] Read more.

An end-to-end joint source–channel (JSC) encoding matrix and a JSC decoding scheme using the proposed bit flipping check (BFC) algorithm and controversial variable node selection-based adaptive belief propagation (CVNS-ABP) decoding algorithm are presented to improve the efficiency and reliability of the joint source–channel coding (JSCC) scheme based on double Reed–Solomon (RS) codes. The constructed coding matrix can realize source compression and channel coding of multiple sets of information data simultaneously, which significantly improves the coding efficiency. The proposed BFC algorithm uses channel soft information to select and flip the unreliable bits and then uses the redundancy of the source block to realize the error verification and error correction. The proposed CVNS-ABP algorithm reduces the influence of error bits on decoding by selecting error variable nodes (VNs) from controversial VNs and adding them to the sparsity of the parity-check matrix. In addition, the proposed JSC decoding scheme based on the BFC algorithm and CVNS-ABP algorithm can realize the connection of source and channel to improve the performance of JSC decoding. Simulation results show that the proposed BFC-based hard-decision decoding (BFC-HDD) algorithm (

ζ

= 1) and BFC-based low-complexity chase (BFC-LCC) algorithm (

ζ

= 1,

η

= 3) can achieve about 0.23 dB and 0.46 dB of signal-to-noise ratio (SNR) defined gain over the prior-art decoding algorithm at a frame error rate (FER) =

10^{- 1}

. Compared with the ABP algorithm, the proposed CVNS-ABP algorithm and BFC-CVNS-ABP algorithm achieve performance gains of 0.18 dB and 0.23 dB, respectively, at FER =

10^{- 3}

. Full article

(This article belongs to the Collection Feature Papers in Information Theory)

► Show Figures

Figure 1

14 pages, 647 KiB

Open AccessArticle

A Soar-Based Space Exploration Algorithm for Mobile Robots

by Fei Luo, Qin Zhou, Joel Fuentes, Weichao Ding and Chunhua Gu

Entropy 2022, 24(3), 426; https://doi.org/10.3390/e24030426 - 19 Mar 2022

Cited by 9 | Viewed by 3241

Abstract

Space exploration is a hot topic in the application field of mobile robots. Proposed solutions have included the frontier exploration algorithm, heuristic algorithms, and deep reinforcement learning. However, these methods cannot solve space exploration in time in a dynamic environment. This paper models [...] Read more.

Space exploration is a hot topic in the application field of mobile robots. Proposed solutions have included the frontier exploration algorithm, heuristic algorithms, and deep reinforcement learning. However, these methods cannot solve space exploration in time in a dynamic environment. This paper models the space exploration problem of mobile robots based on the decision-making process of the cognitive architecture of Soar, and three space exploration heuristic algorithms (HAs) are further proposed based on the model to improve the exploration speed of the robot. Experiments are carried out based on the Easter environment, and the results show that HAs have improved the exploration speed of the Easter robot at least 2.04 times of the original algorithm in Easter, verifying the effectiveness of the proposed robot space exploration strategy and the corresponding HAs. Full article

(This article belongs to the Special Issue Artificial Intelligence and Computational Issues in Engineering Applications)

► Show Figures

Figure 1

19 pages, 3117 KiB

Open AccessArticle

DrawnNet: Offline Hand-Drawn Diagram Recognition Based on Keypoint Prediction of Aggregating Geometric Characteristics

by Jiaqi Fang, Zhen Feng and Bo Cai

Entropy 2022, 24(3), 425; https://doi.org/10.3390/e24030425 - 19 Mar 2022

Cited by 9 | Viewed by 7324

Abstract

Offline hand-drawn diagram recognition is concerned with digitizing diagrams sketched on paper or whiteboard to enable further editing. Some existing models can identify the individual objects like arrows and symbols, but they become involved in the dilemma of being unable to understand a [...] Read more.

Offline hand-drawn diagram recognition is concerned with digitizing diagrams sketched on paper or whiteboard to enable further editing. Some existing models can identify the individual objects like arrows and symbols, but they become involved in the dilemma of being unable to understand a diagram’s structure. Such a shortage may be inconvenient to digitalization or reconstruction of a diagram from its hand-drawn version. Other methods can accomplish this goal, but they live on stroke temporary information and time-consuming post-processing, which somehow hinders the practicability of these methods. Recently, Convolutional Neural Networks (CNN) have been proved that they perform the state-of-the-art across many visual tasks. In this paper, we propose DrawnNet, a unified CNN-based keypoint-based detector, for recognizing individual symbols and understanding the structure of offline hand-drawn diagrams. DrawnNet is designed upon CornerNet with extensions of two novel keypoint pooling modules which serve to extract and aggregate geometric characteristics existing in polygonal contours such as rectangle, square, and diamond within hand-drawn diagrams, and an arrow orientation prediction branch which aims to predict which direction an arrow points to through predicting arrow keypoints. We conducted wide experiments on public diagram benchmarks to evaluate our proposed method. Results show that DrawnNet achieves

2.4 %

,

2.3 %

, and

1.7 %

recognition rate improvements compared with the state-of-the-art methods across benchmarks of FC-A, FC-B, and FA, respectively, outperforming existing diagram recognition systems on each metric. Ablation study reveals that our proposed method can effectively enable hand-drawn diagram recognition. Full article

(This article belongs to the Topic Machine and Deep Learning)

► Show Figures

Figure 1

12 pages, 452 KiB

Open AccessArticle

Low-Complexity Chase Decoding of Reed–Solomon Codes Using Channel Evaluation

by Hao Wang, Wei Zhang, Yanyan Chang, Jiajing Gao and Yanyan Liu

Entropy 2022, 24(3), 424; https://doi.org/10.3390/e24030424 - 18 Mar 2022

Cited by 3 | Viewed by 2399

Abstract

A novel time-varying channel adaptive low-complexity chase (LCC) algorithm with low redundancy is proposed, where only the necessary number of test vectors (TVs) are generated and key equations are calculated according to the channel evaluation to reduce the decoding complexity. The algorithm evaluates [...] Read more.

A novel time-varying channel adaptive low-complexity chase (LCC) algorithm with low redundancy is proposed, where only the necessary number of test vectors (TVs) are generated and key equations are calculated according to the channel evaluation to reduce the decoding complexity. The algorithm evaluates the error symbol numbers by counting the number of unreliable bits of the received code sequence and dynamically adjusts the decoding parameters, which can reduce a large number of redundant calculations in the decoding process. We provide a simplified multiplicity assignment (MA) scheme and its architecture. Moreover, a multi-functional block that can implement polynomial selection, Chien search and the Forney algorithm (PCF) is provided. On this basis, a high-efficiency LCC decoder with adaptive error-correcting capability is proposed. Compared with the state-of-the-art LCC (TV = 16) decoding, the number of TVs of our decoder was reduced by 50.4% without loss of the frame error rate (FER) performance. The hardware implementation results show that the proposed decoder achieved 81.6% reduced average latency and 150% increased throughput compared to the state-of-the-art LCC decoder. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

25 pages, 4200 KiB

Open AccessArticle

Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder

by Shichen Cao, Jingjing Li, Kenric P. Nelson and Mark A. Kon

Entropy 2022, 24(3), 423; https://doi.org/10.3390/e24030423 - 18 Mar 2022

Cited by 9 | Viewed by 5105

Abstract

We present a coupled variational autoencoder (VAE) method, which improves the accuracy and robustness of the model representation of handwritten numeral images. The improvement is measured in both increasing the likelihood of the reconstructed images and in reducing divergence between the posterior and [...] Read more.

We present a coupled variational autoencoder (VAE) method, which improves the accuracy and robustness of the model representation of handwritten numeral images. The improvement is measured in both increasing the likelihood of the reconstructed images and in reducing divergence between the posterior and a prior latent distribution. The new method weighs outlier samples with a higher penalty by generalizing the original evidence lower bound function using a coupled entropy function based on the principles of nonlinear statistical coupling. We evaluated the performance of the coupled VAE model using the Modified National Institute of Standards and Technology (MNIST) dataset and its corrupted modification C-MNIST. Histograms of the likelihood that the reconstruction matches the original image show that the coupled VAE improves the reconstruction and this improvement is more substantial when seeded with corrupted images. All five corruptions evaluated showed improvement. For instance, with the Gaussian corruption seed the accuracy improves by

10^{14}

(from

10^{- 57.2}

to

10^{- 42.9}

) and robustness improves by

10^{22}

(from

10^{- 109.2}

to

10^{- 87.0}

). Furthermore, the divergence between the posterior and prior distribution of the latent distribution is reduced. Thus, in contrast to the

β

-VAE design, the coupled VAE algorithm improves model representation, rather than trading off the performance of the reconstruction and latent distribution divergence. Full article

(This article belongs to the Special Issue Approximate Bayesian Inference)

► Show Figures

Figure 1

13 pages, 9148 KiB

Open AccessArticle

Entropy Analysis of Neonatal Electrodermal Activity during the First Three Days after Birth

by Zuzana Visnovcova, Marek Kozar, Zuzana Kuderava, Mirko Zibolen, Nikola Ferencova and Ingrid Tonhajzerova

Entropy 2022, 24(3), 422; https://doi.org/10.3390/e24030422 - 17 Mar 2022

Viewed by 2347

Abstract

The entropy-based parameters determined from the electrodermal activity (EDA) biosignal evaluate the complexity within the activity of the sympathetic cholinergic system. We focused on the evaluation of the complex sympathetic cholinergic regulation by assessing EDA using conventional indices (skin conductance level (SCL), non-specific [...] Read more.

The entropy-based parameters determined from the electrodermal activity (EDA) biosignal evaluate the complexity within the activity of the sympathetic cholinergic system. We focused on the evaluation of the complex sympathetic cholinergic regulation by assessing EDA using conventional indices (skin conductance level (SCL), non-specific skin conductance responses, spectral EDA indices), and entropy-based parameters (approximate, sample, fuzzy, permutation, Shannon, and symbolic information entropies) in newborns during the first three days of postnatal life. The studied group consisted of 50 healthy newborns (21 boys, average gestational age: 39.0 ± 0.2 weeks). EDA was recorded continuously from the feet at rest for three periods (the first day—2 h after birth, the second day—24 h after birth, and the third day—72 h after birth). Our results revealed higher SCL, spectral EDA index in a very-low frequency band, approximate, sample, fuzzy, and permutation entropy during the first compared to second and third days, while Shannon and symbolic information entropies were lower during the first day compared to other periods. In conclusion, EDA parameters seem to be sensitive in the detection of the sympathetic regulation changes in early postnatal life and which can represent an important step towards a non-invasive early diagnosis of the pathological states linked to autonomic dysmaturation in newborns. Full article

(This article belongs to the Special Issue Entropy in the Application of Biomedical Signals)

► Show Figures

Figure 1

21 pages, 1068 KiB

Open AccessFeature PaperArticle

Statistical Divergences between Densities of Truncated Exponential Families with Nested Supports: Duo Bregman and Duo Jensen Divergences

by Frank Nielsen

Entropy 2022, 24(3), 421; https://doi.org/10.3390/e24030421 - 17 Mar 2022

Cited by 11 | Viewed by 5560

Abstract

By calculating the Kullback–Leibler divergence between two probability measures belonging to different exponential families dominated by the same measure, we obtain a formula that generalizes the ordinary Fenchel–Young divergence. Inspired by this formula, we define the duo Fenchel–Young divergence and report a majorization [...] Read more.

By calculating the Kullback–Leibler divergence between two probability measures belonging to different exponential families dominated by the same measure, we obtain a formula that generalizes the ordinary Fenchel–Young divergence. Inspired by this formula, we define the duo Fenchel–Young divergence and report a majorization condition on its pair of strictly convex generators, which guarantees that this divergence is always non-negative. The duo Fenchel–Young divergence is also equivalent to a duo Bregman divergence. We show how to use these duo divergences by calculating the Kullback–Leibler divergence between densities of truncated exponential families with nested supports, and report a formula for the Kullback–Leibler divergence between truncated normal distributions. Finally, we prove that the skewed Bhattacharyya distances between truncated exponential families amount to equivalent skewed duo Jensen divergences. Full article

(This article belongs to the Special Issue Information and Divergence Measures)

► Show Figures

Graphical abstract

12 pages, 1135 KiB

Open AccessArticle

H_∞ Observer Based on Descriptor Systems Applied to Estimate the State of Charge

by Shengya Meng, Shihong Li, Heng Chi, Fanwei Meng and Aiping Pang

Entropy 2022, 24(3), 420; https://doi.org/10.3390/e24030420 - 17 Mar 2022

Cited by 6 | Viewed by 2349

Abstract

This paper proposes an

H_{\infty}

observer based on descriptor systems to estimate the state of charge (SOC). The battery’s open-current voltage is chosen as a generalized state variable, thereby avoiding the artificial derivative calculation of the algebraic equation for the SOC. Furthermore, [...] Read more.

This paper proposes an

H_{\infty}

observer based on descriptor systems to estimate the state of charge (SOC). The battery’s open-current voltage is chosen as a generalized state variable, thereby avoiding the artificial derivative calculation of the algebraic equation for the SOC. Furthermore, the observer’s dynamic performance is saved. To decrease the impacts of the uncertain noise and parameter perturbations, nonlinear

H_{\infty}

theory is implemented to design the observer. The sufficient conditions for the

H_{\infty}

observer to guarantee the disturbance suppression performance index are given and proved by the Lyapunov stability theory. This paper systematically gives the design steps of battery SOC

H_{\infty}

observers. The simulation results highlight the accuracy, transient performance, and robustness of the presented method. Full article

(This article belongs to the Special Issue Nonlinear Control Systems with Recent Advances and Applications)

► Show Figures

Figure 1

15 pages, 2596 KiB

Open AccessFeature PaperArticle

Condensation and Crystal Nucleation in a Lattice Gas with a Realistic Phase Diagram

by Santi Prestipino and Gabriele Costa

Entropy 2022, 24(3), 419; https://doi.org/10.3390/e24030419 - 17 Mar 2022

Cited by 1 | Viewed by 2282

Abstract

We reconsider model II of Orban et al. (J. Chem. Phys. 1968, 49, 1778–1783), a two-dimensional lattice-gas system featuring a crystalline phase and two distinct fluid phases (liquid and vapor). In this system, a particle prevents other particles from [...] Read more.

We reconsider model II of Orban et al. (J. Chem. Phys. 1968, 49, 1778–1783), a two-dimensional lattice-gas system featuring a crystalline phase and two distinct fluid phases (liquid and vapor). In this system, a particle prevents other particles from occupying sites up to third neighbors on the square lattice, while attracting (with decreasing strength) particles sitting at fourth- or fifth-neighbor sites. To make the model more realistic, we assume a finite repulsion at third-neighbor distance, with the result that a second crystalline phase appears at higher pressures. However, the similarity with real-world substances is only partial: Upon closer inspection, the alleged liquid–vapor transition turns out to be a continuous (albeit sharp) crossover, even near the putative triple point. Closer to the standard picture is instead the freezing transition, as we show by computing the free-energy barrier relative to crystal nucleation from the “liquid”. Full article

(This article belongs to the Special Issue Statistical Mechanics and Thermodynamics of Liquids and Crystals II)

► Show Figures

Figure 1

23 pages, 2127 KiB

Open AccessArticle

Adaptive Hurst-Sensitive Active Queue Management

by Dariusz Marek, Jakub Szyguła, Adam Domański, Joanna Domańska, Katarzyna Filus and Marta Szczygieł

Entropy 2022, 24(3), 418; https://doi.org/10.3390/e24030418 - 17 Mar 2022

Cited by 8 | Viewed by 2342

Abstract

An Active Queue Management (AQM) mechanism, recommended by the Internet Engineering Task Force (IETF), increases the efficiency of network transmission. An example of this type of algorithm can be the Random Early Detection (RED) algorithm. The behavior of the RED algorithm strictly depends [...] Read more.

An Active Queue Management (AQM) mechanism, recommended by the Internet Engineering Task Force (IETF), increases the efficiency of network transmission. An example of this type of algorithm can be the Random Early Detection (RED) algorithm. The behavior of the RED algorithm strictly depends on the correct selection of its parameters. This selection may be performed automatically depending on the network conditions. The mechanisms that adjust their parameters to the network conditions are called the adaptive ones. The example can be the Adaptive RED (ARED) mechanism, which adjusts its parameters taking into consideration the traffic intensity. In our paper, we propose to use an additional traffic parameter to adjust the AQM parameters—degree of self-similarity—expressed using the Hurst parameter. In our study, we propose the modifications of the well-known AQM algorithms: ARED and fractional order

P I^{α} D^{β}

and the algorithms based on neural networks that are used to automatically adjust the AQM parameters using the traffic intensity and its degree of self-similarity. We use the Fluid Flow approximation and the discrete event simulation to evaluate the behavior of queues controlled by the proposed adaptive AQM mechanisms and compare the results with those obtained with their basic counterparts. In our experiments, we analyzed the average queue occupancies and packet delays in the communication node. The obtained results show that considering the degree of self-similarity of network traffic in the process of AQM parameters determination enabled us to decrease the average queue occupancy and the number of rejected packets, as well as to reduce the transmission latency. Full article

► Show Figures

Figure 1

12 pages, 894 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

A Robust Protocol for Entropy Measurement in Mesoscopic Circuits

by Timothy Child, Owen Sheekey, Silvia Lüscher, Saeed Fallahi, Geoffrey C. Gardner, Michael Manfra and Joshua Folk

Entropy 2022, 24(3), 417; https://doi.org/10.3390/e24030417 - 17 Mar 2022

Cited by 13 | Viewed by 3461

Abstract

Previous measurements utilizing Maxwell relations to measure change in entropy, S, demonstrated remarkable accuracy in measuring the spin-1/2 entropy of electrons in a weakly coupled quantum dot. However, these previous measurements relied upon prior knowledge of the charge transition lineshape. This had [...] Read more.

Previous measurements utilizing Maxwell relations to measure change in entropy, S, demonstrated remarkable accuracy in measuring the spin-1/2 entropy of electrons in a weakly coupled quantum dot. However, these previous measurements relied upon prior knowledge of the charge transition lineshape. This had the benefit of making the quantitative determination of entropy independent of scale factors in the measurement itself but at the cost of limiting the applicability of the approach to simple systems. To measure the entropy of more exotic mesoscopic systems, a more flexible analysis technique may be employed; however, doing so requires a precise calibration of the measurement. Here, we give details on the necessary improvements made to the original experimental approach and highlight some of the common challenges (along with strategies to overcome them) that other groups may face when attempting this type of measurement. Full article

(This article belongs to the Special Issue Nature of Entropy and Its Direct Metrology)

► Show Figures

Figure 1

17 pages, 5010 KiB

Open AccessArticle

Heat Transfer and Pressure Drop of Nanofluid with Rod-like Particles in Turbulent Flows through a Curved Pipe

by Wenqian Lin, Ruifang Shi and Jianzhong Lin

Entropy 2022, 24(3), 416; https://doi.org/10.3390/e24030416 - 16 Mar 2022

Cited by 10 | Viewed by 2374

Abstract

Pressure drop, heat transfer, and energy performance of ZnO/water nanofluid with rodlike particles flowing through a curved pipe are studied in the range of Reynolds number 5000 ≤ Re ≤ 30,000, particle volume concentration 0.1% ≤ Φ ≤ 5%, Schmidt number 10⁴ [...] Read more.

Pressure drop, heat transfer, and energy performance of ZnO/water nanofluid with rodlike particles flowing through a curved pipe are studied in the range of Reynolds number 5000 ≤ Re ≤ 30,000, particle volume concentration 0.1% ≤ Φ ≤ 5%, Schmidt number 10⁴ ≤ Sc ≤ 3 × 10⁵, particle aspect ratio 2 ≤ λ ≤ 14, and Dean number 5 × 10³ ≤ De ≤ 1.5 × 10⁴. The momentum and energy equations of nanofluid, together with the equation of particle number density for particles, are solved numerically. Some results are validated by comparing with the experimental results. The effect of Re, Φ, Sc, λ, and De on the friction factor f and Nusselt number Nu is analyzed. The results showed that the values of f are increased with increases in Φ, Sc, and De, and with decreases in Re and λ. The heat transfer performance is enhanced with increases in Re, Φ, λ, and De, and with decreases in Sc. The ratio of energy PEC for nanofluid to base fluid is increased with increases in Re, Φ, λ, and De, and with decreases in Sc. Finally, the formula of ratio of energy PEC for nanofluid to base fluid as a function of Re, Φ, Sc, λ, and De is derived based on the numerical data. Full article

(This article belongs to the Special Issue Entropy Generation Analysis in Near-Wall Turbulent Flow)

► Show Figures

Figure 1

17 pages, 1109 KiB

Open AccessArticle

Adversarially Training MCMC with Non-Volume-Preserving Flows

by Shaofan Liu and Shiliang Sun

Entropy 2022, 24(3), 415; https://doi.org/10.3390/e24030415 - 16 Mar 2022

Viewed by 2243

Abstract

Recently, flow models parameterized by neural networks have been used to design efficient Markov chain Monte Carlo (MCMC) transition kernels. However, inefficient utilization of gradient information of the target distribution or the use of volume-preserving flows limits their performance in sampling from multi-modal [...] Read more.

Recently, flow models parameterized by neural networks have been used to design efficient Markov chain Monte Carlo (MCMC) transition kernels. However, inefficient utilization of gradient information of the target distribution or the use of volume-preserving flows limits their performance in sampling from multi-modal target distributions. In this paper, we treat the training procedure of the parameterized transition kernels in a different manner and exploit a novel scheme to train MCMC transition kernels. We divide the training process of transition kernels into the exploration stage and training stage, which can make full use of the gradient information of the target distribution and the expressive power of deep neural networks. The transition kernels are constructed with non-volume-preserving flows and trained in an adversarial form. The proposed method achieves significant improvement in effective sample size and mixes quickly to the target distribution. Empirical results validate that the proposed method is able to achieve low autocorrelation of samples and fast convergence rates, and outperforms other state-of-the-art parameterized transition kernels in varieties of challenging analytically described distributions and real world datasets. Full article

► Show Figures

Figure 1

17 pages, 2581 KiB

Open AccessArticle

Speaker Recognition Using Constrained Convolutional Neural Networks in Emotional Speech

by Nikola Simić, Siniša Suzić, Tijana Nosek, Mia Vujović, Zoran Perić, Milan Savić and Vlado Delić

Entropy 2022, 24(3), 414; https://doi.org/10.3390/e24030414 - 16 Mar 2022

Cited by 11 | Viewed by 3140

Abstract

Speaker recognition is an important classification task, which can be solved using several approaches. Although building a speaker recognition model on a closed set of speakers under neutral speaking conditions is a well-researched task and there are solutions that provide excellent performance, the [...] Read more.

Speaker recognition is an important classification task, which can be solved using several approaches. Although building a speaker recognition model on a closed set of speakers under neutral speaking conditions is a well-researched task and there are solutions that provide excellent performance, the classification accuracy of developed models significantly decreases when applying them to emotional speech or in the presence of interference. Furthermore, deep models may require a large number of parameters, so constrained solutions are desirable in order to implement them on edge devices in the Internet of Things systems for real-time detection. The aim of this paper is to propose a simple and constrained convolutional neural network for speaker recognition tasks and to examine its robustness for recognition in emotional speech conditions. We examine three quantization methods for developing a constrained network: floating-point eight format, ternary scalar quantization, and binary scalar quantization. The results are demonstrated on the recently recorded SEAC dataset. Full article

(This article belongs to the Special Issue Methods in Artificial Intelligence and Information Processing)

► Show Figures

Figure 1

2 pages, 189 KiB

Open AccessCorrection

Correction: Halun et al. Investigation of Ring and Star Polymers in Confined Geometries: Theory and Simulations. Entropy 2021, 23, 242

by Joanna Halun, Pawel Karbowniczek, Piotr Kuterba and Zoriana Danel

Entropy 2022, 24(3), 413; https://doi.org/10.3390/e24030413 - 16 Mar 2022

Viewed by 1538

Abstract

The authors wish to make the following correction to this paper [...] Full article

(This article belongs to the Special Issue Selected Papers from 45th Conference of the Middle European Cooperation in Statistical Physics)

23 pages, 7455 KiB

Open AccessArticle

ABCAttack: A Gradient-Free Optimization Black-Box Attack for Fooling Deep Image Classifiers

by Han Cao, Chengxiang Si, Qindong Sun, Yanxiao Liu, Shancang Li and Prosanta Gope

Entropy 2022, 24(3), 412; https://doi.org/10.3390/e24030412 - 15 Mar 2022

Cited by 5 | Viewed by 2713

Abstract

The vulnerability of deep neural network (DNN)-based systems makes them susceptible to adversarial perturbation and may cause classification task failure. In this work, we propose an adversarial attack model using the Artificial Bee Colony (ABC) algorithm to generate adversarial samples without the need [...] Read more.

The vulnerability of deep neural network (DNN)-based systems makes them susceptible to adversarial perturbation and may cause classification task failure. In this work, we propose an adversarial attack model using the Artificial Bee Colony (ABC) algorithm to generate adversarial samples without the need for a further gradient evaluation and training of the substitute model, which can further improve the chance of task failure caused by adversarial perturbation. In untargeted attacks, the proposed method obtained 100%, 98.6%, and 90.00% success rates on the MNIST, CIFAR-10 and ImageNet datasets, respectively. The experimental results show that the proposed ABCAttack can not only obtain a high attack success rate with fewer queries in the black-box setting, but also break some existing defenses to a large extent, and is not limited by model structure or size, which provides further research directions for deep learning evasion attacks and defenses. Full article

(This article belongs to the Special Issue Information Theory and Its Applications in Multimedia Security and Processing)

► Show Figures

Figure 1

16 pages, 3543 KiB

Open AccessArticle

Multiband Spectrum Sensing Based on the Sample Entropy

by Yanqueleth Molina-Tenorio, Alfonso Prieto-Guerrero and Rafael Aguilar-Gonzalez

Entropy 2022, 24(3), 411; https://doi.org/10.3390/e24030411 - 15 Mar 2022

Cited by 3 | Viewed by 2144

Abstract

Cognitive radios represent a real alternative to the scarcity of the radio spectrum. One of the primary tasks of these radios is the detection of possible gaps in a given bandwidth used by licensed users (called also primary users). This task, called spectrum [...] Read more.

Cognitive radios represent a real alternative to the scarcity of the radio spectrum. One of the primary tasks of these radios is the detection of possible gaps in a given bandwidth used by licensed users (called also primary users). This task, called spectrum sensing, requires high precision in determining these gaps, maximizing the probability of detection. The design of spectrum sensing algorithms also requires innovative hardware and software solutions for real-time implementations. In this work, a technique to determine possible primary users’ transmissions in a wide frequency interval (multiband spectrum sensing) from the perspective of cognitive radios is presented. The proposal is implemented in a real wireless communications environment using low-cost hardware considering the sample entropy as a decision rule. To validate its feasibility for real-time implementation, a simulated scenario was first tested. Simulation and real-time implementations results were compared with the Higuchi fractal dimension as a decision rule. The encouraging results show that sample entropy correctly detects noise or a possible primary user transmission, with a probability of success around 0.99, and the number of samples with errors at the start and end of frequency edges of transmissions is, on average, only 12 samples. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

15 pages, 1756 KiB

Open AccessArticle

The Potential of a Thick Present through Undefined Causality and Non-Locality

by Alessandro Capurso

Entropy 2022, 24(3), 410; https://doi.org/10.3390/e24030410 - 15 Mar 2022

Cited by 1 | Viewed by 4479

Abstract

This paper elaborates on the interpretation of time and entanglement, offering insights into the possible ontological nature of information in the emergence of spacetime, towards a quantum description of gravity. We first investigate different perspectives on time and identify in the idea of [...] Read more.

This paper elaborates on the interpretation of time and entanglement, offering insights into the possible ontological nature of information in the emergence of spacetime, towards a quantum description of gravity. We first investigate different perspectives on time and identify in the idea of a “thick present” the only element of reality needed to describe evolution, differences, and relations. The thick present is connected to a spacetime information “sampling rate”, and it is intended as a time symmetric potential bounded between a causal past of irreversible events and a still open future. From this potential, spacetime emerges in each instant as a space-like foliation (in a description based on imaginary paths). In the second part, we analyze undefined causal orders to understand how their potential could persist along the thick present instants. Thanks to a C-NOT logic and the concept of an imaginary time, we derive a description of entanglement as the potential of a logically consistent open choice among imaginary paths. We then conceptually map the imaginary paths identified in the entanglement of the undefined orders to Closed Time-like Curves (CTC) in the thick present. Considering a universe described through information, CTC are interpreted as “memory loops”, elementary structures encoding the information potential related to the entanglement in both time and space, manifested as undefined causality and non-locality in the emerging foliation. We conclude by suggesting a possible extension of the introduced concepts in a holographic perspective. Full article

(This article belongs to the Special Issue Exploring Spacetime Emergence from the Quantum Level)

► Show Figures

Figure 1

15 pages, 468 KiB

Open AccessArticle

The Structure and First-Passage Properties of Generalized Weighted Koch Networks

by Jing Su, Mingjun Zhang and Bing Yao

Entropy 2022, 24(3), 409; https://doi.org/10.3390/e24030409 - 15 Mar 2022

Cited by 1 | Viewed by 2060

Abstract

Characterizing the topology and random walk of a random network is difficult because the connections in the network are uncertain. We propose a class of the generalized weighted Koch network by replacing the triangles in the traditional Koch network with a graph [...] Read more.

Characterizing the topology and random walk of a random network is difficult because the connections in the network are uncertain. We propose a class of the generalized weighted Koch network by replacing the triangles in the traditional Koch network with a graph

R_{s}

according to probability

0 \leq p \leq 1

and assign weight to the network. Then, we determine the range of several indicators that can characterize the topological properties of generalized weighted Koch networks by examining the two models under extreme conditions,

p = 0

and

p = 1

, including average degree, degree distribution, clustering coefficient, diameter, and average weighted shortest path. In addition, we give a lower bound on the average trapping time (ATT) in the trapping problem of generalized weighted Koch networks and also reveal the linear, super-linear, and sub-linear relationships between ATT and the number of nodes in the network. Full article

► Show Figures

Figure 1

15 pages, 980 KiB

Open AccessArticle

A Hybrid Method Using HAVOK Analysis and Machine Learning for Predicting Chaotic Time Series

by Jinhui Yang, Juan Zhao, Junqiang Song, Jianping Wu, Chengwu Zhao and Hongze Leng

Entropy 2022, 24(3), 408; https://doi.org/10.3390/e24030408 - 15 Mar 2022

Cited by 10 | Viewed by 3734

Abstract

The prediction of chaotic time series systems has remained a challenging problem in recent decades. A hybrid method using Hankel Alternative View Of Koopman (HAVOK) analysis and machine learning (HAVOK-ML) is developed to predict chaotic time series. HAVOK-ML simulates the time series by [...] Read more.

The prediction of chaotic time series systems has remained a challenging problem in recent decades. A hybrid method using Hankel Alternative View Of Koopman (HAVOK) analysis and machine learning (HAVOK-ML) is developed to predict chaotic time series. HAVOK-ML simulates the time series by reconstructing a closed linear model so as to achieve the purpose of prediction. It decomposes chaotic dynamics into intermittently forced linear systems by HAVOK analysis and estimates the external intermittently forcing term using machine learning. The prediction performance evaluations confirm that the proposed method has superior forecasting skills compared with existing prediction methods. Full article

(This article belongs to the Topic Artificial Intelligence and Computational Methods: Modeling, Simulations and Optimization of Complex Systems)

► Show Figures

Figure 1

11 pages, 408 KiB

Open AccessArticle

f-Gintropy: An Entropic Distance Ranking Based on the Gini Index

by Tamás Sándor Biró, András Telcs, Máté Józsa and Zoltán Néda

Entropy 2022, 24(3), 407; https://doi.org/10.3390/e24030407 - 14 Mar 2022

Cited by 3 | Viewed by 2137

Abstract

We consider an entropic distance analog quantity based on the density of the Gini index in the Lorenz map, i.e., gintropy. Such a quantity might be used for pairwise mapping and ranking between various countries and regions based on income and wealth inequality. [...] Read more.

We consider an entropic distance analog quantity based on the density of the Gini index in the Lorenz map, i.e., gintropy. Such a quantity might be used for pairwise mapping and ranking between various countries and regions based on income and wealth inequality. Its generalization to f-gintropy, using a function of the income or wealth value, distinguishes between regional inequalities more sensitively than the original construction. Full article

(This article belongs to the Special Issue Information Geometry, Complexity Measures and Data Analysis)

► Show Figures

Figure 1

10 pages, 1932 KiB

Open AccessArticle

An Approach for Security Enhancement of Certain Encryption Schemes Employing Error Correction Coding and Simulated Synchronization Errors

by Miodrag J. Mihaljević, Lianhai Wang and Shujiang Xu

Entropy 2022, 24(3), 406; https://doi.org/10.3390/e24030406 - 14 Mar 2022

Cited by 2 | Viewed by 2070

Abstract

An approach for the cryptographic security enhancement of encryption is proposed and analyzed. The enhancement is based on the employment of a coding scheme and degradation of the ciphertext. From the perspective of the legitimate parties that share a secret key, the degradation [...] Read more.

An approach for the cryptographic security enhancement of encryption is proposed and analyzed. The enhancement is based on the employment of a coding scheme and degradation of the ciphertext. From the perspective of the legitimate parties that share a secret key, the degradation appears as a transmission of the ciphertext through a binary erasure channel. On the other hand, from the perspective of an attacker the degradation appears as a transmission of the ciphertext over a binary deletion channel. Cryptographic security enhancement is analyzed based on the capacity of the related binary deletion channel. An illustrative implemementation framework is pointed out. Full article

(This article belongs to the Special Issue Information Theoretical Security and Privacy)

► Show Figures

Figure 1

19 pages, 3314 KiB

Open AccessArticle

Tipping the Balance: A Criticality Perspective

by Indrani Bose

Entropy 2022, 24(3), 405; https://doi.org/10.3390/e24030405 - 14 Mar 2022

Cited by 3 | Viewed by 2545

Abstract

Cell populations are often characterised by phenotypic heterogeneity in the form of two distinct subpopulations. We consider a model of tumour cells consisting of two subpopulations: non-cancer promoting (NCP) and cancer-promoting (CP). Under steady state conditions, the model has similarities with a well-known [...] Read more.

Cell populations are often characterised by phenotypic heterogeneity in the form of two distinct subpopulations. We consider a model of tumour cells consisting of two subpopulations: non-cancer promoting (NCP) and cancer-promoting (CP). Under steady state conditions, the model has similarities with a well-known model of population genetics which exhibits a purely noise-induced transition from unimodality to bimodality at a critical value of the noise intensity

σ^{2}

. The noise is associated with the parameter λ representing the system-environment coupling. In the case of the tumour model, λ has a natural interpretation in terms of the tissue microenvironment which has considerable influence on the phenotypic composition of the tumour. Oncogenic transformations give rise to considerable fluctuations in the parameter. We compute the

λ - σ^{2}

phase diagram in a stochastic setting, drawing analogies between bifurcations and phase transitions. In the region of bimodality, a transition from a state of balance to a state of dominance, in terms of the competing subpopulations, occurs at λ = 0. Away from this point, the NCP (CP) subpopulation becomes dominant as λ changes towards positive (negative) values. The variance of the steady state probability density function as well as two entropic measures provide characteristic signatures at the transition point. Full article

(This article belongs to the Special Issue The Principle of Dynamical Criticality)

► Show Figures

Figure 1

38 pages, 10253 KiB

Open AccessArticle

B-DP: Dynamic Collection and Publishing of Continuous Check-In Data with Best-Effort Differential Privacy

by Youqin Chen, Zhengquan Xu, Jianzhang Chen and Shan Jia

Entropy 2022, 24(3), 404; https://doi.org/10.3390/e24030404 - 14 Mar 2022

Cited by 3 | Viewed by 2375

Abstract

Differential privacy (DP) has become a de facto standard to achieve data privacy. However, the utility of DP solutions with the premise of privacy priority is often unacceptable in real-world applications. In this paper, we propose the best-effort differential privacy (B-DP) to promise [...] Read more.

Differential privacy (DP) has become a de facto standard to achieve data privacy. However, the utility of DP solutions with the premise of privacy priority is often unacceptable in real-world applications. In this paper, we propose the best-effort differential privacy (B-DP) to promise the preference for utility first and design two new metrics including the point belief degree and the regional average belief degree to evaluate its privacy from a new perspective of preference for privacy. Therein, the preference for privacy and utility is referred to as expected privacy protection (EPP) and expected data utility (EDU), respectively. We also investigate how to realize B-DP with an existing DP mechanism (KRR) and a newly constructed mechanism (EXP

_{Q}

) in the dynamic check-in data collection and publishing. Extensive experiments on two real-world check-in datasets verify the effectiveness of the concept of B-DP. Our newly constructed EXP

_{Q}

can also satisfy a better B-DP than KRR to provide a good trade-off between privacy and utility. Full article

(This article belongs to the Special Issue Adversarial Intelligence: Secrecy, Privacy, and Robustness)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Entropy, Volume 24, Issue 3 (March 2022) – 127 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI