Permutation Entropy & Its Interdisciplinary Applications

Dear Colleagues,

Physics, as well as other scientific disciplines, such as biology or finance, can be considered observational sciences, that is, they try to infer properties of an unfamiliar system from the analysis of a measured time record of its behavior (time series). Dynamical systems are systems that evolve in time. In practice, in general, one may only be able to measure a scalar time series X(t) which may be a function of variables V = {v₁, v₂, …, v_k} describing the underlying dynamics (i.e., dV/dt = f(V)). Then, the natural question is, how much we can learn from X(t) about the dynamics of the system. In a more formal way, given a system, be it natural or man-made, and given an observable of such a system whose evolution can be tracked through time, a natural question arises: how much information is this observable encoding about the dynamics of the underlying system? The information content of a system is typically evaluated via a probability distribution function (PDF) P describing the apportionment of some measurable or observable quantity, generally a time series X(t) = {x_t, t =1, …, M}. Quantifying the information content of a given observable is therefore largely tantamount to characterizing its probability distribution. This is often done with a wide family of measures called Information Theory quantifiers (i.e., Shannon entropy and generalized entropy forms, relative entropies, Fisher information, statistical complexity, etc.). We can define Information Theory quantifiers as measures able to characterize relevant properties of the PDF associated with these time series, and in this way we should judiciously extract information on the dynamical system under study.

The evaluation of the Information Theory quantifiers supposes some prior knowledge about the system; specifically, a probability distribution associated to the time series under analysis should be provided beforehand. The determination of the most adequate PDF is a fundamental problem because the PDF P and the sample space Ω are inextricably linked. Usual methodologies assign a symbol from a finite alphabet A to each time point of the series X(t), thus creating a symbolic sequence that can be regarded to as a non causal coarse grained description of the time series under consideration. As a consequence, order relations and the time scales of the dynamics are lost. The usual histogram technique corresponds to this kind of assignment. Causal information may be duly incorporated if information about the past dynamics of the system is included in the symbolic sequence, i.e., symbols of alphabet A are assigned to a portion of the phase-space or trajectory.

Many methods have been proposed for a proper selection of the probability space (Ω, P). Among others, of non causal coarse grained type, we can mention frequency counting, procedures based on amplitude statistics, binary symbolic dynamics, Fourier analysis, or wavelet transform. The suitability of each of the proposed methodologies depends on the peculiarity of data, such as stationarity, length of the series, the variation of the parameters, the level of noise contamination, etc. In all these cases, global aspects of the dynamics can be somehow captured, but the different approaches are not equivalent in their ability to discern all relevant physical details.

In a seminal paper, Bandt and Pompe (BP) [Permutation Entropy: A Natural Complexity Measure for Time Series. Phys. Rev. Lett. 1972, 88, 174102] introduced a simple and robust symbolic methodology that takes into account the time causality of the time series (causal coarse grained methodology) by comparing neighboring values in a time series. The symbolic data are (i) created by ranking the values of the series; and (ii) defined by reordering the embedded data in ascending order, which is tantamount to a phase space reconstruction with embedding dimension (pattern length) D ≥ 2, D ∈ ℕ and time lag τ ∈ ℕ. In this way, it is possible to quantify the diversity of the ordering symbols (patterns) derived from a scalar time series. Note that the appropriate symbol sequence arises naturally from the time series, and no model-based assumptions are needed. In fact, the necessary “partitions” are devised by comparing the order of neighboring relative values rather than by apportioning amplitudes according to different levels. This technique, as opposed to most of those in current practice, takes into account the temporal structure of the time series generated by the physical process under study. As such, it allows us to uncover important details concerning the ordinal structure of the time series and can also yield information about temporal correlation.

It is clear that this type of analysis of a time series entails losing details of the original series' amplitude information. Nevertheless, by just referring to the series' intrinsic structure, a meaningful difficulty reduction has indeed been achieved by BP with regard to the description of complex systems. The symbolic representation of time series by recourse to a comparison of consecutive (τ = 1 ) or nonconsecutive (τ > 1 ) values allows for an accurate empirical reconstruction of the underlying phase-space, even in the presence of weak (observational and dynamic) noise. Furthermore, the ordinal patterns associated with the PDF are invariant with respect to nonlinear monotonous transformations. Accordingly, nonlinear drifts or scaling artificially introduced by a measurement device will not modify the estimation of quantifiers, a nice property if one deals with experimental data. These advantages make the BP methodology more convenient than conventional methods based on range partitioning, i.e., a PDF based on histograms.

Additional advantages of the method reside in (i) its simplicity (it requires few parameters: the pattern length/embedding dimension D and the time lag τ, and (ii) and the extremely fast nature of the calculation process. The BP methodology can be applied not only to time series representative of low dimensional dynamical systems, but also to any type of time series (regular, chaotic, noisy, or reality based). In fact, the existence of an attractor in the D-dimensional phase space is not assumed. The only condition for the applicability of the BP method is a very weak stationary assumption: for k ≤ D, the probability for x_t < x_t+k should not depend on t.

In summary, the Bandt–Pompe proposal for associating probability distributions to time series (of an underlying symbolic nature) constitutes a significant advance in the study of complex dynamical systems, as well as a clear improvement in the quality of Information Theory-based quantifiers. The power and usefulness of the Bandt–Pompe approach has been validated in many subsequent papers, as shown by the fast increment of the number of citations of the cornerstone paper through time. Many extensions of the original methodogy have been proposed in order to include the time series amplitude in the patterns’ contributions, as well as extensions for multichanel time series, amongh others. The Bandt–Pompe permutation PDF applications include a great variety of fields such as nonlinear dynamics and stochastic system descriptions; physics of lasers; mechanical engineering; plasma physics; climate time series; econophysics; neural dynamics; brain activity and epilepsy; electrocardiogram; and anesthesia, to cite just some of the many interdisciplinary applications.

Dr. Osvaldo Anibal Rosso
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

• Information Theory Quantifiers
• Time Causality
• Permutation Entropy
• Interdisciplinary Applications