On the Connections of Generalized Entropies With Shannon and Kolmogorov–Sinai Entropies

Abstract

We consider the concept of generalized Kolmogorov–Sinai entropy, where instead of the Shannon entropy function, we consider an arbitrary concave function defined on the unit interval, vanishing in the origin. Under mild assumptions on this function, we show that this isomorphism invariant is linearly dependent on the Kolmogorov–Sinai entropy.

1. Introduction

The Boltzmann-Gibbs entropy and the Shannon entropy are measures that lie on the foundations of statistical physics and modern information theory, respectively. The dynamical counterparts of the latter—the dynamical and the Kolmogorov-Sinai entropy—had a great impact on the modern theory of dynamical systems and mathematical physics (see, e.g., [1,2]). They were extensively studied and successfully applied, among others, to statistical physics and quantum information. It appeared to be an exceptionally powerful tool for exploring nonlinear systems. One of the biggest advantages of the Kolmogorov-Sinai entropies lies in the fact that they make it possible to distinguish the formally regular systems (those with the measure-theoretic entropy equal to zero) from the chaotic ones (with positive entropy, which implies the positivity of topological entropy [3]).

The Kolmogorov-Sinai entropy of a given transformation T acting on a probability space (X, Σ, μ) is defined as the supremum over all finite measurable partitions $𝒫$ of the dynamical entropy of T with respect to $𝒫$, denoted by h(T, $𝒫$). As a dynamical counterpart of Shannon entropy, the entropy of transformation T with respect to a given partition $𝒫$ is defined as the limit of the sequence ${\left(\frac{1}{n}H({𝒫}_n)\right)}_{n=1}^{\infty }$, where:

$$H({𝒫}_n)=\sum _{A\in {𝒫}_n}\eta (\mu (A))$$

with η being the Shannon function given by η(x) = −x log x for x > 0 with η(0) : = 0, and $𝒫$_n is the join partition of partitions T⁻ⁱ$𝒫$ for i = 0, ..., n – 1. The existence of the limit in the definition of the dynamical entropy follows from the subadditivity of η. The most common interpretation of this quantity is the average (over time and the phase space) one-step gain of information about the initial state. Taking the supremum over all finite partitions, we obtain an isomorphism invariant that measures the rate of producing randomness (chaos) by the system.

Since Shannon’s seminal paper [4], several refinements of the concept of Shannon static entropy were considered (see, e.g., Rényi [5], Arimoto [6], Wu, Verdú [7] and Csiszár’s survey article [8]). Their dynamical and measure-theoretic counterparts were also considered by a few authors. De Paly [9] proposed generalized dynamical entropies based on the concept of the relative static entropies. Unfortunately, it appeared that, despite some special cases [9,10], the explicit calculation of this invariant may not be possible. Grassberger and Procaccia proposed in [11] a dynamical counterpart of the well-known generalization of Shannon entropy—the Rényi entropy—and its measure-theoretic counterpart were considered by Takens and Verbitski. They showed that for ergodic transformations with positive measure-theoretic entropy, Rényi entropies of a measure-theoretic transformation are either infinite or equal to the measure-theoretic entropy [12]. The answer for non-ergodic aperiodic transformations is different: for Rényi entropies of order α > 1, they are equal to the essential infimum of measure-theoretic entropies of measures forming the decomposition of a given measure into ergodic components, while for α < 1, they are still infinite [13]. In particular, this means that Rényi entropies of order α < 1 are metric invariants sensitive to ergodicity. A similar generalization was made by Mesón and Vericat [14–16] for so-called Havrda–Charvát–Tsallis entropy [17], and their results were similar to the ones obtained by Takens and Verbitski in [12].

In this paper, we present a generalization of the concept of dynamical entropy. This was made for few reasons. First of all, the entropy used in the theory of dynamical systems is a natural isomorphism invariant. However, outside of the class of Bernoulli automorphisms, it is not a complete invariant, i.e., two systems with the same entropy need not be isomorphic. In particular, it is not applicable to a wide class of systems with zero entropy. Moreover, we can ask whether considering of a function different than the Shannon function (at the expense of losing its interpretations from the information theory) can give a significantly new invariant. This can show which properties of the Shannon function are crucial for the usage of entropy in dynamical systems. Finally, different dynamical generalizations of entropy have been used by the physicists for years [18–22]. However, up till now, there have not been many rigorous results. We hope that this note will help to fill in this gap.

Our approach is based on the generalization suggested by Rényi and extended by Arimoto applied to the dynamical case. Instead of the Shannon function η, we consider a concave function g :[0, 1] ↦ ℝ, such that $\underset{x\to 0^{+}}{\text{lim}}g(x)=g\left(0\right)=0$ and define the dynamical g-entropy of the finite partition $𝒫$ as:

$$h(g,T,𝒫)=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}\sum _{A\in {𝒫}_n}g(\mu (A)).$$

The behavior of the quotient g(x)/η(x) as x converges to zero appears to be crucial for our considerations. Defining:

$$\text{Ci}(g):=\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g(x)}{\eta (x)}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{and}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{Cs}(g):=\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g(x)}{\eta (x)}$$

we will prove that:

$$\text{Ci}(g)\cdot h(T,𝒫)\le h(g,T,𝒫)\le \text{Cs}(g)\cdot h(T,𝒫).$$

In the case of Ci(g) = ∞, we will show that in every aperiodic system and for every γ ≥ 0, there exists a finite partition $𝒫$, such that h(g, T, $𝒫$) ≥ γ.

Considering the supremum over all partitions, we obtain a Kolmogorov entropy-like isomorphism invariant, which we will call the measure-theoretic g-entropy of a transformation with respect to an invariant measure. One might ask whether this invariant may give any new information about the system. We will prove (Theorem 5) that for g with Cs(g) < ∞, this new invariant is linearly dependent on Kolmogorov–Sinai entropy. It shows that Shannon entropy function is the special one from the point of view of the theory of dynamical entropies—it is the most natural one—not only does it have all of the properties that the entropy function should have [1], but also considering different entropy functions, we will not obtain an essentially different invariant. This result might have the other interpretation. Ornstein and Weiss showed in [23] that every finitely observable invariant for the class of all ergodic processes has to be a continuous function of the entropy. It is easy to see that any continuous function of the entropy is finitely observable; one simply composes the entropy estimators with the continuous function itself. In other words, an isomorphism invariant is finitely observable if and only if it is a continuous function of the Kolmogorov-Sinai entropy. Therefore, our result implies that the generalized measure-theoretic entropy is, in fact, finitely observable. It should be possible to give a more direct proof of the finite observability of the generalized measure-theoretic entropy, but the proof cannot be easier than the proof that entropy itself is finitely observable [24,25]. On the other hand, different entropy functions might be still of use, e.g., in the case of zero entropy systems, where we may consider generalizations of concepts used in this case: entropy convergence rates [26,27], generalized topological entropies [28] or entropy dimensions [29]. The generalization of entropy convergence rates can be found in [30]. The result implies that, from the point of view of the theory of entropy in dynamical systems, the crucial property of the Shannon function η is its behavior in the neighborhood of zero.

The note is organized as follows: in the next section, we introduce the dynamical g-entropy and establish its basic properties. The subsequent section is devoted to the construction of a zero dynamical entropy process with sufficiently large g-entropy. Finally, in the last section, we define a measure-theoretic g-entropy of a transformation and show connections between this new invariant and the Kolmogorov-Sinai entropy.

2. Results

2.1. Basic Facts and Definitions

Let (X, Σ, μ) be a Lebesgue space, and let g : [0, 1] ↦ ℝ be a concave function with $g\left(0\right)=\underset{x\to 0^{+}}{\text{lim}}g(x)=0$ (We might assume only that g(0) = 0, but then, the idea of the dynamical g-entropy would fail, since if $𝒫$_n+1 ≠ $𝒫$_n for every n and $\underset{x\to 0^{+}}{\text{lim}}g(x)\ne 0$, then the dynamical g-entropy of the partition $𝒫$ would be infinite. Therefore, if g is not well-defined at zero, we will assume that $g\left(0\right):=\underset{x\to 0^{+}}{\text{lim}}g(x)$.). By $𝒢$₀, we will denote the set of all such functions. Every g ∈$𝒢$₀ is subadditive, i.e., g(x + y) ≤ g(x)+ g(y) for every x, y, x + y ∈ [0, 1], and quasi-homogenic, i.e., φ_g :(0, 1] → ℝ defined by φ_g(x):= g(x)/x is decreasing (see [31]). If g is fixed, we will omit the index, writing just φ.

Any finite family of pairwise disjoint subsets of X, such that $\underset{A_i\in 𝒫}{\cup }{A}_i=X$, is called a partition. The set of all finite partitions will be denoted by 𝔅. Foragiven $𝒫$ ∈ 𝔅, we define the g-entropy of the partition $𝒫$ as:

$$H(g,𝒫):=\sum _{A\in 𝒫}g(\mu (A)).$$

(1)

For g = η, the latter is equal to the Shannon entropy of the partition $𝒫$. For $𝒫$, $𝒬$ ∈ 𝔅 of the space X, we define a new partition $𝒫$ ∨ $𝒬$ (the joint partition of $𝒫$ and $𝒬$) consisting of the subsets of the form B ∩ C, where B ∈ $𝒫$ and C ∈ $𝒬$.

2.1.1. Dynamical g-Entropies

For an automorphism T : X ↦ X and a partition $𝒫$ = {E₁, ..., E_k}, we put:

$$T^{-j}𝒫:=\left\{T^{-j}{E}_1,\dots ,T^{-j}{E}_k\right\}$$

and

$${𝒫}_n=𝒫\vee T^{-1}𝒫\vee \dots \vee T^{-n+1}𝒫.$$

Now, for a given g ∈ $𝒢$₀ and a finite partition $𝒫$, we can define the dynamical g-entropy of the transformation T with respect to $𝒫$ as:

$$h_{\mu }(g,T,𝒫)=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}H(g,{𝒫}_n).$$

(2)

Alternatively we will call it the g-entropy of the process (X, Σ, μ, T, $𝒫$). If the dynamical system (X, Σ, T, μ) is fixed, then we omit T, writing just h(g, $𝒫$). As in the case of Shannon dynamical entropies, we are interested in the existence of the limit of ${\left(\frac{1}{n}H(g,{𝒫}_n)\right)}_{n=1}^{\infty }$. If g = η, we obtain the Shannon dynamical entropy h(T, $𝒫$). However, in the general case, we cannot replace an upper limit in Equation (2) by the limit, since it might not exist. The existence of the limit in the case of the Shannon function follows from the subadditivity of static Shannon entropy. This property has every subderivative function, i.e., a function for which the inequality g(xy) ≤ xg(y) + yg(x) holds for any x, y ∈ [0, 1], but this is not true in general (an appropriate example will be given in Section 2.1.3). Therefore, we propose more general classes of functions for which this limit exists:

$${𝒢}_0^0:=\left\{g\in {𝒢}_0|\underset{x\to 0^{+}}{\text{lim}}\frac{g(x)}{\eta (x)}=0\right\}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{or}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{𝒢}_0^{\text{Sh}}:=\left\{g\in {𝒢}_0|0<\underset{x\to 0^{+}}{\text{lim}}\frac{g(x)}{\eta (x)}<\infty \right\}.$$

It is easy to show that if g is subderivative, then the limit $\underset{x\to 0^{+}}{\text{lim}}g(x)/\eta (x)$ is finite. Moreover, we will see that values of dynamical g-entropies depend on the behavior of g in the neighborhood of zero. We will prove that if $g\in {𝒢}_0^0\cup {𝒢}_0^{\text{Sh}}$, then there is a linear dependence between the dynamical g-entropy and the Shannon dynamical entropy of a given partition. Before we give the general result (Theorem 1), we will state a few facts, which we will use in the proof of this theorem. We give the following lemmas, omitting their elementary proofs.

Lemma 1. Let b_i > 0, a_i ∈ ℝ for i = 1,...,m, then

$$\underset{i=1,\dots ,m}{\text{min}}\frac{a_i}{b_i}\le \frac{{\sum }_{i=1}^m{a}_i}{{\sum }_{i=1}^m{b}_i}\le \underset{i=1,\dots ,m}{\text{max}}\frac{a_i}{b_i}.$$

Lemma 2. If $𝒫$ ∈ 𝔅, δ > 0 and g :[0, 1] ↦ ℝ, then:

$$\sum _{A\in 𝒫,\mu \left(A\right)\ge \delta }g(\mu (A))\le \frac{1}{\delta }\underset{x\in \left[\delta ,1\right]}{\text{max}}g(x).$$

(3)

The following lemma states that the value of the dynamical g-entropy is given by the behavior of g in the neighborhood of zero.

Lemma 3. If g₁,g₂ ∈$𝒢$₀ and there exists c > 0, such that g₁(x) = g₂(x) for x ∈ [0, c], then for every $𝒫$∈ 𝔅 h(g₁, $𝒫$) = h(g₂, $𝒫$).

Proof. Let $𝒫$ ∈ 𝔅 and g₁, g₂ ∈$𝒢$₀, c > 0, fulfill the assumptions. Because g ∈ $𝒢$₀ is bounded, we have:

$$\begin{array}{ll}\left|H(g_1,{𝒫}_n)\right|-H(g_2,{𝒫}_n)\hfill & =\left|\sum _{A\in {𝒫}_n:\mu \left(A\right)>c}\left(g_1(\mu (A))-g_2(\mu (A))\right)\right|\hfill \\ \hfill & \le \frac{1}{c}\underset{x\in \left[c,1\right]}{\text{max}}\left|g_1(x)-g_2(x)\right|.\hfill \end{array}$$

Dividing by n and converging to infinity, we obtain:

$$h(g_1,𝒫)=h(g_2,𝒫).$$

We may state now the main theorem of this section.

Theorem 1. Let $𝒫$ ∈ 𝔅.

(1)

If g ∈ $𝒢$₀ is such that g′(0) < ∞, then h(g, $𝒫$) = 0.

(2)

If g₁,g₂ ∈ $𝒢$₀ are such that g′₁(0) = g′₂(0) = ∞,

$$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}<\infty ,$$

and h(g₂, $𝒫$) < ∞, then:

$$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}\cdot h(g_2,𝒫)\le h(g_1,𝒫).$$

If, additionally, $\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}<\infty$, then:

$$h(g_1,𝒫)\le \underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}\cdot h(g_2,𝒫).$$

(3)

If h(g₂, $𝒫$) = ∞ and$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}>0$, then h(g₁, $𝒫$) = ∞.

Remark 1. Whenever g₂ :[0, 1] ↦ ℝ is a nonnegative concave function satisfying g₂(0) = 0 and g′₂(0) = ∞, we can have any pair 0 < a ≤ b ≤ ∞ as a limit inferior and limit superior of g₁/g₂ in zero, choosing a suitable function g₁. The idea is as follows: construct g₁ piecewise linear. To do so, define inductively a strictly decreasing sequence x_k → 0 and a decreasing sequence of values y_k = g₁(x_k) → 0, thus defining intervals J_k : = [x_k+1, x_k], where g is affine. The only constraint to get a concave function is that the slope of g on each interval J_k has to be smaller than y_k/x_k and increasing with respect to k; this is not an obstruction to approach any limit inferior and limit superior for g₁(x)/g₂(x), provided that x_k+1 > 0 is chosen small enough.

Proof of Theorem 1. Let $𝒫$ ∈ 𝔅. Suppose that g ∈ $𝒢$₀ and g′(0) < ∞. Then:

$$h(g,𝒫)=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}H(g,{𝒫}_n)\le \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}\phi \left(1/\text{card}\hspace{0.17em}{𝒫}_n\right)\le \underset{x\to \infty }{\text{lim}}\frac{g^{\prime }\left(0\right)}{n}=0,$$

which completes the proof of Point 1. To show Point 2, let g₁, g₂ ∈ $𝒢$₀ be such that g′₁(0) = g′₂(0) = ∞ and h(g₂, $𝒫$) < ∞. W.l.o.g, we can assume that g₁(x), g₂(x) > 0 for x ∈ (0, 1), since if there exists x₀ ∈ (0, 1), such that g_i(x₀) = 0 for i = 1 or i = 2, then we can define g̃_i :[0, 1] ↦ ℝ as:

$${\tilde{g}}_i(x):=\{\begin{array}{ll}{g}_i(x),\hfill & \text{for}\hspace{0.17em}\hspace{0.17em}x\hspace{0.17em}\in \left[0,s_i\right)\hfill \\ g_i(s_i),\hfill & \text{for}\hspace{0.17em}\hspace{0.17em}x\hspace{0.17em}\in \left[s_i,1\right]\hfill \end{array}$$

where s_i ∈ (0, 1] is such that $\underset{x\in \left[0,1\right]}{\text{max}}g(x)=g(s_i)$. Then, g̃ is strictly positive, and by Lemma 3 we have:

$$h({\tilde{g}}_i𝒫)=h(g_i,𝒫).$$

Let us assume that:

$$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}<\infty .$$

Since g is subadditive, the sequence ${\left(H(g,{𝒫}_n)\right)}_{n=1}^{\infty }$ is nondecreasing, and there exists the limit of H(g₂, $𝒫$_n). If it is finite, then h(g₂, $𝒫$) = 0, and by Equation (3) and Lemma 1, we have:

$$\begin{array}{ll}\sum _{A\in {𝒫}_n}{g}_1(\mu (A))\hfill & \le \sum _{A\in {𝒫}_n:\mu \left(A\right)<\frac{1}{2}}{g}_1(\mu (A))+2\underset{x\in \left[\frac{1}{2},1\right]}{\text{max}}\hspace{0.17em}\hspace{0.17em}{g}_1(x)\hfill \\ \hfill & \le \underset{x\in \left(0,\frac{1}{2}\right)}{\text{sup}}\frac{g_1(x)}{g_2(x)}\cdot \sum _{A\in {𝒫}_n:\mu \left(A\right)<\frac{1}{2}}{g}_2(\mu (A))+2\underset{x\in \left[\frac{1}{2},1\right]}{\text{max}}{g}_1(x).\hfill \end{array}$$

Because $\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}<\infty$, there exists M > 0, such that g₁(x)/g₂(x) < M for x < 1/2. Therefore, $\underset{x\in \left(0,\frac{1}{2}\right)}{\text{sup}}\frac{g_1(x)}{g_2(x)}<\infty$, and by Lemma 1, we obtain:

$$\begin{array}{ll}0\le h(g_1,𝒫)\hfill & =\hspace{0.17em}\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\hspace{0.17em}\frac{1}{n}\frac{H(g_1,{𝒫}_n)}{H(g_2,{𝒫}_n)}H(g_2,{𝒫}_n)\hfill \\ \hfill & \le \underset{x\in \left(0,\frac{1}{2}\right)}{\text{sup}}\frac{g_1(x)}{g_2(x)}\cdot \underset{x\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}H(g_2,{𝒫}_n)=0.\hfill \end{array}$$

Thus, we can assume that $\underset{n\to \infty }{\text{lim}}H(g_2,{𝒫}_n)=\infty$.

Fix ε > 0. There exists δ > 0, such that, for x ∈ (0, δ], we have:

$$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}-\epsilon <\frac{g_1(x)}{g_2(x)}\le \underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}+\epsilon .$$

Lemma 1 implies that:

$$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}-\epsilon \le \frac{\sum _{A\in {𝒫}_n,\mu \left(A\right)<\delta }{g}_1(\mu (A))}{\sum _{A\in {𝒫}_n,\mu \left(A\right)<\delta }{g}_2(\mu (A))}\le \underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}+\epsilon .$$

(4)

Using Equation (3) for every n > 0, we get:

$$\sum _{A\in {𝒫}_n,\mu \left(A\right)\ge \delta }{g}_i\left(\mu \left(B\right)\right)\le \frac{1}{\delta }\overline{G_{\delta }^i}.$$

where $\overline{G_{\delta }^i}:=\underset{x\in \left[\delta ,1\right]}{\text{max}}{g}_i(x)$ for i = 1, 2. Therefore:

$$\frac{\sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_1(\mu (A))}{\sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_2(\mu (A))+\frac{1}{\delta }\overline{G_{\delta }^2}}\le \frac{\sum _{A\in {𝒫}_n}{g}_1(\mu (A))}{\sum _{A\in {𝒫}_n}{g}_2(\mu (A))}\le \frac{\sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_1(\mu (A))+\frac{1}{\delta }\overline{G_{\delta }^1}}{\sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_2(\mu (A))}.$$

and $\sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_2(\mu (A))\to \infty \left(n\to \infty \right)$. Dividing sums by $\sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_2(\mu (A))$ and from Equation (4), we obtain:

$$\begin{array}{ll}\frac{\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}-\epsilon }{1+\overline{G_{\delta }^2}/\delta \sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_2(\mu (A))}\hfill & \le \frac{\sum _{A\in {𝒫}_n}{g}_1(\mu (A))}{\sum _{A\in {𝒫}_n}{g}_2(\mu (A))}\hfill \\ \hfill & \le \underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}+\epsilon +\overline{G_{\delta }^1}/\delta \sum _{A\in {𝒫}_n:\mu \left(A\right)<\delta }{g}_2(\mu (A))\hfill \end{array}$$

Converging with n to infinity, we obtain:

$$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}-\epsilon \le \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{inf}}\frac{H(g_1,{𝒫}_n)}{H(g_2,{𝒫}_n)}\le \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{H(g_1,{𝒫}_n)}{H(g_2,{𝒫}_n)}\le \underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_2(x)}+\epsilon .$$

Therefore:

$$\begin{array}{ll}\left(\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}-\epsilon \right)h(g_2,𝒫)\hfill & \le \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{inf}}\frac{H(g_1,{𝒫}_n)}{H(g_2,{𝒫}_n)}\cdot \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}H(g_2,{𝒫}_n)\hfill \\ \hfill & \le \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}H(g_1,{𝒫}_n)\hfill \\ \hfill & \le \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{H(g_1,{𝒫}_n)}{H(g_2,{𝒫}_n)}.\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}H(g_2,{𝒫}_n)\hfill \end{array}$$

$$\le \left(\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g_1(x)}{g_1(x)}+\epsilon \right)h(g_2,𝒫).$$

Thus, we obtain the assertion. In the case of the infinite upper limit of the quotient g₁(x)/g₂(x), we can repeat the above reasoning, just omitting the upper bound for the considered expressions.

If $\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}>0$ and h(g₂, $𝒫$) = ∞, then $\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{inf}}\frac{g_1(x)}{g_2(x)}>\epsilon$, and using similar arguments, we obtain Point 3.

Similar arguments lead us to the following statement:

Theorem 2. Let g₁, g₂ ∈$𝒢$₀ be such that$\underset{x\to 0^{+}}{\text{lim}}{g}_1(x)/g_2(x)=\infty$, and let a finite partition $𝒫$ have positive g₂-entropy. Then, h(g₁, $𝒫$) is infinite.

Theorems 1 and 2 imply a few corollaries:

Corollary 1. If there exists the limit$\underset{x\to 0^{+}}{\text{lim}}\frac{g_1(x)}{g_2(x)}<\infty$, then$h(g_1,𝒫)=\underset{x\to 0^{+}}{\text{lim}}\frac{g_1(x)}{g_2(x)}\cdot h(g_2,𝒫)$

Let ${𝒢}_0^{\infty }:=\left\{g\in {𝒢}_0|\underset{x\to 0^{+}}{\text{lim}}\frac{g(x)}{\eta (x)}=\infty \right\}$. If g₁ = g, g₂ = η, then we have the following corollary:

Corollary 2. Let $𝒫$∈ 𝔅 and g ∈ $𝒢$₀, then:

(1)

If Ci(g) < ∞, then h(g, $𝒫$) ≥ Ci(g) · h($𝒫$).

(2)

If Cs(g) < ∞, then h(g, $𝒫$) ∈ (Ci(g) · h($𝒫$), Cs(g) · h($𝒫$)).

(3)

If$g\in {𝒢}_0^0\cup {𝒢}_0^{\text{Sh}}$, then h(g, $𝒫$)=C(g) · h($𝒫$).

(4)

If$g\in {𝒢}_0^{\infty }$ and h($𝒫$) > 0, then h(g, $𝒫$) = ∞.

Corollary 3. If (X, Σ, μ, T) has positive Kolmogorov-Sinai entropy and g ∈$𝒢$₀ then: Cs(g) < ∞ ⇒ g-entropy of any process (X, Σ, μ, T, $𝒫$) is finite ⇒ Ci(g) < ∞.

Corollary 4. If$g\in {𝒢}_0^0\cup {𝒢}_0^{\text{Sh}}$, then$h(g,𝒫)=\underset{n\to \infty }{\text{lim}}\frac{1}{n}H(g,{𝒫}_n)$.

2.1.2. Case of $g\in {𝒢}_0^{\infty }$

We will show that for every $g\in {𝒢}_0^{\infty }$, any aperiodic automorphism T and every γ ∈ ℝ, there exists a partition $𝒫$ ∈ 𝔅 such that h(g, $𝒫$) ≥ γ. Since we omit the assumption of ergodicity, we will use different techniques, mainly based on the well-known Rokhlin Lemma, which guarantees the existence of so-called Rokhlin towers of a given height, covering a sufficiently large part of X. Using such towers, we will find lower bounds for the g-entropy of a process.

We will assume that we have an aperiodic system, i.e., (X, Σ, μ, T) for which

$$\mu \left(\left\{x\in X:{\exists }_n\in \mathbb{N}{T}^{n}x=x\right\}\right)=0.$$

If M₀,..., M_n−1 ⊂ X are pairwise disjoint sets of equal measure, then τ = (M₀, M₁,..., M_n−1) is called a tower. If additionally M_k = T^{−(n−k−1)} M_n−1 for k = 1,..., n – 1, then τ is called a Rokhlin tower (It is also known as a Rokhlin–Halmos or Rokhlin–Kakutani tower.). By the same bold letter τ, we will denote the set ${\cup }_{k=0}^{n-1}{M}_k$. Obviously μ(τ) = nμ(M_n−1). Integer n is called the height of tower τ. Moreover, for i < j, we define a sub-tower:

$${\tau }_i^j:=(M_i,\dots ,M_j)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{and}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mathit{\tau }}_i^j=\underset{k=i}{\overset{j}{\cup }}{M}_k.$$

In aperiodic systems, there exist Rokhlin towers of a given length that cover a sufficiently large part of X:

Lemma 4 ([32]). If T is an aperiodic and surjective transformation of Lebesgue space (X, Σ, μ), then for every ε > 0 and every integer n ≥ 2, there exists a Rokhlin tower τ of height n with μ(τ) > 1 – ε.

Our goal is to find a lower bound for the dynamical g-entropy of a given partition. For this purpose, we will use Rokhlin towers, and we will calculate dynamical g-entropy with respect to a given Rokhlin tower. This leads to the following quantity: Let $𝒫$ be a finite partition of X and F ∈ Σ then, we define the (static) g-entropy of $𝒫$ restricted to F as:

$$H_F(g,𝒫):=\sum _{B\in 𝒫}g\left(\mu \left(B\cap F\right)\right).$$

The following lemma gives the estimation for H(g, $𝒫$) from below by the value of the g-entropy restricted to a subset of X.

Lemma 5. Let g ∈$𝒢$₀. Let $𝒫$ be a finite partition, such that there exists a set E ∈ $𝒫$ with 0 < μ(E) < 1. If F ∈ Σ, then:

$$H(g,𝒫)\ge H_F(g,𝒫)-3d_{\text{max}},$$

where$d_{\text{max}}:=\underset{x,y\in \left[0,1\right]}{\text{max}}\left|g(x)-g(y)\right|$.

Proof. Let A ∈ $𝒫$. Three-slope inequality implies that:

$$\frac{g(\mu (A))-g\left(\mu \left(A\cap F\right)\right)}{\mu \left(A\right)-\mu \left(A\cap F\right)}\ge \frac{g\left(1\right)-g(\mu (A))}{1-\mu \left(A\right)}.$$

Thus, for sets of measure $\mu \left(A\right)\le \frac{1}{2}$, we have:

$$\begin{array}{ll}g(\mu (A))-g\left(\mu \left(A\cap F\right)\right)\hfill & \ge -\frac{\mu \left(A\right)-\mu \left(A\cap F\right)}{1-\mu \left(A\right)}·d_{\text{max}}\hfill \\ \hfill & \ge -2\left(\mu \left(A\right)-\mu \left(A\cap F\right)\right)·d_{\text{max}}.\hfill \end{array}$$

Therefore, we obtain:

$$\sum _{A\in 𝒫:\mu \left(A\right)\le 1/2}\left(g(\mu (A))-g\left(\mu \left(A\cap F\right)\right)\right)\ge -2d_{\text{max}}\cdot \sum _{A\in 𝒫:\mu \left(A\right)\le 1/2}\mu \left(A\backslash F\right)\ge -2d_{\text{max}}$$

and:

$$\sum _{A\in 𝒫:\mu \left(A\right)>1/2}\left(g(\mu (A))-g\left(\mu \left(A\cap F\right)\right)\right)\ge -d_{\text{max}},$$

which implies that:

$$H(g,𝒫)-H_F(g,𝒫)\ge -3d_{\text{max}}.$$

The following lemma will play an important role in the proof of the main theorem of this section.

Lemma 6. Let n ∈ ℕ, E ∈ Σ. Let $𝒫$^E : = {E, X\E}. Suppose that g ∈ $𝒢$₀ is nonnegative in [0,α], where α is some positive number. Then, there exist δ > 0 and s ∈ (0,α), such that:

$$\left|H\left(g,{𝒫}_n^E\right)-H\left(g,{𝒫}_n^F\right)\right|\le 1+\frac{2}{s}{d}_{\text{max}}$$

for every F ∈ Σ s.t. μ(EΔF) < δ (Δ denotes the symmetric difference), where$d_{\text{max}}:=\underset{x,y\in \left[0,1\right]}{\text{max}}\left|g(x)-g(y)\right|$.

Proof. It is easy to show that for every n ∈ ℕ, E ∈ Σ, ε > 0, there exists δ > 0, such that:

$$\underset{\pi }{\text{min}}\sum _{A_i\in {𝒫}_n^E,B_{\pi \left(i\right)}\in {𝒫}_n^F}\mu \left(A_i\Delta B_{\pi \left(i\right)}\right)<\epsilon \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{for}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}F\in \Sigma \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{such}\hspace{0.17em}\text{that}:\mu \left(E\Delta F\right)<\delta .$$

W.l.o.g., we may assume that $\sum _{i=1}^{2^n}\mu \left(A_i\Delta B_i\right)<\epsilon$ (adding empty sets if necessary). The nonnegativity of g for x ∈ [0, α] and its concavity imply that there exists s ∈ (0, α), such that g is nondecreasing in [0, s]. Fix n ∈ ℕ and E ∈ Σ. There exists ε ∈ (0, s/2), such that:

$$g\left(\epsilon \right)<2^{-n}.$$

(5)

It is easy to see that for x ∈ [0, s], the monotonicity and subadditivity of g implies that:

$$\left|g(y)-g(x)\right|\le g\left(\left|y-x\right|\right).$$

(6)

Let F ∈ Σ be such that μ(EΔF) ≤ δ. Define ${𝒟}_s^n=\left\{i\in \left\{1,\dots ,2^n\right\}|\text{max}\left\{\mu (A_i),\mu (B_i)\right\}<s\right\}$. | max{μ(A_i), μ(B_i)} < s}. From Equations (5) and (6) and the monotonicity of g in [0, s], we obtain:

$$\begin{array}{ll}\left|H\left(g,{𝒫}_n^E\right)-H\left(g,{𝒫}_n^F\right)\right|\hfill & \le \sum _{i\in {𝒟}_s}\left|g\left(\mu (A_i)\right)-g\left(\mu (B_i)\right)\right|+\frac{2}{s}{d}_{\text{max}}\hfill \\ \hfill & \le \sum _{i\in {𝒟}_s}g\left(\left|\mu (A_i)-\mu (B_i)\right|\right)+\frac{2}{s}{d}_{\text{max}}\hfill \\ \hfill & \le \sum _{i\in {𝒟}_s}g\left(\mu \left(A_i\Delta B_i\right)\right)+\frac{2}{s}{d}_{\text{max}}\hfill \\ \hfill & \le \sum _{i=1}^{m}g\left(\mu \left(A_i\Delta B_i\right)\right)+\frac{2}{s}{d}_{\text{max}}\hfill \\ \hfill & \le 2^{n}g\left(\delta \right)+\frac{2}{s}{d}_{\text{max}}\le 1+\frac{2}{s}{d}_{\text{max}}\hfill \end{array}$$

To find the lower bound for the g-entropy of a partition, we will use so-called independent sets. We construct the independent set in the following way: Let τ be a tower of height m. We divide the highest level of this tower (M_m−1) into two sets of equal measure, let us say I^(m−1) and M_m−1\I^(m−1). Next, we consider T⁻¹I^(m−1) and T⁻¹(M_m−1\I^(m−1)). We divide each of them into two sets of equal measure, obtaining sets $I_1^{\left(m-2\right)},I_2^{\left(m-2\right)},I_3^{\left(m-2\right)},I_4^{\left(m-2\right)}$, and define set I^(m−2) as the algebraic sum of two of those sets—one subset of T⁻¹I^(m−1) and one of T⁻¹(M_m−1\I^(m−1)). We perform this algorithm, until we achieve the lowest level of the tower — M₀ (see Figure 1). Eventually, we define $I:=\underset{j=0}{\overset{m-1}{\cup }}{I}^{\left(j\right)}$. We call this set an independent set in τ.

We can make this construction, because every aperiodic system does not have atoms of positive measure, and in every non-atomic Lebesgue space for every measurable set A and every α ∈ [0, α], there exists B ⊂ A, such that μ(B) = α.

We are able to give an explicit formula for the g-entropy of the partition generated by the independent set in τ.

Lemma 7. Let τ = (M, TM,..., T²ⁿ⁻¹M) be the Rokhlin tower of height 2n and I ∈ Σ be an independent set in τ. If$g\in {𝒢}_0^{\infty }$, then:

$$H_{{\mathit{\tau }}_0^{n-1}}\left(g,{𝒫}_n^I\right)=\frac{\mu \left(\mathit{\tau }\right)}{2}\phi \left(\frac{\mu \left(\mathit{\tau }\right)}{2^{n+1}}\right),$$

where φ(x) = g(x)/x for x > 0.

Proof. The independence of I in τ implies that the partition:

$${𝒫}_n^I\cap {\mathit{\tau }}_0^{n-1}$$

is a partition of ${\mathit{\tau }}_0^{n-1}$ into 2ⁿ sets of equal measure 2⁻ⁿ$\mu \left({\mathit{\tau }}_0^{n-1}\right)$. Therefore:

$$\begin{array}{ll}{H}_{{\mathit{\tau }}_0^{n-1}}\left(g,{𝒫}_n^I\right)\hfill & =\sum _{A\in {𝒫}_n^I}g\left(\mu \left(A\cap {\mathit{\tau }}_0^{n-1}\right)\right)\hfill \\ \hfill & =2^{n}g\left(\frac{\mu \left({\mathit{\tau }}_0^{n-1}\right)}{2^n}\right)=\mu \left({\mathit{\tau }}_0^{n-1}\right)\phi \left(\frac{\mu \left({\mathit{\tau }}_0^{n-1}\right)}{2^n}\right)\hfill \\ \hfill & =\frac{\mu \left(\mathit{\tau }\right)}{2}\phi \left(\frac{\mu \left(\mathit{\tau }\right)}{2^{n+1}}\right).\hfill \end{array}$$

Theorem 3. Let$g\in {𝒢}_0^{\infty }$ and T be an aperiodic, surjective automorphism of a Lebesgue space (X, Σ, μ), and let γ ∈ ℝ, then, there exists a partition $𝒫$∈ 𝔅, such that:

$$h(g,𝒫)\ge \gamma .$$

Proof. We will prove that for any γ > 0, there exists a partition $𝒫$^E = {E, X\E}, such that h(g, $𝒫$) ≥ γ. We define recursively a sequence of sets E_n ∈ Σ. Let:

$$E_0:=\varnothing ,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{N}_0:={\delta }_0:=1.$$

Let n > 0, and assume that we have already defined E_n−1, N_n−1 and δ_n−1. Using Lemma 6, we can choose δ_n > 0, such that:

$${\delta }_n<\frac{1}{2}{\delta }_{n-1}$$

(7)

$$\left|H\left(g,{𝒫}_n^{E_{n-1}}\right)-H\left(g,{𝒫}_{N_n}^F\right)\right|<1+\frac{2}{s}{d}_{\text{max}.}$$

(8)

for any F ∈ Σ, for which μ(E_n−1 ΔF) < 2δ_n.

Since:

$$\underset{x\to 0^{+}}{\text{lim}}\frac{g(x)}{\eta (x)}=\infty ,$$

we can choose such N_n ∈ ℕ that:

$$\frac{\phi \left({\delta }_n{2}^{-N_n-1}\right)}{{\phi }_{\eta }\left({\delta }_n{2}^{-N_n-1}\right)}>\frac{2\gamma }{{\delta }_n\text{log}2}.$$

(9)

By Lemma 4, there exists M_n ∈ Σ, such that τ_n = (M_n, TM_n,...,T^2N_n−1M_n) is a Rokhlin tower of measure μ(τ_n) = δ_n. Let I_n ⊂ τ_n be an independent set in τ_n and:

$$E_n:=\left(E_{n-1}\backslash {\mathit{\tau }}_n\right)\cup I_n.$$

Then:

$$\mu \left(E_{n-1}\Delta E_n\right)\le \mu \left({\mathit{\tau }}_n\right)={\delta }_n.$$

for all positive integers n. By Equation (7), we have δ_n < 2⁻ⁿ, and we conclude that ${\left({\mathbf{1}}_{E_n}\right)}_{n=0}^{\infty }$ is a Cauchy sequence in L₁(X). Therefore, there exist E ∈ Σ, such that 1_En converges to 1_E. For this set, we have:

$$\mu \left(E_n\Delta E\right)\le \sum _{k=n+1}^{\infty }\mu \left(E_k\Delta E_{k-1}\right)\le \sum _{k=n+1}^{\infty }{\delta }_k<2{\delta }_{n+1}.$$

Since E_n ∩ τ_n = I_n, applying Equation (8) and Lemmas 5 and 7, we obtain that for N_n, such that δ_n · 2^−N_n−1 < s:

$$\begin{array}{lll}H\left(g,{𝒫}_{N_n}^E\right)\hfill & \ge \hfill & H\left(g,{𝒫}_{N_n}^{E_n}\right)-1-\frac{2}{s}{d}_{\text{max}}\hfill \\ \hfill & \ge \hfill & H_{{\left({\mathit{\tau }}_n\right)}_0^{N_n-1}}\left(g,{𝒫}_{N_n}^{E_n}\right)-\left(\frac{2}{s}+3\right)d_{\text{max}}-1\hfill \\ \hfill & \ge \hfill & H_{{\left({\mathit{\tau }}_n\right)}_0^{N_n-1}}\left(g,{𝒫}_{N_n}^{I_n}\right)-\left(\frac{2}{s}+3\right)d_{\text{max}}-1\hfill \\ \hfill & \ge \hfill & \left[\frac{\mu \left({\mathit{\tau }}_n\right)\text{ln}2}{2}\right]\left(N_n+1\right)-\frac{\mu \left({\mathit{\tau }}_n\right)\text{ln}\mu \left({\mathit{\tau }}_n\right)}{2}.\frac{\phi \left(\mu \left({\mathit{\tau }}_n\right)2^{-N_n-1}\right)}{-\text{ln}\left(\mu \left({\mathit{\tau }}_n\right)2^{-N_n-1}\right)}\hfill \\ \hfill & \hfill & -\left(\frac{2}{s}+3\right)d_{\text{max}}-1\hfill \\ \hfill & \ge \hfill & \frac{\text{ln}2}{2}\cdot {\delta }_n\cdot \left(N_n+1\right)\cdot \frac{\phi \left({\delta }_n{2}^{-N_n-1}\right)}{{\phi }_{\eta }\left({\delta }_n{2}^{-N_n-1}\right)}-\left(\frac{2}{s}+3\right)d_{\text{max}}-1.\hfill \end{array}$$

From Equation (9), we obtain that:

$$\underset{x\to \infty }{\text{lim}}\frac{H\left(g,{𝒫}_{N_n}^E\right)}{N_n}\ge \frac{\text{ln}2}{2}\underset{n\to \infty }{\text{lim}}{\delta }_n\cdot \frac{N_n+1}{N_n}\cdot \frac{\phi \left({\delta }_n{2}^{-N_n-1}\right)}{{\phi }_{\eta }\left({\delta }_n{2}^{-N_n-1}\right)}\ge \gamma .$$

Thus,

$$\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}H(g,{𝒫}_n)\ge \gamma .$$

2.1.3. Bernoulli Shifts

Let 𝒜 = {1,...,k} be a finite alphabet. Let $X=\left\{x={\left\{x_i\right\}}_{i=-\infty }^{\infty }:x_i\in 𝒜\right\}$ and σ be a left shift:

$$\sigma {(x)}_i=x_{i+1}.$$

For any s ≤ t and block [ω₀, ... ,ω_t−s] with a_i ∈𝒜, we define a cylinder:

$$C_s^t\left({\omega }_0,\dots ,{\omega }_{t-s}\right)=\left\{x\in X:x_i={\omega }_{i-s}\hspace{0.17em}\hspace{0.17em}\text{for}\hspace{0.17em}i=s,\dots ,t\right\}.$$

We consider a Borel σ-algebra with respect to the metric, which is given by d(x, y) = 2^−N, where N = min{|i| : x_i ≠ y_i}. Let p = (p₁,..., p_k) be a probability vector. We define a measure ρ = ρ(p) on 𝒜 by setting ρ({i}) = p_i. Then, μ_p is a corresponding product measure on X = 𝒜^𝕑. Thus, the static g-entropy of a partition $𝒫$^𝒜 = {[1], [2],..., [k]} is equal to:

$$H_{{\mu }_p}\left(g,{𝒫}_n^{𝒜}\right)=\sum _{\omega \in {𝒜}^n}g\left(\mu \left(C_0^{n-1}\left({\omega }_0,\dots ,{\omega }_{n-1}\right)\right)\right)=\sum _{\omega \in {𝒜}^n}g\left(p_{{\omega }_0}\cdots p_{{\omega }_{n-1}}\right),$$

where ω = (ω₀,...,ω_n−1). By the concavity of g, we have:

$$H_{{\mu }_p}\left(g,{𝒫}_n^{𝒜}\right)\le \phi \left(1/k^n\right)$$

where equality holds only when $p=p^{*}=\left(\frac{1}{k},\dots ,\frac{1}{k}\right)$. Before calculating the dynamical g-entropy of $𝒫$^𝒜 with respect to μ_p^*, we give the following lemma, the proof of which will be given later:

Lemma 8. If g ∈$𝒢$₀, then:

$$\text{Cs}(\text{g})=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{g\left({\kappa }^{-n}\right)}{\eta \left({\kappa }^{-n}\right)}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\mathit{and}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{Ci}(g)=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{inf}}\frac{g\left({\kappa }^{-n}\right)}{\eta \left({\kappa }^{-n}\right)}\hspace{0.17em}$$

for any κ > 1.

Therefore, applying Lemma 8 for the partition $𝒫$^𝒜 and κ = k, we obtain:

$$h_{{\mu }_{p^{*}}}\left(g,{𝒫}^{𝒜}\right)=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}\phi \left(\frac{1}{k^n}\right)=\{\begin{array}{ll}\text{Cs}(g)\cdot \text{log}k,\hfill & \text{if}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{Cs}(g)<\infty ;\hfill \\ \infty ,\hfill & \text{otherwise}.\hfill \end{array}$$

(10)

Remark 2. If we consider the lower limit instead of the upper limit, we would obtain:

$$\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{inf}}\frac{1}{n}\phi \left(\frac{1}{k^n}\right)=\{\begin{array}{ll}\text{Ci}(g)\cdot \text{log}k,\hfill & if\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{Ci}(g)<\infty ;\hfill \\ \infty ,\hfill & \mathit{otherwise}.\hfill \end{array}$$

Therefore, we cannot replace an upper limit by the limit in the definition of the dynamical g-entropy.

Proof of Lemma 8. We will show just the equality for the upper limit, since the equality for the lower limit may be obtained analogously. Let ${(x_n)}_{n=1}^{\infty }$ and ${\left(m_n\right)}_{n=1}^{\infty }$ be such that $\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}g(x_n)/\eta (x_n)=c$ and x_n ∈ (κ^−m_n,κ^−m_n+1) for every n ∈ ℕ. Then, –log x_n ≥ – log κ^−m_n+1. Every g ∈ $𝒢$₀ is quasi-homogenic; so, for every positive integer n occurs:

$$\frac{g(x_n)}{x_n}<\frac{g\left({\kappa }^{-m_n}\right)}{{\kappa }^{-m_n}}.$$

Therefore:

$$\begin{array}{ll}\frac{g(x_n)}{\eta (x_n)}\hfill & =\frac{g(x_n)}{x_n}\frac{1}{-\text{log}{x}_n}\le \frac{g\left({\kappa }^{-m_n}\right)}{{\kappa }^{-m_n}}\frac{1}{\left(m_n-1\right)\text{log}\kappa }\hfill \\ \hfill & =\frac{g\left({\kappa }^{-m_n}\right)}{\eta \left({\kappa }^{-m_n}\right)}\cdot \frac{m_n}{m_n-1},\hfill \end{array}$$

and:

$$\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g(x)}{\eta (x)}=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{g\left({\kappa }^{-n}\right)}{\eta \left({\kappa }^{-n}\right)}.$$

2.2. Kolmogorov-Sinai Entropy-Like Invariant

The basic tool considered in the ergodic theory is the Kolmogorov–Sinai entropy, which is a supremum of Shannon dynamical entropies over all finite partitions:

$$h_{\mu }\left(T\right)=\underset{𝒫\in 𝔅}{\text{sup}}h(T,𝒫).$$

It is invariant under metric isomorphism. Following the Kolmogorov proposition, we take the supremum over all partitions of the dynamical g-entropy of a partition. For a given system (X, Σ, μ, T), we define:

$$h_{\mu }\left(g,T\right)=\underset{𝒫\in 𝔅}{\text{sup}}h\left(g,T,𝒫\right)$$

(11)

and call it the measure-theoretic g-entropy of transformation T with respect to measure μ.

It is easy to see that it is an isomorphism invariant. Ornstein and Weiss [23] showed the striking result that measure-theoretic entropy is the only finitely observable invariant for the class of all ergodic processes. More precisely, every finitely observable invariant for a class of all ergodic processes is a continuous function of entropy. Of course, in the case of $g\in {𝒢}_0^0\cup {𝒢}_0^{\text{Sh}}$ by Corollary 2, we have:

$$h_{\mu }\left(g,T\right)=\underset{x\to 0^{+}}{\text{lim}}\frac{g(x)}{\eta (x)}\cdot h_{\mu }\left(T\right).$$

We will show that for a wider class of functions, namely for those for which:

$$\text{Cs}(g)=\underset{x\to 0^{+}}{\text{lim}\hspace{0.17em}\text{sup}}\frac{g(x)}{\eta (x)}<\infty$$

we have:

$$h_{\mu }\left(g,T\right)=\text{Cs}(g)\cdot h_{\mu }\left(T\right)$$

for any ergodic transformation T. This shows that the measure-theoretic g-entropy is, in fact, finitely observable: one might simply compose the entropy estimators [25] with the linear function itself. Our proof will be similar to the proof of ([12], Theorem 1.1) where Takens and Verbitski showed that for ergodic transformations, the supremum over all finite partitions of dynamical Rényi entropies of order α > 1 are equal to the measure-theoretic entropy of T with respect to measure μ.

Let us introduce necessary definitions and facts. Let T_i be automorphisms of Lebesgue space (X_i, Σ_i, μ_i) for i = 1, 2, respectively. Then, we say that T₂ is a factor of transformation T₁, if there exists a homomorphism ϕ: X₁ ↦ X₂, such that:

$$\varphi T_1=T_2\varphi \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mu }_1\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{a}.\text{e}.\hspace{0.17em}\hspace{0.17em}\text{on}\hspace{0.17em}\hspace{0.17em}{X}_1.$$

Suppose that T₂ is a factor of T₁ under homomorphism ϕ. Then, for an arbitrary finite partition $𝒫$ of X₂, we have:

$$H\left(g,\underset{i=0}{\overset{k-1}{\vee }}{T}_2^{-i}𝒫\right)=H\left(g,\underset{i=0}{\overset{k-1}{\vee }}{\varphi }^{-1}{T}_2^{-i}𝒫\right)=H\left(g,\underset{i=0}{\overset{k-1}{\vee }}{T}_1^{-i}{\varphi }^{-1}𝒫\right).$$

Hence, h(g, T₂, $𝒫$) = h(g, T₁, ϕ⁻¹ $𝒫$). Therefore:

$$h_{\mu }\left(g,T_2\right)=\underset{𝒫-\text{finite}}{\text{sup}}h\left(g,T_2,𝒫\right)=\underset{𝒫-\text{finite}}{\text{sup}}h\left(g,T_1,{\varphi }^{-1}𝒫\right)\le h\left(g,T_1\right).$$

This implies the following proposition:

Proposition 1. If T₂ is a factor of T₁, then for every function g ∈ $𝒢$₀:

$$h_{\mu }\left(g,T_2\right)\le h_{\mu }\left(g,T_1\right).$$

2.2.1. Measure-Theoretic g-Entropies for Bernoulli Automorphisms

An automorphism T on (X, Σ, μ) is called Bernoulli automorphism if it is isomorphic to some Bernoulli shift. The crucial role in the proof of the main theorem of this section (Theorem 5) will be played by a well-known Sinai theorem:

Theorem 4 (Sinai [33]). Let T be an arbitrary ergodic automorphism of some Lebesgue space (X, Σ, μ). Then, each Bernoulli automorphism with h_μ(T₁) ≤ h_μ(T) is a factor of the automorphism T.

We start proving the following proposition:

Proposition 2. Let T be an arbitrary ergodic automorphism with h_μ(T) ≥ log M for some integer M ≥ 2. Then, for every g ∈$𝒢$₀:

$$h_{\mu }\left(g,T\right)\ge \text{Cs}(g)\cdot \text{log}M.$$

Proof. Consider a shift σ over the alphabet 𝒜 = {0, 1 ..., M – 1} with the corresponding Bernoulli measure generated by $p_1=\dots =p_M=\frac{1}{M}$. It is easy to see that h_μ(σ) = log M. From Theorem 4, we conclude that σ is a factor of T . Therefore, applying formula (10), we obtain:

$$h_{\mu }\left(g,T\right)\ge h_{\mu }\left(g,\sigma \right)\ge h\left(g,\sigma ,{𝒫}^{𝒜}\right)=\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{n}\phi \left(M^{-n}\right)=\text{log}M\cdot \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{\phi \left(M^{-n}\right)}{{\phi }_{\eta }\left(M^{-n}\right)}.$$

Applying Lemma 8 completes the proof.

2.2.2. Main Theorem

Our goal in this section is the following result:

Theorem 5. Let T be an ergodic automorphism of Lebesgue space (X, Σ, μ), and g ∈$𝒢$₀ be such that Cs(g) ∈ (0, ∞). Then:

$$h_{\mu }\left(g,T\right)=\{\begin{array}{ll}\text{Cs}(g)\cdot h_{\mu }\left(T\right),\hfill & if\hspace{0.17em}\hspace{0.17em}{h}_{\mu }\left(T\right)<\infty ,\hfill \\ \infty ,\hfill & \mathit{otherwise}.\hfill \end{array}$$

If$g\in {𝒢}_0^0$, then h_μ(g, T) = 0. If g ∈$𝒢$₀ is such that Cs(g) = ∞ and T has a positive measure-theoretic entropy, then h_μ(g, T) = ∞.

Moreover, for $g\in {𝒢}_0^{\infty }$ from Theorem 3, we have:

Corollary 5. Let$g\in {𝒢}_0^{\infty }$. If (X, T) is aperiodic and surjective, then h_μ(g, T) = ∞.

To prove Theorem 5, we need the first few preliminary lemmas.

Lemma 9. If T is an automorphism of the Lebesgue space (X, Σ, μ), then for every g ∈$𝒢$₀:

$$h_{\mu }=\left(g,T^m\right)\le m{h}_{\mu }\left(g,T\right).$$

Proof. Let $𝒫$∈ 𝔅, m ∈ ℕ. We have:

$$\begin{array}{ll}h\left(g,T,𝒫\right)\hfill & =\underset{k\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{1}{k}H\left(g,𝒫\vee T^{-m}𝒫\vee \dots \vee T^{-m\left(k-1\right)}𝒫\right)\hfill \\ \hfill & =\underset{n\to \infty }{\text{lim}}\underset{k\ge n}{\text{sup}}\frac{1}{k}H\left(g,𝒫\vee T^{-m}𝒫\vee \dots \vee T^{-m\left(k-1\right)}𝒫\right).\hfill \end{array}$$

Fix k ∈ ℕ. Then, $\underset{i=0}{\overset{n-1}{\vee }}{T}^{-i}𝒫$ is a refinement of $𝒫$ ∨ T^−m$𝒫$ ∨ ...∨ T^−m(k−1)$𝒫$ for n = km,..., k(m+1) − 1.

Therefore:

$$\begin{array}{ll}\frac{1}{k}H\left(g,𝒫\vee T^{-m}𝒫\vee \dots \vee T^{-m\left(k-1\right)}𝒫\right)\hfill & \le \frac{1}{k}H(g,{𝒫}_{n-1})=\frac{n}{k}\frac{1}{n}H\left(g,{𝒫}_{n-1}\right)\hfill \\ \hfill & \le \frac{km+m-1}{k}\frac{1}{n}H\left(g,{𝒫}_{n-1}\right)\hfill \\ \hfill & \le m\left(1+\frac{1}{k}\right)\frac{1}{n}H\left(g,{𝒫}_{n-1}\right)\hfill \end{array}$$

(12)

for n = km,..., k(m +1) – 1. Let us introduce the following notation:

$$c_k:=\frac{1}{k}H\left(g,𝒫\vee T^{-m}𝒫\vee \dots \vee T^{-m\left(k-1\right)}𝒫\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}_n:=\frac{1}{n}H\left(g,{𝒫}_{n-1}\right).$$

Then, we can rewrite Equation (12) in the form:

$$c_k\le m\left(1+\frac{1}{k}\right)a_n$$

(13)

for n = km,..., km + m – 1. Taking the supremum in Equation (13), we obtain:

$$\underset{l\ge k}{\text{sup}}{c}_l\le m\left(1+\frac{1}{k}\right)\underset{n=lm,\dots ,l\left(m+1\right)-1}{\text{sup}}{a}_n\le m\left(1+\frac{1}{k}\right)\underset{n\ge km}{\text{sup}}{a}_n.$$

Therefore:

$$\underset{k\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}{c}_k\le m\underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}{a}_n,$$

and this is equivalent to the statement:

$$h\left(g,T^m,𝒫\right)\le mh\left(g,T,𝒫\right).$$

Taking the supremum over all finite partitions, we obtain the assertion.

Next, the lemma will be just a weaker version of Theorem 5.

Lemma 10. If an automorphism T^m of a Lebesgue space (X, Σ, μ) is ergodic for every m ∈ ℕ, then for every g ∈$𝒢$₀, such that Cs(g) < ∞ holds:

$$h_{\mu }\left(g,T\right)\le \text{Cs}(g)\cdot h_{\mu }\left(T\right).$$

If$g\in {𝒢}_0^0$, then h_μ(g, T) = 0. If g ∈$𝒢$₀ is such that Cs(g) = ∞ and T has a positive Kolmogorov-Sinai entropy, then h_μ(g, T) = ∞.

Proof. The case of $g\in {𝒢}_0^0$ follows from Corollary 2. Suppose that there exists such $g\in {𝒢}_0\backslash {𝒢}_0^0$, which fulfills the assumptions of the lemma and for which we have:

$$\text{Cs}(g)\cdot h_{\mu }\left(T\right)-h_{\mu }\left(g,T\right)>0.$$

Then, applying Lemma 9 to the transformation T^m and using equality h_μ(T^m) = mh_μ(T) (see [2], Theorem 4.3.16), we obtain:

$$\text{Cs}(g)h_{\mu }\left(T^m\right)-h_{\mu }\left(g,T^m\right)\ge m\hspace{0.17em}\left(\text{Cs}(g)h_{\mu }\left(T\right)-h_{\mu }\left(g,T\right)\right)\to \infty \hspace{0.17em}\hspace{0.17em}\text{as}\hspace{0.17em}\hspace{0.17em}m\to \infty .$$

Therefore, for sufficiently large m, there exists an integer M for which:

$$h_{\mu }\left(g,T^m\right)\le m{h}_{\mu }\left(g,T\right)<\hspace{0.17em}\text{Cs}(g)\text{log}M\le m\hspace{0.17em}\text{Cs}(g)h_{\mu }\left(T\right)=\text{Cs}(g)h_{\mu }\left(T^m\right).$$

(14)

Proposition 2 applied to the transformation T^m guarantees that for every g ∈ $𝒢$₀ with positive (finite) Cs(g), we have:

$$h_{\mu }\left(g,T^m\right)\ge \text{Cs}(g)\text{log}M.$$

(15)

Comparing Equations (14) and (15), we obtain the contradiction, which implies that

$$h_{\mu }\left(g,T\right)=\text{Cs}(g)h_{\mu }\left(T\right).$$

If Cs(g) = ∞ and h_μ(T) > 0, then there exists such an integer m > 0 that:

$$h_{\mu }\left(T^m\right)=m{h}_{\mu }\left(T\right)>\text{log}M$$

and by Proposition 2 and Lemma 9:

$$h_{\mu }\left(g,T\right)=h_{\mu }\left(g,T^m\right)=\infty$$

which completes the proof.

Proof of Theorem 5. If h_μ(T) = 0, the statement is true, because for any $𝒫$∈ 𝔅, we have:

$$0\le h(g,𝒫)\le \text{Cs}(g)h\left(𝒫\right)=0.$$

Suppose that 0 < h_μ(T) < ∞. Automorphism T is ergodic. Thus, it has a factor, which is a Bernoulli automorphism T′ with h_μ(T) = h_μ(T′). Every Bernoulli automorphism is mixing, so T^m is ergodic for each m. Applying Lemma 10, we obtain:

$$h_{\mu }\left(g,T^{\prime }\right)=\text{Cs}(g)h_{\mu }\left(T^{\prime }\right)=\text{Cs}(g)h_{\mu }\left(T\right).$$

Since T′ is a factor of T, Proposition 1 implies that:

$$\text{Cs}(g)h_{\mu }\left(T\right)=\text{Cs}(g)h_{\mu }\left(T^{\prime }\right)=h_{\mu }\left(g,T^{\prime }\right)\le h_{\mu }\left(g,T\right)\le \text{Cs}(g)h_{\mu }\left(T\right)$$

which completes the proof of the case of finite h_μ(T). If h_μ(T) = ∞, then Proposition 2 implies that:

$$h_{\mu }\left(g,T\right)\ge \text{Cs}(g)\text{log}M$$

for every M > 0.

2.2.3. Generator Theorem Counterpart

In the case of $\text{g}\in {𝒢}_0^{\infty }$, there is no counterpart of a Kolmogorov-Sinai generator theorem, which states that the measure-theoretic entropy of the transformation T is realized on every generator of the σ-algebra Σ. Let us consider Sturm shifts, shifts that model translations of the circle 𝕋 = [0, 1). Let β ∈ [0, 1) and consider the translation ϕ_β :[0, 1) ↦ [0, 1) defined by ϕ_β(x) = x + β (mod 1). Let $𝒫$ denote the partition of [0, 1) given by $𝒫$ = {[0,β), [β, 1)}. Then, we associate a binary sequence to each t ∈ [0, 1) according to its itinerary relative to $𝒫$; that is, we associate to t ∈ [0, 1) the bi-infinite sequence x defined by x_i =0 if ${\varphi }_{\beta }^i$ (t) ∈ [0,β) and x_i =1 if ${\varphi }_{\beta }^i$ (t) ∈ [β, 1). The set of such sequences is not necessary closed, but it is shift-invariant and so its closure is a shift space called the Sturmian shift. If β is irrational, then Sturmian shift is minimal, i.e., there is no proper subshift. Moreover, for a minimal Sturmian shift, the number of n-blocks that occur in an infinite shift space is exactly n +1. Therefore for zero-coordinate partition $𝒫$^𝒜, which is a finite generator of σ-algebra Σ, and for any function g ∈$𝒢$₀, we have:

$$H\left(g,{𝒫}_n^{𝒜}\right)=\sum _{A\in {𝒫}_n^{𝒜}}g\left({\mu }_S\left(A\right)\right)\le \phi \left(\frac{1}{n+1}\right)$$

where μ_S is the unique invariant measure for Sturm shift. Thus,

$$h\left(g,{𝒫}^{𝒜}\right)\le \underset{n\to \infty }{\text{lim}\hspace{0.17em}\text{sup}}\frac{n+1}{n}g\left(\frac{1}{n+1}\right)=0.$$

On the other hand, since it is strictly ergodic (thus aperiodic), Theorem 3 implies that for any $g\in {𝒢}_0^{\infty }$:

$$h_{\mu }\left(g,T\right)=\infty .$$

Therefore, we have a finite generator, for which the supremum is not attained.

3. Discussion

In this note, we discussed the generalization of the dynamical and Kolmogorov-Sinai entropy based on the idea of considering in the definition of the dynamical entropy a concave function vanishing at the origin instead of the Shannon entropy function η. The connections between dynamical entropies and g-entropies show that the crucial property of η for applications of the Shannon entropy in the dynamical systems is the behavior of η in the neighborhood of zero. Additionally, the main result of the paper obtained for the generalization of the KS entropy states that, usually, there is a linear dependence between the obtained invariant and the KS entropy. It also implies (due to the fact that it is a continuous function of the entropy) that the measure-theoretic g-entropy is finitely observable (see, e.g., [23]). Moreover, considering functions that behave in the neighborhood of zero differently than the Shannon function usually trivializes the theory. On the other hand, we showed that if the limit of g/η at zero is infinite, then for every positive number γ, there exists a partition for which the g-entropy will be greater than or equal to γ. Thus, the measure-theoretic g-entropy will be infinite. The example from Section 2.2.3, based on this result, implies that there is no counterpart of the generator theorem (e.g., Tsallis entropies for α< 1, which we obtain considering $g(x)=\frac{x=x^{\alpha }}{\alpha -1}$ with α ∈ (0, 1), fit into this scheme). However, the concept of g-entropies is still of use, e.g., considering the rate of convergence of partial g-entropies may give additional information about the system [30]. The most promising direction in this context seems to be considering functions for which C(g) = 0 and g′(0) = ∞.