Time Evolution of Relative Entropies for Anomalous Diffusion

Abstract

The entropy production paradox for anomalous diffusion processes describes a phenomenon where one-parameter families of dynamical equations, falling between the diffusion and wave equations, have entropy production rates (Shannon, Tsallis or Renyi) that increase toward the wave equation limit unexpectedly. Moreover, also surprisingly, the entropy does not order the bridging regime between diffusion and waves at all. However, it has been found that relative entropies, with an appropriately chosen reference distribution, do. Relative entropies, thus, provide a physically sensible way of setting which process is “nearer” to pure diffusion than another, placing pure wave propagation, desirably, “furthest” from pure diffusion. We examine here the time behavior of the relative entropies under the evolution dynamics of the underlying one-parameter family of dynamical equations based on space-fractional derivatives.

1. Introduction

Relative entropies, like Kullback-Leibler [1,2] or the Tsallis relative entropy [3,4,5,6], provide a means to discuss (directed) distances between different probability distributions. These measures are usually not symmetric, so they do not provide a metric on the distributions. Nonetheless, relative entropies allow the comparison of distributions occurring in a variety of different contexts [7,8,9,10,11,12,13,14]. Here, we will use these measures to analyze certain features of distributions describing the dynamics of physical systems.

In particular, our motivation is to study anomalous diffusion processes and the corresponding distribution functions of the dispersing particles. Anomalous diffusion processes differ from classical diffusion in that the dispersion of particles proceeds faster (superdiffusion) or slower (subdiffusion) than for the regular case. These anomalous diffusion processes do, for instance, occur in biological tissues [15,16] or in chemical systems [17]. They can also be observed in porous media [18,19] or turbulent diffusion [20,21]. Such processes are also important in other areas, like target search [22,23,24] or the design of optical materials in which light waves perform a Lévy flight [25].

The theoretical treatment of such processes has lead to the study of evolution equations using non-linear dependencies on the probability density functions (PDF) [26,27] or employing fractional derivatives [28,29,30,31,32,33,34,35,36].

Here, we focus on the space-fractional diffusion equation

$$\begin{array}{c}\hfill \frac{\partial }{\partial t}P(x,t)=D\frac{{\partial }^{\alpha }}{\partial x^{\alpha }}P(x,t)\end{array}$$

(1)

not as a modeling tool for an interesting class of superdiffusion processes with remarkable features [37,38,39], but as a bridge to link the usually unrelated classical diffusion equation (iconic irreversibility) to the wave equation (iconic reversibility). For that the parameter α must vary between one (the (half) wave case) and the (the diffusion case). This bridging regime has been analyzed under different perspectives [40,41,42] and has shown unexpected features. The space-fractional diffusion equation represents a family of processes in the bridge regime that can be ordered by the parameter α, which will be called the bridge ordering.

In [43], the relative entropies (e.g., Kullback-Leibler), in contrast to the regular entropies, order the PDF’s from Equation (1), because of a monotonic relationship in α, placing the wave and diffusion limits “farthest” from each other, even if not in a metrical sense. This establishes relative entropies as a natural measure for the bridging regime. However, [43] considered circumstances at one particular time. This paper extends the previous work by asking whether this ordering is preserved over all time. This question is addressed by examining direct computation and deducing asymptotic expressions valid for long times, based on a saddle point approach.

After briefly setting up the formalism of the known probability densities that solve Equation (1) and the asymptotic methods that are employed, the time dependence of the Kullback-Leibler entropy is discussed, and the cross entropy is introduced. Direct computational analysis of the relative entropy is shown that confirms that the desired monotonic behavior of the relative entropy is persistent at a number of widely ranging times. Then, the asymptotic form is computed for large time, using the saddle point method applied to the cross entropy. This yields an asymptotic form valid for the entire relative entropy. By examining the asymptotic form of the derivative with respect to α, the sign is shown to be preserved, confirming monotonicity for large times. Comparisons are made between direct computational analysis and the resulting asymptotic forms, showing good agreement.

Completing the picture, a similar treatment is performed for Tsallis relative entropies ($q\ne 1$), resulting in similar outcomes. Monotonicity is preserved at finite values and asymptotically. An interesting feature is noted, namely that for the Tsallis relative entropies, the values are bounded, which is not so for the Kullback-Leibler case.

2. Stable Distributions

The solution of the space-fractional diffusion equation can be expressed in terms of a stable distribution, $\mathbf{S}\left(\left.x\right|\alpha ,\beta ,\gamma ,{\delta }_n;n\right)$ [37,43,44], given as

$$\begin{array}{cc}\hfill P_{\alpha }(x,t)& =\mathbf{S}\left(\left.x\right|\alpha ,1,{\left(D_{\alpha }t\right)}^{1/\alpha },0;1\right)=\frac{1}{{\left(D_{\alpha }t\right)}^{1/\alpha }}\mathbf{S}\left(\left.\frac{x}{{\left(D_{\alpha }t\right)}^{1/\alpha }}\right|\alpha ,1,1,0;1\right)\hfill \end{array}$$

(2)

where $D_{\alpha }=-Dcos\left(\frac{\alpha \pi }{2}\right)$ and the parameters are chosen appropriately with

$$\begin{array}{cc}\hfill \beta & =1\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill \gamma & ={\left[-Dtcos\left(\frac{\alpha \pi }{2}\right)\right]}^{1/\alpha }={\left(D_{\alpha }t\right)}^{1/\alpha }\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\delta }_1& =0\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill n& =1\hfill \end{array}$$

(6)

An important feature of stable distributions is their so-called fat tails, i.e., the probability density in the tails falls off with a power law rather than exponentially, thus leading to the non-existence of higher moments. Fat tails are responsible for the non-existence of the Kullback-Leibler entropy for certain combinations of stable distributions.

The following presents properties of stable distributions needed for the calculations below. The stable distribution rescales as

$$\begin{array}{c}\hfill \mathbf{S}\left(\left.x\right|\alpha ,\beta ,\gamma ,{\delta }_n;n\right)=\frac{1}{\gamma }\mathbf{S}\left(\left.\frac{x-{\delta }_n}{\gamma }\right|\alpha ,\beta ,1,0;n\right)\end{array}$$

(7)

For the parameters discussed here, the support of the stable distributions is the full real line. Apart from special cases, there are no general closed representations of the stable distribution in terms of elementary functions. However, the density at a certain point x can be expressed as an integral

$$\begin{array}{c}\hfill \mathbf{S}\left(\left.x\right|\alpha ,\beta ,1,0;1\right)=\frac{1}{\pi }{\int }_0^{\infty }{e}^{-u^{\alpha }}cos(xu+\zeta u^{\alpha })\mathrm{d}u\end{array}$$

(8)

where $\zeta =-\beta tan\left(\frac{\alpha \pi }{2}\right)$. For $x=0$, the integral can be evaluated in closed form, leading to

$$\begin{array}{c}\hfill \mathbf{S}\left(\left.0\right|\alpha ,\beta ,1,0;1\right)=\frac{1}{\alpha \pi }\Gamma \left(\frac{1}{\alpha }\right)cos\left({\theta }_0\right){\left[cos\left(\alpha {\theta }_0\right)\right]}^{\frac{1}{\alpha }}\end{array}$$

(9)

where ${\theta }_0=-arctan\left(\zeta \right)/\alpha$ and $\Gamma \left(x\right)$ is the Gamma function. Similarly, there are expressions for the derivatives of the density based on Equation (8). The $2k$-th and the $(2k+1)$-th derivatives with respect to x are

$$\begin{array}{cc}\hfill {\mathbf{S}}^{\left(2k\right)}(\left.x\right|\alpha ,\beta ,1,0;1)& =\frac{{(-1)}^k}{\pi }{\int }_0^{\infty }{u}^{2k}{e}^{-u^{\alpha }}cos(xu+\zeta u^{\alpha })\mathrm{d}u\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\mathbf{S}}^{(2k+1)}(\left.x\right|\alpha ,\beta ,1,0;1)& =\frac{{(-1)}^{(k+1)}}{\pi }{\int }_0^{\infty }{u}^{2k+1}{e}^{-u^{\alpha }}sin(xu+\zeta u^{\alpha })\mathrm{d}u\hfill \end{array}$$

(11)

We will need the first two derivatives at $x=0$, which can again be expressed in closed form as

$$\begin{array}{cc}\hfill {\mathbf{S}}^{\left(1\right)}(\left.0\right|\alpha ,\beta ,1,0;1)& =-\frac{1}{\pi }\Gamma \left(\frac{3}{\alpha }\right){\left[cos\left(\alpha {\theta }_0\right)\right]}^{\frac{2}{\alpha }}sin\left(2{\theta }_0\right)\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\mathbf{S}}^{\left(2\right)}(\left.0\right|\alpha ,\beta ,1,0;1)& =-\frac{1}{\pi }\Gamma \left(\frac{4}{\alpha }\right){\left[cos\left(\alpha {\theta }_0\right)\right]}^{\frac{3}{\alpha }}cos\left(3{\theta }_0\right)\hfill \end{array}$$

(13)

For $\beta =1$ and angles in $(-\pi /2,\pi /2)$

$$\begin{array}{cc}\hfill {\theta }_0& =-\frac{1}{\alpha }arctan\left(tan\frac{(2-\alpha )\pi }{2}\right)=\frac{\pi }{2}-\frac{\pi }{\alpha }\hfill \end{array}$$

(14)

The abbreviations

$$\begin{array}{cc}\hfill {\mathbf{S}}_{\alpha }^{\left(n\right)}\left(x\right)& ={\mathbf{S}}^{\left(n\right)}(\left.x\right|\alpha ,1,1,0;1)\hfill \\ \mathrm{(15)}& \hfill {\Gamma }_n& =\frac{\Gamma \left(\frac{n}{\alpha }\right)}{\Gamma \left(\frac{1}{\alpha }\right)}\hfill \hfill {\Psi }_n& =\Psi \left(\frac{n}{\alpha }\right)=\frac{{\Gamma }^{\prime }\left(\frac{n}{\alpha }\right)}{\Gamma \left(\frac{n}{\alpha }\right)}\hfill \end{array}$$

will be used in the following. Ψ is the digamma function. Thus, Equations (12) and (13) become

$$\begin{array}{cc}\hfill {\mathbf{S}}_{\alpha }^{\left(1\right)}\left(0\right)& =-\alpha {\Gamma }_{3}sin\left(\frac{2\pi }{\alpha }\right){\left[-cos\left(\frac{\alpha \pi }{2}\right)\right]}^{1/\alpha }{\mathbf{S}}_{\alpha }\left(0\right)\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\mathbf{S}}_{\alpha }^{\left(2\right)}\left(0\right)& =-\alpha {\Gamma }_{4}sin\left(\frac{3\pi }{\alpha }\right){\left[-cos\left(\frac{\alpha \pi }{2}\right)\right]}^{1/\alpha }{\mathbf{S}}_{\alpha }\left(0\right)\hfill \end{array}$$

(17)

For $\alpha =2$, Equation (1) reduces to the classical diffusion equation. That is reflected in its solution $P_{\mathrm{D}}(x,t)$, which is the well-known Gaussian. That solution can also be expressed in terms of a stable distribution for $\alpha =2$ as

$$\begin{array}{cc}\hfill P_{\mathrm{D}}(x,t)& =\mathbf{S}\left(\left.x\right|2,1,\sqrt{Dt},0;1\right)=\mathcal{N}(0,2Dt)\hfill \end{array}$$

(18)

$$\begin{array}{cc}& =\frac{1}{2\sqrt{\pi Dt}}exp\left(-\frac{x^2}{4Dt}\right)\hfill \end{array}$$

(19)

where $\mathcal{N}(\mu ,{\sigma }^2)$ is the normal distribution with mean $\mu =0$ and variance ${\sigma }^2=2Dt$ [44].

In the limit $\alpha \to 1$, the scale parameter, $\gamma ={\left[-Dtcos\left(\frac{\alpha \pi }{2}\right)\right]}^{1/\alpha }$, of the stable distribution goes to zero and the mode (i.e., maximum of the distribution) ${\widehat{x}}_{\alpha }\to -Dt$. This constitutes a time-moving δ-distribution centered at $-Dt$, representing the one-sided solution of the wave equation with an initial δ-distribution.

Writing Equation (1) in the $\alpha =1$ limit

$$\begin{array}{c}\hfill \left(\frac{\partial }{\partial t}-D\frac{\partial }{\partial x}\right)P(x,t)=0\end{array}$$

(20)

When the operator $\left(\frac{\partial }{\partial t}+D\frac{\partial }{\partial x}\right)$ is applied to Equation (20), the standard wave equation is recovered, ensuring that solutions of Equation (20) are also solutions of the full wave equation. Equation (20) thus reflects one of two operator factors for the wave equation, and it is known, accordingly, as the half wave equation. The classical advection equation also reduces to the half wave equation in the one-dimensional solenoidal case.

3. Relative Entropies

Kullback-Leibler entropy (“Kullback” for short) between two probability density functions, $P_a(x,t)$ and $P_b(x,t)$, is defined in this context as

$$\begin{array}{c}\hfill K(P_a(x,t),P_b(x,t))={\int }_{-\infty }^{\infty }{P}_a(x,t)ln\left(\frac{P_a(x,t)}{P_b(x,t)}\right)\mathrm{d}x\end{array}$$

(21)

which is the $q=1$ limit of the Tsallis relative entropy

$$\begin{array}{cc}\hfill T_q(P_a(x,t),P_b(x,t))& =\frac{1}{q-1}\left({\int }_{-\infty }^{\infty }{P}_a^q(x,t)P_b^{1-q}(x,t)\mathrm{d}x-1\right)\hfill \end{array}$$

(22)

These can both be regarded as mean values of appropriately chosen functions. $K(P_a,P_b)$ can be regarded as the mean of $ln\left(P_a/P_b\right)$, while $T_q(P_a,P_b)$ can be seen as the shifted mean of ${(P_a/P_b)}^{q-1}$. In the following, we shall see that in the large time limit, the density function $P_a$ with respect to which these averages are taken, will become sharply peaked, making the saddle point method a natural way [45] to get to asymptotic expressions for long times.

4. Saddle Point Asymptotic Expansion

In general, the saddle point approach [46] can be used to provide asymptotic expansions of integrals of the form:

$$\begin{array}{c}\hfill I\left(A\right)={\int }_{y_1}^{y_2}f\left(y\right)e^{Ag\left(y\right)}\mathrm{d}y\end{array}$$

(23)

where $f\left(y\right)$ and $g\left(y\right)$ are real functions, and we consider large positive values of A. If $g\left(y\right)$ is unimodal and has its maximum at $y_0$, then with increasing A, the function $e^{Ag\left(y\right)}$ will become more and more peaked, so that only values of $f\left(y\right)$ close to $y_0$ will contribute significantly to the integral. This induces an asymptotic series expansion in powers of $1/\sqrt{A}$. Setting $z=\sqrt{A}(y-y_0)$,

$$\begin{array}{cc}\hfill I\left(A\right)& =\frac{f\left(y_0\right)e^{Ag\left(y_0\right)}}{\sqrt{A}}{\int }_{-\infty }^{\infty }{e}^{z^2{g}^{\prime \prime }\left(y_0\right)/2}\left(1+\sum _{n=1}^{\infty }{A}^{-n/2}{P}_n\left(z\right)\right)\mathrm{d}z\hfill \end{array}$$

(24)

$$\begin{array}{cc}& =\frac{f\left(y_0\right)e^{Ag\left(y_0\right)}}{\sqrt{A}}\sqrt{\frac{2\pi }{-A{g}^{\prime \prime }\left(y_0\right)}}\left(1+\sum _{n=1}^{\infty }\frac{C_{2n}}{A^n}\right)\hfill \end{array}$$

(25)

Here, $P_n\left(z\right)$ are polynomials in z, and $C_{2n}$ contain derivatives of the functions $f\left(y\right)$ and $g\left(y\right)$, evaluated at $y_0$. In particular, one finds

$$\begin{array}{cc}\hfill C_2& =-\frac{f_2}{4f_0{g}_2}+\frac{f_1{g}_3}{8f_0{g}_2^2}+\frac{g_4}{32g_2^2}-\frac{5g_3^2}{192g_2^3},\hfill \end{array}$$

(26)

where $f_i=\frac{{\partial }^i}{\partial y^i}f\left(y\right){|}_{y=y_0}$ and $g_i=\frac{{\partial }^i}{\partial y^i}g\left(y\right){|}_{y=y_0}$. While higher orders can be determined, we here will use the expansion only up to order $1/A$.

5. Kullback-Leibler Entropy Time Dependence by Direct Computational Analysis

In [43], on which this paper builds, it was shown that the Kullback and Tsallis relative entropy might serve to order solutions of the space-fractional diffusion Equation (1) at a particular time. Here, we show that this property extends over time. One complication dealt with previously is that $K(P_{\alpha },P_{\mathrm{D}})$ does not exist, due to the fat tails of $P_{\alpha }$. Thus, the focus was on $K(P_{\mathrm{D}},P_{\alpha })$, which does exist.

The Kullback is given by

$$\begin{array}{cc}\hfill K(P_{\mathrm{D}},P_{\alpha })& ={\int }_{-\infty }^{\infty }{P}_{\mathrm{D}}(x,t)ln\left(\frac{P_{\mathrm{D}}(x,t)}{P_{\alpha }(x,t)}\right)\mathrm{d}x\hfill \\ & ={\int }_{-\infty }^{\infty }{P}_{\mathrm{D}}(x,t)ln{P}_{\mathrm{D}}(x,t)\mathrm{d}x-{\int }_{-\infty }^{\infty }{P}_{\mathrm{D}}(x,t)ln{P}_{\alpha }(x,t)\mathrm{d}x\hfill \\ \mathrm{(27)}& & =-1/2(1+ln\left(4\pi Dt\right))+K^{†}(P_{\mathrm{D}},P_{\alpha })\hfill \end{array}$$

where $K^{†}(P_{\mathrm{D}},P_{\alpha })$ is known as the cross entropy.

Figure 1. The Kullback-Leibler entropy, $K(P_{\mathrm{D}},P_{\alpha })$, is plotted over α for different times t. One can see that for all times, $K(P_{\mathrm{D}},P_{\alpha })$ exhibits a monotonic decreasing behavior, thus confirming the bridge ordering property of $K(P_{\mathrm{D}},P_{\alpha })$.

Figure 1. The Kullback-Leibler entropy, $K(P_{\mathrm{D}},P_{\alpha })$, is plotted over α for different times t. One can see that for all times, $K(P_{\mathrm{D}},P_{\alpha })$ exhibits a monotonic decreasing behavior, thus confirming the bridge ordering property of $K(P_{\mathrm{D}},P_{\alpha })$.

The previous analysis of $K(P_{\mathrm{D}},P_{\alpha })$ at $t=1$ showed that the Kullback was compatible with the bridge ordering. However, here in Figure 1, we show $K(P_{\mathrm{D}},P_{\alpha })$ as a function of α for a wide range of different times ($t={10}^0,{10}^2,{10}^4,{10}^6$). The label DCA stands for “Direct Computational Analysis.” For the analysis, $K(P_{\mathrm{D}},P_{\alpha })$ is obtained by a numerical scheme that makes use of the fact that the tail behavior of the stable distributions is known analytically. Here and in all other figures, we set $D=1$. As for $\alpha \to 1$, $P_{\alpha }$ approaches a δ-distribution. Thus, our direct computations were restricted in practice to $\alpha \in [1.01,2]$.

In Figure 1, for all cases, $K(P_{\mathrm{D}},P_{\alpha })$ falls off quickly for α close to one, but more slowly near $\alpha =2$. The monotonic relation between $K(P_{\mathrm{D}},P_{\alpha })$ and α confirms that the bridge ordering at $t=1$ is maintained across a wide interval of times. Note here that the graph of $K(P_{\mathrm{D}},P_{\alpha })$ for $t=1$ crosses those at later times. Thus, while the graph shows curves monotonic in α for fixed t, they are not generally monotonic in t for fixed α.

6. Kullback-Leibler Entropy for Long Times

Though ordering is preserved for a wide range of times, that does not mean that the ordering is preserved for long times. Thus the long-time behavior of $K(P_{\mathrm{D}},P_{\alpha })$ is deduced in the following.

The cross entropy from Equation (27) becomes

$$\begin{array}{cc}\hfill K^{†}(P_{\mathrm{D}},P_{\alpha })& =-{\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{x^2}{2{\sigma }^2}}ln\mathbf{S}\left(\left.x\right|\alpha ,1,\gamma ,0;1\right)\mathrm{d}x\hfill \\ \mathrm{(28)}& & =ln\gamma -{\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{x^2}{2{\sigma }^2}}ln{\mathbf{S}}_{\alpha }\left(\frac{x}{\gamma }\right)\mathrm{d}x\hfill \end{array}$$

where we have made use of Equation (7) and the normalization property of the Gaussian. Using the substitution $y=x/\gamma$ we then get

$$\begin{array}{cc}\hfill K^{†}(P_{\mathrm{D}},P_{\alpha })& =ln\gamma -\frac{\gamma }{\sqrt{2\pi }\sigma }{\int }_{-\infty }^{\infty }{e}^{-\frac{y^2{\gamma }^2}{2{\sigma }^2}}ln{\mathbf{S}}_{\alpha }\left(y\right)\mathrm{d}y\hfill \end{array}$$

(29)

Note that due to $\gamma ={\left(D_{\alpha }t\right)}^{(1/\alpha )}$, $\sigma =\sqrt{2Dt}$

$$\begin{array}{cc}\hfill \frac{\sigma }{\gamma }& =\frac{\sqrt{2Dt}}{{\left(D_{\alpha }t\right)}^{(1/\alpha )}}=\frac{\sqrt{2D}}{D_{\alpha }^{(1/\alpha )}}{t}^{1/2-1/\alpha }\hfill \end{array}$$

(30)

is an α-dependent quantity, where the exponent of t is always negative for $1<\alpha <2$. It follows that for $t\to \infty$, the inverse ratio $\frac{\gamma }{\sigma }$ will diverge. For large t, it will be large too, and thus, we can now make use of the saddle point method.

Using the results of Section 4 together with the following definitions

$$\begin{array}{cc}\hfill A& ={\left(\frac{\gamma }{\sigma }\right)}^2\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill f\left(y\right)& =ln{\mathbf{S}}_{\alpha }\left(y\right)\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill g\left(y\right)& =-\frac{y^2}{2}\hfill \end{array}$$

(33)

and noting that the maximum of $g\left(y\right)$ is at $y_0=0$, we obtain an asymptotic expansion for the cross entropy

$$\begin{array}{cc}\hfill K^{†}(P_{\mathrm{D}},P_{\alpha })& =ln\gamma -\frac{ln{\mathbf{S}}_{\alpha }\left(0\right)}{\sqrt{2\pi }\frac{\sigma }{\gamma }}\sqrt{\frac{2\pi }{{\left(\frac{\gamma }{\sigma }\right)}^2}}\left(1+\sum _{n=1}^{\infty }\frac{C_{2n}}{A^n}\right)\hfill \end{array}$$

(34)

$$\begin{array}{cc}& =ln\gamma -ln{\mathbf{S}}_{\alpha }\left(0\right)\left[1+\frac{C_2}{A}+O\left(\frac{1}{A^2}\right)\right]\hfill \end{array}$$

(35)

Combining Equations (27) and (35), we conclude

$$\begin{array}{cc}\hfill K(P_{\mathrm{D}},P_{\alpha })=& -\frac{1}{2}\left[1+ln\left(4\pi Dt\right)\right]+\frac{ln\left(D_{\alpha }t\right)}{\alpha }-ln{\mathbf{S}}_{\alpha }\left(0\right)\left[1+\frac{C_2}{A}+O\left(\frac{1}{A^2}\right)\right]\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill \sim & K_1+K_2\left(t\right)+\left(\frac{1}{\alpha }-\frac{1}{2}\right)lnt\hfill \end{array}$$

(37)

where $K_1$ is a constant in time, and for $1<\alpha <2$, the factor multiplying $lnt$ is positive. $K_2\left(t\right)$ decays with time, and thus, for large times, $K(P_{\mathrm{D}},P_{\alpha })$ will diverge.

The constant $K_1$ and the time-dependent $K_2\left(t\right)$ can be evaluated further making use of the properties of the stable distribution. We note that $C_2$ simplifies considerably as ${\mathrm{d}}^{\nu }g\left(y\right)/\mathrm{d}y{}^{\nu }=0$ for $\nu >2$. Using

$$\begin{array}{cc}\hfill C_2& =\frac{1}{4{\mathbf{S}}_{\alpha }\left(0\right)}\left[\frac{{\mathbf{S}}_{\alpha }^{\left(2\right)}\left(0\right)}{{\mathbf{S}}_{\alpha }\left(0\right)}-{\left(\frac{{\mathbf{S}}_{\alpha }^{\left(1\right)}\left(0\right)}{{\mathbf{S}}_{\alpha }\left(0\right)}\right)}^2\right]\hfill \end{array}$$

(38)

we find

$$\begin{array}{cc}\hfill K_1& =\frac{1}{\alpha }lnD-\frac{1}{2}ln\left(4\pi D\right)-ln\left(\frac{{\Gamma }_1}{\alpha \pi }sin\frac{\pi }{\alpha }\right)-\frac{1}{2}\hfill \end{array}$$

(39)

and

$$\begin{array}{cc}\hfill K_2\left(t\right)& =\frac{{\left(Dt\right)}^{1-\frac{2}{\alpha }}}{2sin\left(\frac{\pi }{\alpha }\right)}\left[(\alpha {\Gamma }_4+{\alpha }^2{{\Gamma }_3}^2)sin\left(\frac{3\pi }{\alpha }\right)+{\alpha }^2{{\Gamma }_3}^{2}sin\left(\frac{\pi }{\alpha }\right)\right]\hfill \end{array}$$

(40)

Figure 2. A comparison between the direct numerical calculation of the Kullback-Leibler entropy, $K(P_{\mathrm{D}},P_{\alpha })$ (DCA), the saddle point method of the zeroth order (SP0) and the saddle point method of first order (SP1) is shown over logarithmic time t for two different values of α. One observes that the approximations approach the DCA data points for large times and fit the data quite well already for $t>1$.

Figure 2. A comparison between the direct numerical calculation of the Kullback-Leibler entropy, $K(P_{\mathrm{D}},P_{\alpha })$ (DCA), the saddle point method of the zeroth order (SP0) and the saddle point method of first order (SP1) is shown over logarithmic time t for two different values of α. One observes that the approximations approach the DCA data points for large times and fit the data quite well already for $t>1$.

In Figure 2, a comparison between the numerical evaluation of $K(P_{\mathrm{D}},P_{\alpha })$ (DCA) and its approximations SP0 and SP1 based on the saddle point method is displayed. Here, SP0 is the approximation excluding terms of $O\left(\frac{1}{A}\right)$, while SP1 excludes terms of $O\left(\frac{1}{A^2}\right)$. One sees easily how the approximations approach the data points obtained by direct numerical calculation for large times. Especially for $\alpha =1.7$, that the quality increases by using the higher order SP1 is obvious. In that case, SP1 covers nearly all times $t>1$. This is not so for $\alpha =1.3$, where one sees a sizable deviation for $t<10$. However, all of the asymptotic forms produce good agreement with DCA values for $t>{10}^6$.

While the long time behavior is thus understood, the short time behavior shows a surprising feature. For short times, $K(P_{\mathrm{D}},P_{\alpha })$ shows an initial decay until a minimum is reached. Only then does the Kullback start to grow and approach its long time behavior.

In order to understand this effect, the sequence of graphs in Figure 3 shows how the distribution $P_{\mathrm{D}}$ and $ln{P}_{\alpha }$ change in time. The plots are for $t={10}^0,{10}^2,{10}^4,{10}^6$, when $\alpha =1.3$. We see that the Gaussian $P_{\mathrm{D}}$ acts like a window that suppresses $ln{P}_{\alpha }$, where $P_{\mathrm{D}}$ is exponentially small. For small times, $ln{P}_{\alpha }$ varies considerably in that window, and thus, it cannot be approximated reasonably by its value and its first two derivatives at the peak position of the Gaussian. This changes as the time increases. $ln{P}_{\alpha }$ becomes flatter and flatter, and for $t={10}^6$, the approximation works very well.

Figure 3. Four plots of $P_{\mathrm{D}}$ and $ln{P}_{\alpha }$ over x are given for $t=1$ (a), $t={10}^2$ (b), $t={10}^4$ (c) and $t_1={10}^6$ (d) for $\alpha =1.3$. It can be seen that within the width of $P_{\mathrm{D}}$, the distribution $ln{P}_{\alpha }$ becomes flatter with increasing time. Thus, $ln{P}_{\alpha }$ can be approximated well by its function value and its first two derivatives.

Figure 3. Four plots of $P_{\mathrm{D}}$ and $ln{P}_{\alpha }$ over x are given for $t=1$ (a), $t={10}^2$ (b), $t={10}^4$ (c) and $t_1={10}^6$ (d) for $\alpha =1.3$. It can be seen that within the width of $P_{\mathrm{D}}$, the distribution $ln{P}_{\alpha }$ becomes flatter with increasing time. Thus, $ln{P}_{\alpha }$ can be approximated well by its function value and its first two derivatives.

Based on the asymptotic time behavior of $K(P_{\mathrm{D}},P_{\alpha })$, we can now see how $K(P_{\mathrm{D}},P_{\alpha })$ changes with α at a given time t. We do so by taking the derivative of the asymptotic form of $K(P_{\mathrm{D}},P_{\alpha })$ with respect to α

$$\begin{array}{cc}\hfill \frac{{\partial }^{}}{\partial {\alpha }^{}}K(P_{\mathrm{D}},P_{\alpha })\left(t\right)& \sim \frac{{\partial }^{}}{\partial {\alpha }^{}}{K}_1+\frac{{\partial }^{}}{\partial {\alpha }^{}}{K}_2\left(t\right)-\frac{1}{{\alpha }^2}lnt\hfill \end{array}$$

(41)

For the different terms in Equation (41), we get

$$\begin{array}{cc}\hfill \frac{{\partial }^{}}{\partial {\alpha }^{}}{K}_1=& \frac{1}{\alpha }+\frac{1}{{\alpha }^2}\left({\Psi }_1+\pi cot\left(\frac{\pi }{\alpha }\right)-lnD\right)\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill \frac{{\partial }^{}}{\partial {\alpha }^{}}{K}_2\left(t\right)=& \frac{{\left(Dt\right)}^{(1-\frac{2}{\alpha })}}{2}\left(\frac{F_1}{\alpha }+\frac{F_2}{{\alpha }^2}\right)+\frac{2K_2\left(t\right)}{{\alpha }^2}lnt\hfill \end{array}$$

(43)

where

$$\begin{array}{cc}\hfill F_1\equiv & {\Gamma }_{4}sin\left(\frac{3\pi }{\alpha }\right)+4{\Gamma }_3^{2}cos\left(\frac{\pi }{\alpha }\right)sin\left(\frac{2\pi }{\alpha }\right)\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill F_2\equiv & {\Gamma }_{4}sin\left(\frac{3\pi }{\alpha }\right)\left[{\Psi }_1-4{\Psi }_4+\pi cot\left(\frac{\pi }{\alpha }\right)+3\pi cot\left(\frac{3\pi }{\alpha }\right)+lnD\right]\hfill \\ \mathrm{(45)}& & +2{\Gamma }_3^{2}sin\left(\frac{2\pi }{\alpha }\right)cos\left(\frac{\pi }{\alpha }\right)\left(2{\Psi }_1-6{\Psi }_3+\pi cot\left(\frac{\pi }{\alpha }\right)+lnD\right)\hfill & +2{\Gamma }_3^{2}cos\left(\frac{\pi }{\alpha }\right)\left[\pi -3\pi cos\left(\frac{2\pi }{\alpha }\right)\right]\hfill \end{array}$$

Inserting Equations (42)–(45) into Equation (41) and collecting all time-dependent terms leads to

$$\begin{array}{cc}\hfill \frac{{\partial }^{}}{\partial {\alpha }^{}}K(P_{\mathrm{D}},P_{\alpha })\left(t\right)& =\frac{2K_2\left(t\right)-1}{{\alpha }^2}lnt\hfill \\ \mathrm{(46)}& & +\left(\frac{F_1}{\alpha }+\frac{F_2}{{\alpha }^2}\right)\frac{{\left(Dt\right)}^{(1-\frac{2}{\alpha })}}{2}+\frac{1}{\alpha }+\frac{{\Psi }_1+\pi cot\left(\frac{\pi }{\alpha }\right)-lnD}{{\alpha }^2}\hfill \end{array}$$

For large times, this simplifies to:

$$\begin{array}{c}\hfill \frac{{\partial }^{}}{\partial {\alpha }^{}}K(P_{\mathrm{D}},P_{\alpha })\left(t\right)\approx \frac{2K_2\left(t\right)-1}{{\alpha }^2}lnt\approx -\frac{lnt}{{\alpha }^2}<0(t\to \infty )\end{array}$$

(47)

From Equation (40), we see that $|K_2\left(t\right)|\to 0$ in the long-time limit. Thus, the Kullback is a monotonic decreasing function of α in the large time limit, ensuring that it retains its ordering property in α for large times. This result anchors the ordering property at infinity, already established in the preceding sections for finite times.

7. The Tsallis Relative Entropy Time Dependence by Direct Computational Analysis

The Tsallis relative entropy provides another means to establish an ordering between $P_{\mathrm{D}}$ and $P_{\alpha }$. Contrary to the Kullback case, both $T_q(P_{\alpha },P_{\mathrm{D}})$ and $T_q(P_{\mathrm{D}},P_{\alpha })$ exist. In [43], we showed that these two Tsallis relative entropies are defined for $0\le q<1$. Based on this insight, one finds an interesting relation between the two

$$\begin{array}{cc}\hfill T_q(P_{\alpha },P_{\mathrm{D}})& =\frac{1}{q-1}\left({\int }_{-\infty }^{\infty }{P}_{\alpha }^q(x,t)P_{\mathrm{D}}^{1-q}(x,t)\mathrm{d}x-1\right)\hfill \end{array}$$

(48)

and

$$\begin{array}{cc}\hfill T_{1-q}(P_{\mathrm{D}},P_{\alpha })& =\frac{1}{(1-q)-1}\left({\int }_{-\infty }^{\infty }{P}_{\mathrm{D}}^{1-q}(x,t)P_{\alpha }^q(x,t)\mathrm{d}x-1\right)\hfill \end{array}$$

(49)

from which we obtain

$$\begin{array}{cc}\hfill (-q)T_{1-q}(P_{\mathrm{D}},P_{\alpha })& =(q-1)T_q(P_{\alpha },P_{\mathrm{D}})\hfill \end{array}$$

(50)

It thus suffices to analyze the time behavior of one of the two. Here, we choose $T_q(P_{\mathrm{D}},P_{\alpha })$ with

$$\begin{array}{cc}\hfill T_q(P_{\mathrm{D}},P_{\alpha })& =\frac{1}{q-1}\left[{\int }_{-\infty }^{\infty }{\left(\frac{e^{-\frac{x^2}{2{\sigma }^2}}}{\sqrt{2\pi }\sigma }\right)}^q\mathbf{S}{\left(\left.x\right|\alpha ,1,\gamma ,0;1\right)}^{1-q}\mathrm{d}x-1\right]\hfill \\ \mathrm{(51)}& & =\frac{1}{q-1}\left[{\int }_{-\infty }^{\infty }\frac{{\gamma }^{q-1}}{{\left(\sqrt{2\pi }\sigma \right)}^q}\frac{exp(-\frac{q{x}^2}{2{\sigma }^2})}{{\mathbf{S}}_{\alpha }{\left(\frac{x}{\gamma }\right)}^{q-1}}\mathrm{d}x-1\right]\hfill \end{array}$$

$T_q(P_{\mathrm{D}},P_{\alpha })$ was determined for four different times $(t={10}^0,{10}^2,{10}^4,{10}^6)$ by direct computation analysis. As in the Kullback case, the data were obtained by a numerical integration procedure, which for the Tsallis case also requires a precise treatment of the known fat tail behavior of the stable distributions. Figure 4 depicts the results for $q=0.5$. One can see that $T_q(P_{\alpha },P_{\mathrm{D}})$ decays monotonically with α, thus showing that it provides an ordering compatible with the bridge ordering for the times presented. This figure also seems to indicate that $T_q(P_{\alpha },P_{\mathrm{D}})$ increases monotonically with time for fixed α. That, however, is not true. For short times, the DCA data in Figure 6 shows that $T_{0.5}(P_{\mathrm{D}},P_{\alpha })$ does not increase monotonically in time. Instead, it decreases first and then, around $t=1$, starts to increase.

Figure 4. For the case of $q=0.5$, the Tsallis relative entropy, $T_{0.5}(P_{\mathrm{D}},P_{\alpha })$, is given over α for different times $(t={10}^0,{10}^2,{10}^4,{10}^6)$. One can observe that with increasing time, the monotonic decreasing behavior is preserved, and thus, the bridge ordering property of $T_q(P_{\mathrm{D}},P_{\alpha })$ is confirmed.

Figure 4. For the case of $q=0.5$, the Tsallis relative entropy, $T_{0.5}(P_{\mathrm{D}},P_{\alpha })$, is given over α for different times $(t={10}^0,{10}^2,{10}^4,{10}^6)$. One can observe that with increasing time, the monotonic decreasing behavior is preserved, and thus, the bridge ordering property of $T_q(P_{\mathrm{D}},P_{\alpha })$ is confirmed.

8. Tsallis Relative Entropy for Long Times

As in the Kullback case, we want to analyze the time dependence further to show that the bridge ordering is preserved at least for large enough times. The method used is again a saddle point approximation. Using the substitution $y=x/\gamma$, we get

$$\begin{array}{cc}\hfill T_q(P_{\mathrm{D}},P_{\alpha })& =\frac{1}{q-1}\left[\frac{{\gamma }^q}{{\left(\sqrt{2\pi }\sigma \right)}^q}{\int }_{-\infty }^{\infty }{e}^{-\frac{q{y}^2{\gamma }^2}{2{\sigma }^2}}{\mathbf{S}}_{\alpha }{\left(y\right)}^{1-q}\mathrm{d}y-1\right]\hfill \end{array}$$

(52)

The ratio $\frac{\gamma }{\sigma }$ is the same as in the Kullback section and thus, diverges for large times. Based on Equation (25), we make the following definitions

$$\begin{array}{cc}\hfill A& ={\left(\frac{\gamma }{\sigma }\right)}^2\hfill \end{array}$$

(53)

$$\begin{array}{cc}\hfill f\left(y\right)& ={\mathbf{S}}_{\alpha }{\left(y\right)}^{1-q}\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill g\left(y\right)& =-q\frac{y^2}{2}\hfill \end{array}$$

(55)

Again, the maximum of $g\left(y\right)$ is at $y_0=0$, and by utilizing Equation (25), we find up to $O(1/A^2)$

$$\begin{array}{cc}\hfill T_q(P_{\mathrm{D}},P_{\alpha })& =\frac{1}{q-1}\frac{1}{\sqrt{q}}{\left(\frac{\sigma }{\gamma }\sqrt{2\pi }{\mathbf{S}}_{\alpha }\left(0\right)\right)}^{1-q}\left[1+\frac{C_2}{A}+O\left(\frac{1}{A^2}\right)\right]-\frac{1}{q-1}\hfill \end{array}$$

(56)

Here, $C_2$ depends on q and takes the form

$$\begin{array}{cc}\hfill C_{2,q}& =\frac{1-q}{4q}\left[\frac{{\mathbf{S}}_{\alpha }^{\left(2\right)}\left(0\right)}{{\mathbf{S}}_{\alpha }\left(0\right)}-q{\left(\frac{{\mathbf{S}}_{\alpha }^{\left(1\right)}\left(0\right)}{{\mathbf{S}}_{\alpha }\left(0\right)}\right)}^2\right]\hfill \end{array}$$

(57)

Collecting terms, we find the asymptotic time behavior for the Tsallis relative entropy for large t

$$\begin{array}{cc}\hfill T_q(P_{\mathrm{D}},P_{\alpha })\sim & \frac{{\left[\sqrt{2\pi }{\mathbf{S}}_{\alpha }\left(0\right)\right]}^{1-q}}{\sqrt{q}(q-1)}\left[{\left(\frac{\sigma }{\gamma }\right)}^{1-q}+C_{2,q}{\left(\frac{\sigma }{\gamma }\right)}^{3-q}\right]-\frac{1}{q-1}\hfill \end{array}$$

(58)

Unlike the Kullback case, this asymptotic form has a finite limit for $t\to \infty$. The limit is $1/(1-q)$, because ${lim}_{t\to \infty }\sigma /\gamma =0$, and both exponents are positive, due to $0\le q<1$. This can be seen in the unapproximated form, too

$$\begin{array}{cc}\hfill T_q(P_{\mathrm{D}},P_{\alpha })& =\frac{1}{q-1}\left[\frac{{\gamma }^q}{{\left(\sqrt{2\pi }\sigma \right)}^q}{\int }_{-\infty }^{\infty }{e}^{-\frac{q{y}^2{\gamma }^2}{2{\sigma }^2}}{\mathbf{S}}_{\alpha }{\left(y\right)}^{1-q}\mathrm{d}y-1\right]\hfill \\ \mathrm{(59)}& \hfill =\frac{1}{q-1}& \left[{\left(2\pi \right)}^{\frac{1-q}{2}}{\left(\frac{\sigma }{\gamma }\right)}^{1-q}{\int }_{-\infty }^{\infty }\left\{\frac{1}{\sqrt{2\pi {\Sigma }^2}}{e}^{-\frac{y^2}{2{\Sigma }^2}}\right\}{\mathbf{S}}_{\alpha }{\left(y\right)}^{1-q}\mathrm{d}y-1\right]\hfill \end{array}$$

where $\Sigma =\frac{1}{\sqrt{q}}\left(\frac{\sigma }{\gamma }\right)$. We see in the $t\to \infty$ limit that the Gaussian becomes a δ-function. The δ-function picks the value of ${\mathbf{S}}_{\alpha }{\left(y\right)}^{1-q}$ at $y=0$, making the integral finite. The prefactor is decaying towards zero, and thus, the approach of $T_q(P_{\mathrm{D}},P_{\alpha })$ to the limiting value $1/(1-q)$ is confirmed.

Figure 5. A comparison of direct numerical calculation of the Tsallis relative entropy, $T_q(P_{\mathrm{D}},P_{1.3})$ (DCA), and the saddle point method of the first order (SP1) is given over logarithmic time t for different values of q.

Figure 5. A comparison of direct numerical calculation of the Tsallis relative entropy, $T_q(P_{\mathrm{D}},P_{1.3})$ (DCA), and the saddle point method of the first order (SP1) is given over logarithmic time t for different values of q.

Figure 6. A comparison of direct numerical calculation of the Tsallis relative entropy, $T_{0.5}(P_{\mathrm{D}},P_{\alpha })$ (DCA), and the saddle point method of the first order (SP1) is given over logarithmic time t for different values of α. Note that $T_q(P_{\mathrm{D}},P_{\alpha })$ is not monotonic in time for larger values of α, but has a clear minimum around $t=1$.

Figure 6. A comparison of direct numerical calculation of the Tsallis relative entropy, $T_{0.5}(P_{\mathrm{D}},P_{\alpha })$ (DCA), and the saddle point method of the first order (SP1) is given over logarithmic time t for different values of α. Note that $T_q(P_{\mathrm{D}},P_{\alpha })$ is not monotonic in time for larger values of α, but has a clear minimum around $t=1$.

The quality of the saddle point approximation is shown in Figure 5 and Figure 6. We compare the saddle point approximations SP1 of $T_q(P_{\mathrm{D}},P_{\alpha })$ with the numerically obtained values (DCA) as a function of time. The approach of the approximation to the numerical data depends highly on the choice of parameters α and q. For small values of α and q, the approximation fits the numerical data relatively fast, as shown in Figure 5. On the other hand, Figure 6 shows that the approximations work less well for larger α.

It can also be seen in Figure 5 and Figure 6 that for times $t<1$, the asymptotic approach exhibits different curve shapes depending on the choice of α and q. This can be understood by looking on Equation (58) after applying Equations (9)–(13) and Equation (57), which leads to

$$\begin{array}{cc}\hfill T_q(P_{\mathrm{D}},P_{\alpha })\sim & \frac{{\left[\frac{2\sqrt{\pi }}{{(-cos\left(\frac{\alpha \pi }{2}\right))}^{\frac{1}{\alpha }}}{\mathbf{S}}_{\alpha }\left(0\right)\right]}^{1-q}}{\sqrt{q}(q-1)}{\left(Dt\right)}^{\left(\frac{1}{2}-\frac{1}{\alpha }\right)(1-q)}\left[1+H_{2,q}·{\left(Dt\right)}^{\left(1-\frac{2}{\alpha }\right)}\right]-\frac{1}{q-1}\hfill \end{array}$$

(60)

with

$$\begin{array}{cc}\hfill H_{2,q}=& \frac{2\alpha (q-1)}{4q}\left[\frac{{\Gamma }_{4}sin\left(\frac{3\pi }{\alpha }\right)}{{(-cos\left(\frac{\alpha \pi }{2}\right))}^{\frac{1}{\alpha }}}+q\alpha {{\Gamma }_3}^2{sin}^2\left(\frac{2\pi }{\alpha }\right)\right]\hfill \end{array}$$

(61)

The first factor of Equation (60) is always positive, but because of $sin\left(\frac{3\pi }{\alpha }\right)$, $H_{2,q}$ can change sign. This depends on the parameters α and q, and is insignificant for large t.

Finally, the asymptotic form shows that $T_q(P_{\mathrm{D}},P_{\alpha })$ decreases monotonically with α for large times because

$$\begin{array}{cc}\hfill \frac{\partial }{\partial \alpha }{T}_q& \sim \left[G_{1,q}+G_{2,q}ln\left(t\right)\right]t^{(\frac{1}{2}-\frac{1}{\alpha })(1-q)}+\left[G_{3,q}+G_{4,q}ln\left(t\right)\right]t^{(\frac{1}{2}-\frac{1}{\alpha })(3-q)}\hfill \end{array}$$

(62)

where the $G_{i,q}$ are constants in t. On $1<\alpha <2$ and $0<q<1$, the inequalities

$$\begin{array}{cc}\hfill \left(\frac{1}{2}-\frac{1}{\alpha }\right)(3-q)& <\left(\frac{1}{2}-\frac{1}{\alpha }\right)(1-q)<0\hfill \end{array}$$

(63)

hold. Thus, $\frac{\partial }{\partial \alpha }{T}_q$ will be dominated by the $G_{2,q}$ term in Equation (62) for long times, as it decays most slowly. Since

$$\begin{array}{cc}\hfill G_{2,q}& =-\frac{{\left(\sqrt{2\pi }{\mathbf{S}}_{\alpha }\left(0\right)\right)}^{1-q}}{\sqrt{q}{\alpha }^2}{\left(\frac{\sqrt{2D}}{D_{\alpha }^{\frac{1}{\alpha }}}\right)}^{1-q}<0\hfill \end{array}$$

(64)

then

$$\begin{array}{c}\hfill \frac{\partial }{\partial \alpha }{T}_q(P_{\mathrm{D}},P_{\alpha })\sim G_{2,q}ln\left(t\right)t^{(\frac{1}{2}-\frac{1}{\alpha })(1-q)}<0\end{array}$$

(65)

for large enough times. Thus, like the Kullback case, the Tsallis relative entropy remains monotonically decreasing in the infinite time limit, confirming its ordering property at both finite times and infinity. However, while signs remain fixed, the Tsallis case differs from the Kullback case in that the α-slope goes to zero in the long-time limit, while that slope diverges in the Kullback case. Moreover, the Kullback also diverges with t, while $T_q(P_{\mathrm{D}},P_{\alpha })$ approaches a constant, $1/(1-q)$, for large t.

9. Summary

This paper is a sequel to [43], in which relative entropies were introduced to address the physical issue of irreversibility in the mathematical context of a one-parameter bridging regime between the diffusion and wave equations. This peculiar regime represents a definitive family of processes between diffusion and waves representing a direct formal mathematical test of our understanding of the differences between the reversible and irreversible. Not only does this regime exhibit paradoxical entropy production behavior (i.e., entropy production rates increase toward the wave limit), but entropy has a maximum in the interval, which has the consequence that one cannot determine which system is relatively “closer or further” from the pure diffusion limit by such means. However, the preceding paper showed that relative entropies could do what neither entropy nor entropy production rates could do: provide an intuitively sensible ordering, where diffusion stands at a maximum among all other bridge processes which become well ordered, due to a monotonically decreasing relationship with the parameter α.

The goal of this paper was to extend the treatment in [43] by addressing the issue of time. This paper asked whether the well-defined ordering produced by relative entropies was just a feature of a single time, or if the monotonic structure persist over all time. The relevant formalism was briefly set up in two parts. First, the necessary features of stable distributions, which are solutions to the dynamics in Equation (1), were presented. Then, the appropriate asymptotic representations of integrals in terms of the saddle point method were put into place. From this foundation, the two cases of the Kullback-Leibler and Tsallis relative entropies were explored for long times. Direct numerical computation showed that both types of relative entropies preserved the α-ordering for a wide range of time-scales. Then, asymptotic methods were compared to these direct computations, confirming that they agree well with each other for long times. It was then shown that the ordering did persist in the long-time limit in both cases. However, the Tsallis and Kullback-Leibler cases differ in that the former reached a finite limit, $1/(1-q)$, and its α-derivative vanished in the limit of long times, while the latter diverged in both relative entropy and its α-derivative.

Time evolution of relative entropies under a single process, particularly the Kullback-Leibler entropy, is well known to be connected to H-theorems. Thus, it would not be surprising if the time evolution of probability densities might make one think, at least initially, in terms of H-theorem questions; so, it is worth noting parenthetically that this is not an H-theorem scenario. The time evolution in this context is between pairs of processes belonging to the bridging families and not to a single process. There is no reason to expect that pairs of densities would relax to each other when subject to different processes. However, it brings back the issue of different internal quicknesses [40,41,42], although now articulated from the standpoint of relative entropies. Moreover, this paper focuses on the ordering property, which ultimately is about the differences between the densities. Nonetheless, we could consider nearby processes with infinitesimal differences in families and ask whether an H-theorem-like result might hold then, but that is a topic for future work.