Coarse Graining Shannon and von Neumann Entropies

Abstract

The nature of coarse graining is intuitively “obvious”, but it is rather difficult to find explicit and calculable models of the coarse graining process (and the resulting entropy flow) discussed in the literature. What we would like to have at hand is some explicit and calculable process that takes an arbitrary system, with specified initial entropy S, and that monotonically and controllably drives the entropy to its maximum value. This does not have to be a physical process, in fact for some purposes it is better to deal with a gedanken-process, since then it is more obvious how the “hidden information” is hiding in the fine-grain correlations that one is simply agreeing not to look at. We shall present several simple mathematically well-defined and easy to work with conceptual models for coarse graining. We shall consider both the classical Shannon and quantum von Neumann entropies, including models based on quantum decoherence, and analyse the entropy flow in some detail. When coarse graining the quantum von Neumann entropy, we find it extremely useful to introduce an adaptation of Hawking’s super-scattering matrix. These explicit models that we shall construct allow us to quantify and keep clear track of the entropy that appears when coarse graining the system and the information that can be hidden in unobserved correlations (while not the focus of the current article, in the long run, these considerations are of interest when addressing the black hole information puzzle).

1. Introduction

In this article, we will aim to clarify some issues related to the “coarse graining” process that is typically asserted to be associated with the monotone increase of entropy implied by the second law of thermodynamics. While many sources discuss the “coarse graining” process in a somewhat intuitive manner, it is extremely difficult to find explicit, controllable and fully-calculable models; see, for instance, [1,2,3,4,5,6,7,8] (a more general background, on coarse graining in various renormalization group settings, can be found in [9,10,11,12,13,14,15,16,17]). What we would be seeking in this current article is some process (either purely mathematical or possibly physical) that takes an arbitrary system with specified initial entropy S and that monotonically drives the entropy to its maximum value; that is, $lnN$, for any finite classical or quantum system described by the Shannon or von Neumann entropies [18,19,20]. Whenever possible, it would be desirable to introduce an adjustable smearing parameter to continuously interpolate between the original system and the smeared system.

For some purposes, it is better to deal with a gedanken-process, since a gedanken-process is in principle reversible, and then, it is more obvious how the “hidden information” is hiding in the correlations. The techniques that work best for classical Shannon entropy often do not carry over to quantum von Neumann entropy, or vice versa. We shall develop a number of different and complementary models for coarse graining, adapted to either classical or quantum behaviour, and (classically) to the continuum versus discretium character of the system. (The discretium is often alternatively referred to as the discretum; the choice is a matter of taste. Discretum is more Latinate; discretium is more Anglophone.)

The relevance of having explicit, controllable and calculable methods available to describe coarse graining emerges when we deal with physical issues of information; specifically, the “hidden information” lurking in the correlations rendered unobservable by the coarse graining. The current study provides some tools to treat these processes, which we plan to adapt to understand what is happening with the quantum information in black hole formation and evaporation, where the classical gravitational collapse that initially forms the black hole is an extremely dramatic coarse graining process; one that encodes ideas about a possible resolution of the black hole information puzzle [21,22,23,24,25,26,27,28,29].

2. Structure of the Article

We shall first present a brief historical survey of the coarse graining problem in Section 3 and then discuss coarse graining in the context of classical Shannon entropy in Section 4. For Shannon entropy in the continuum, a coarse graining in terms of an idealized and generalized diffusion process (possibly a gedanken-process, rather than a real physical diffusion) is the easiest to implement. More generally, for Shannon entropy, the distinction between continuum and discretium is important, and any coarse graining that maps the continuum into the discretium is particularly delicate. We then turn to the quantum von Neumann entropy in Section 5, elucidating several coarse graining processes based on maximal mixing, on partial traces and on decoherence. We introduce Hawking’s super-scattering operator in Section 6, with an aim to using it in Section 7 where we define a wide class of diffusion-like processes on Hilbert space (rather than in physical space). Finally, we wrap up with a discussion in Section 8.

3. History

Coarse graining plays a crucial role in our description of the world. It allows us to work with very large or complex systems by choosing a more reasonable and tractable number of variables (and consequently, reducing the observable degrees of freedom of the system under consideration). See, for instance, [1,2,3,4,5,6,7,8]. Coarse graining should be chosen so as to still describe the key properties we are most interested in, “integrating over” properties of perceived lesser importance. These coarse graining processes are used in very different fields (from biology to physics to astronomy), but for the current article, we are mostly interested applications to quantum information theory (though in the future, we plan to consider quantum fields in curved spacetimes).

The price of coarse graining is that some properties or physical effects, corresponding to the degrees of freedom that we are not looking at, will be “hidden” in the system. When calculating the entropy of these coarse-grained systems, there is an associated increase of system entropy due to uncertainty regarding the coarse-grained degrees of freedom. We shall see that the difference of entropy after imposing coarse graining can be understood as “hidden information”:

$$I_{\mathrm{hidden}}=S_{\mathrm{coarse}-\mathrm{grained}}-S_{\mathrm{before}-\mathrm{coarse}-\mathrm{graining}}.$$

(1)

Because coarse graining can sometimes be viewed as a gedanken-process, an agreement to simply “not look at” certain features of the system, the coarse-grained entropy is unavoidably contextual and may be observer dependent.

Classically, the way to describe entropy associated with information is provided by the Shannon entropy, which first introduced the interpretation of entropy as a measure of the lack of information [18,19]. In the language of communications engineers:

$$S=-\sum _{i=1}^N{p}_i{log}_2{p}_i,$$

(2)

where $p_i$ is the probability associated with each microstate and the entropy is measured in “shannons”/“bits”. Physicists prefer to use the natural logarithm, ln, and measure the entropy in “natural units” (nats). There are several ways to define information, though in general, the information can be considered as the reduction of uncertainty. The maximum information, can be defined as the difference between the maximum possible entropy and the value we measure:

$$I_{\max }=S_{\max }-S=lnN-S.$$

(3)

This is often called negentropy [18,19], and it is a measure of distance to complete uncertainty, $p_i=\frac{1}{N}$. In the context of quantum resource theory, this can be interpreted as the unique measure of information that is stable under a subset of “noisy operations” [30]. In terms of our hidden information due to coarse graining:

$$I_{\mathrm{hidden}}\in [0,I_{\max }].$$

(4)

In the realm of quantum mechanics, the previous picture changes because the information is stored in a different way, and the states can be entangled. The operator that represents the statistics of the physical states can be represented as a density matrix. The analogous entropy now refers to the entanglement entropy between different subsystems that were coarse-grained, and it is formulated by the von Neumann entropy [20]:

$$S=-\mathrm{tr}(\rho ln\rho ).$$

(5)

Here, $\rho$ is the density matrix, and $\mathrm{tr}$ denotes the trace. This von Neumann entropy has to be conserved in unitary quantum evolution. However, there exists the possibility (or necessity in some cases) that we decide to “look” only at one part of the system and forget about the rest. This process is different when quantum properties have to be taken into account and will often correspond to “trace” over a specified subsystem. In this quantum case, it can also be seen that coarse graining corresponds to an increment of the entropy completely related with the “hidden” quantum information.

Other interesting questions emerge when the transition from quantum to classical systems is taken into account. Despite valiant efforts, this process, the emergence of classicality and its interpretation are still not completely well-understood. One of the strong possibilities comes from the idea of decoherence [31,32,33,34,35,36,37], that is based on the separation of relevant and irrelevant degrees of freedom of the system, and removing the interference terms (off-diagonal terms of the density matrix) that precisely comes from the interaction of the emergent classical system (relevant degrees of freedom) with the environment (irrelevant degree of freedom). It is easy to see that this process arises from a specific process for coarse graining the system.

Therefore, in this approximation, the classical realm corresponds to a set of coarse-grained non-interfering histories (called decoherent histories approach to quantum mechanics) [2,37]. It is important to note that there also exist other interesting ideas about classical emergence, that also involve some notion of coarse graining (see, for instance, [3,4].)

In both cases, classical and quantum, it is important to understand that the “hidden information” corresponds to correlations that we decide not to look at with our particular coarse graining method. This implies that the measured entropy depends on the choice of the particular coarse graining developed; entropy is contextual and can be observer dependent. For this reason, it is interesting for the physical interpretation to have some knowledge of “where” the hidden information is hidden, “what” we hide, and how we hide it, in order to have complete control of our system. The importance of understanding these processes can be understood for instance in the study of some open questions related with information (as for instance, the black hole information puzzle [21,22,23,24,25,26,27,28,29], where it is not completely well understood where the information flows during the Hawking evaporation process). In previous literature, coarse graining has been widely considered (for instance, in [1,2,3,4,5,6,7,8] and [38,39,40,41,42,43,44,45,46]), but in most of those cases, coarse graining is assumed ab initio, preventing one from having a complete control of the system. In this paper, we shall study different quantifiable models of coarse graining the entropy, and the selection of a suitable tunable smearing parameter that will allow us to manipulate coarse graining in a controllable manner and understand what is hidden and what is correlated.

4. Coarse Graining the Classical Shannon Entropy

For a classical system with a finite, or at worst denumerable, number of states (a discretium of states), the Shannon entropy can be defined as:

$$S=-\sum p_{i}ln{p}_i;\sum p_i=1.$$

(6)

There are many abstract theorems that can be proven for this quantity, but which will not be the main focus here [47,48,49,50]. In contrast, for a non-denumerable number of states (a continuum of states; say three-space for definiteness), it is better to write:

$$S(\rho ;{\rho }_{*})=-\int \rho \left(x\right)ln\left\{\frac{\rho \left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x;\int \rho \left(x\right)d^{3}x=1.$$

(7)

Here, ${\rho }_{*}$ is some arbitrary-but-fixed parameter that is needed for dimensional consistency; it is just a convenient normalizing constant that effectively sets the zero for S (in particular, continuum entropy differences are always independent of ${\rho }_{*}$). Note that under re-scaling, we have the trivial result:

$$S(\rho ;\lambda {\rho }_{*})=S(\rho ;{\rho }_{*})+ln\lambda .$$

(8)

Below, we exhibit three forms of classical coarse graining suitable for three situations:

• Continuum ⟶ continuum (independent of ${\rho }_{*}$);
• Continuum ⟶ discretium; (not manifestly independent of ${\rho }_{*}$; care and discretion required);
• Discretium ⟶ discretium (trivially independent of ${\rho }_{*}$).

4.1. Continuum ⟶ Continuum: Diffusion-Based Forms of Coarse Graining

Starting from the probability density $\rho \left(x\right)$, coarse-grain using some sort of convolution process:

$$\rho \left(x\right)\to {\rho }_t\left(x\right)=\int K(x,y;t)\rho \left(y\right)d^{3}y;\int K(x,y;t)d^{3}y=1;\int \dot{K}(x,y;t)d^{3}y=0.$$

(9)

Here, $K(x,y;t)$ is some positive normalized kernel; t is not necessarily physical time; it is simply a parameter that controls how far you smear out the probability distribution. This kernel can be viewed as representing a “defocussing” process. A delta-function initially concentrated at $x_0$, that is $\delta (x-x_0)$, becomes “smeared out” to a time-dependent and space-dependent “blob” $K(x,x_0,t)$. Now, from the basic definition of Shannon entropy (in the continuum), we have:

$$S_t=-\int {\rho }_t\left(x\right)ln\left\{\frac{{\rho }_t\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x;{\dot{S}}_t=-\int {\dot{\rho }}_t\left(x\right)ln\left\{\frac{{\rho }_t\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x.$$

(10)

Even before specializing the kernel K, we see:

$${\dot{S}}_t=-\int \int \dot{K}(x,y;t)\rho \left(y\right)ln\left\{\frac{{\rho }_t\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}y{d}^{3}x.$$

(11)

Here, thanks to the normalizing condition, the RHS is actually independent of ${\rho }_{*}$.

Now, let us be more specific: let K be an ordinary diffusion operator (that is, formally we can write $K=exp\left\{t\sigma {\nabla }^2\right\}$), where $\sigma$ is the diffusion constant. Then:

$$\dot{K}=\sigma {\nabla }^{2}K;\mathrm{whence}{\dot{\rho }}_t=\sigma {\nabla }^2{\rho }_t.$$

(12)

Therefore, in this ordinary diffusion situation,

$${\dot{S}}_t=-\sigma \int \left({\nabla }^2{\rho }_t\left(x\right)\right)ln\left\{\frac{{\rho }_t\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x.$$

(13)

Integration by parts (noting that ${\rho }_{*}$ drops out of the calculation) now yields:

$${\dot{S}}_t=\sigma \int \frac{\nabla {\rho }_t\left(x\right)·\nabla {\rho }_t\left(x\right)}{{\rho }_t\left(x\right)}{d}^{3}x\ge 0.$$

(14)

Therefore, under diffusion-like-smoothing, entropy always rises as the smearing parameter t increases. We can easily generalize this to position-dependent, direction-dependent (and even density-dependent) diffusivity. Suppose the kernel (and so the probability density) satisfies:

$$\dot{K}={\nabla }_i\left({\sigma }^{ij}(x,{\rho }_t\left(x\right)){\nabla }_{j}K\right);{\dot{\rho }}_t={\nabla }_i\left({\sigma }^{ij}(x,{\rho }_t\left(x\right)){\nabla }_j{\rho }_t\right).$$

(15)

This now describes a generalized diffusion process, with ${\sigma }^{ij}(x,\rho \left(x\right))$ being a position-dependent, direction-dependent (and even density-dependent), diffusion “constant”. Then, under the same logic as previously:

$${\dot{S}}_t=\int {\sigma }^{ij}(x,{\rho }_t\left(x\right))\frac{{\nabla }_i{\rho }_t\left(x\right)·{\nabla }_j{\rho }_t\left(x\right)}{{\rho }_t\left(x\right)}{d}^{3}x\ge 0,$$

(16)

with non-negativity holding as long as the matrix ${\sigma }^{ij}(x,{\rho }_t\left(x\right))$ is positive semidefinite. For either simple or generalized diffusion, we can then define:

$$I_{\mathrm{correlations}}=S_t-S_0\ge 0.$$

(17)

Here, t is not necessarily physical time; it is simply the smearing parameter. If we view the diffusion process as a gedanken-process, a simple de-focussing of our vision described by the smearing parameter t, then the effect is clearly reversible, and $I_{\mathrm{correlations}}$ is simply the information hiding in the correlations that we can no longer see due to our deliberately blurred vision; but this is information we can easily regain simply by refocussing our vision. Overall, this construction yields a nice explicitly controllable entropy flow as a function of the coarse graining parameter t.

4.2. Continuum ⟶ Discretium: Box-Averaging Forms of Coarse Graining

Starting from the continuum, let us now divide the universe up into a denumerable set of non-overlapping boxes $B_i$, which nevertheless cover the entire universe. Then, define individual probabilities $p_i$ for each box,

$$p_i={\int }_{B_i}\rho \left(x\right)d^{3}x,$$

(18)

so that you are throwing away (agreeing not to look at) detailed information of the probability distribution inside each individual box. Now, compare the continuum and discretium entropies:

$$S(\rho ,{\rho }_{*})=-\int \rho \left(x\right)ln\left\{\frac{\rho \left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x;\mathrm{and}{S}_B=-\sum p_{i}ln{p}_i.$$

(19)

We shall consider two cases; a “simple” all-or-nothing box averaging and a more subtle box averaging where one first diffuses the probability density in the continuum before implementing box-averaging.

4.2.1. Simple Box-Averaging

For “simple” box-averaging, where we fix individual boxes $B_i$, but do not have any tunable parameter to work with, we shall show that (unfortunately, the ${\rho }_{*}$ dependence is unavoidable):

$$S_B\ge S(\rho ,{\rho }_{*})+(\mathrm{some}{\rho }_{*}-\mathrm{dependent}\mathrm{normalization}\mathrm{to}\mathrm{set}\mathrm{the}\mathrm{zero}\mathrm{of}\mathrm{entropy}).$$

(20)

To prove this, use the idea of (classical) relative entropy (a measure of distinguishability between two entropies). It is a standard result that if $\rho \left(x\right)$ and ${\rho }_0\left(x\right)$ are any two normalized probability distributions, then:

$$\int \rho \left(x\right)ln\left\{\frac{\rho \left(x\right)}{{\rho }_0\left(x\right)}\right\}{d}^{3}x\ge 0.$$

(21)

Let us now define a “box-wise-constant” probability density ${\rho }_B\left(x\right)$ as follows:

$${\rho }_B\left(x\right):\mathrm{if}x\in \mathrm{int}\left(B_i\right)\mathrm{then}{\rho }_B\left(x\right)=\frac{p_i}{\mathrm{volume}\left(B_i\right)}=\frac{{\int }_{B_i}\rho \left(x\right)d^{3}x}{{\int }_{B_i}{d}^{3}x}.$$

(22)

(We do not care what happens on the box boundaries, simply because they are of measure zero.)

Then, ${\rho }_B\left(x\right)$ is certainly a normalized probability distribution, $\int {\rho }_B\left(x\right)d^{3}x=1$. Consequently:

$$\int \rho \left(x\right)ln\left\{\frac{\rho \left(x\right)}{{\rho }_B\left(x\right)}\right\}{d}^{3}x\ge 0.$$

(23)

Whence:

$$\int \rho \left(x\right)ln\left\{\frac{\rho \left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x-\int \rho \left(x\right)ln\left\{\frac{{\rho }_B\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x\ge 0.$$

(24)

This further implies:

$$-\int \rho \left(x\right)ln\left\{\frac{{\rho }_B\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x\ge -\int \rho \left(x\right)ln\left\{\frac{\rho \left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x.$$

(25)

However, since ${\rho }_B\left(x\right)$ is piecewise constant (in fact, box-wise constant) on the LHS, we then have:

$$-\int \rho \left(x\right)ln\left\{\frac{{\rho }_B\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x=-\int {\rho }_B\left(x\right)ln\left\{\frac{{\rho }_B\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x=S\left({\rho }_B\right),$$

(26)

and we see that $S\left({\rho }_B\right)\ge S\left(\rho \right)$. However, we can actually do quite a bit more, since working in a slightly different direction, we have:

$$\begin{array}{ccc}\hfill S\left({\rho }_B\right)& =& -\int {\rho }_B\left(x\right)ln\left\{\frac{{\rho }_B\left(x\right)}{{\rho }_{*}}\right\}{d}^{3}x\hfill \\ & =& -\sum p_{i}ln\left(\frac{p_i}{{\rho }_{*}\mathrm{volume}\left(B_i\right)}\right)\hfill \\ & =& -\sum _p{p}_{i}ln{p}_i+\sum p_{i}ln\left[{\rho }_{*}\mathrm{volume}\left(B_i\right)\right]\hfill \\ & =& S_B+\sum p_{i}ln\left[{\rho }_{*}\mathrm{volume}\left(B_i\right)\right].\hfill \end{array}$$

(27)

Now, using the normalization ${\sum }_i{p}_i=1$, we have:

$$\sum _i{p}_{i}ln\left[{\rho }_{*}\mathrm{volume}\left(B_i\right)\right]=ln\left[{\rho }_{*}\prod _i\mathrm{volume}{\left(B_i\right)}^{p_i}\right].$$

(28)

However, note that we can interpret:

$$\prod _i\mathrm{volume}{\left(B_i\right)}^{p_i}=\overline{V},$$

(29)

where $\overline{V}$ is just the (probability weighted) geometric mean volume of the boxes $B_i$. That is, we have:

$$S\left({\rho }_B\right)=S_B+ln\left[{\rho }_{*}\overline{V}\right]\ge S(\rho ,{\rho }_{*}).$$

(30)

Rearranging, we have the rigorous inequality:

$$S_B=S({\rho }_B,{\rho }_{*})-ln\left[{\rho }_{*}\overline{V}\right]\ge S(\rho ,{\rho }_{*})-ln\left[{\rho }_{*}\overline{V}\right].$$

(31)

The RHS is actually independent of ${\rho }_{*}$. To verify this note:

$$\frac{d}{d{\rho }_{*}}\left(\phantom{|}S(\rho ,{\rho }_{*})-ln\left[{\rho }_{*}\overline{V}\right]\right)=\int \frac{\rho \left(x\right)}{{\rho }_{*}}{d}^{3}x-\frac{1}{{\rho }_{*}}=\frac{1}{{\rho }_{*}}-\frac{1}{{\rho }_{*}}=0.$$

(32)

Therefore, we are free to pick ${\rho }_{*}$ to taste. Three particularly useful options are:

• Simply pick ${\rho }_{*}={\overline{V}}^{-1}$, in which case, we have:

  $$S_B\ge S\left(\rho ,{\overline{V}}^{-1}\right).$$

  (33)
• If all boxes have the same volume V, then automatically, we have $\overline{V}=V$. In this situation, we simply pick ${\rho }_{*}=V^{-1}$, in which case:

  $$S_B\ge S\left(\rho ,V^{-1}\right).$$

  (34)
• Alternatively, keep ${\rho }_{*}$ arbitrary, and simply live with the fact that the best we can say is:

  $$S_B\ge S(\rho ,{\rho }_{*})-ln\left[{\rho }_{*}\overline{V}\right].$$

  (35)

The central result is always:

$$S_B>S(\rho ,{\rho }_{*})+(\mathrm{some}{\rho }_{*}-\mathrm{dependent}\mathrm{normalization}\mathrm{to}\mathrm{set}\mathrm{the}\mathrm{zero}\mathrm{of}\mathrm{entropy}).$$

(36)

Subject to these technical quibbles, we can then define:

$$I_{\mathrm{correlations}}=S_B-S(\rho ,{\rho }_{*})+ln\left[{\rho }_{*}\overline{V}\right]\ge 0.$$

(37)

This is both independent of ${\rho }_{*}$ and guaranteed non-negative. Physically, this represents the “hidden information” encoded in the density-density correlations inside each of the boxes $B_i$, correlations that we are simply agreeing not to look into. Note that box-averaging in this context is an all-or-nothing proposition, there is no tunable parameter to adjust.

4.2.2. Continuum Diffusion Followed by Box-Averaging

Now, let us introduce a tunable parameter t, allowing the continuum probability density $\rho \left(x\right)$ to evolve under some diffusion process (either physical or a gedanken-process), so that $\rho \left(x\right)\to {\rho }_t\left(x\right)$ before box-averaging. Then, in terms of the (now time-dependent) boxed probability density:

$${\rho }_{B_t}\left(x\right):\mathrm{if}x\in \mathrm{int}\left(B_i\right)\mathrm{then}{\rho }_{B_t}\left(x\right)=\frac{{\left(p_i\right)}_t}{\mathrm{volume}\left(B_i\right)}=\frac{{\int }_{B_i}{\rho }_t\left(x\right)d^{3}x}{{\int }_{B_i}{d}^{3}x},$$

(38)

we immediately have without any further need of calculation:

$$S\left({\rho }_B\right)\ge S\left(\rho \right);S\left({\rho }_{B_t}\right)\ge S\left({\rho }_t\right).$$

(39)

However, a more subtle result is this:

$$\dot{S}\left({\rho }_{B_t}\right)=-\int {\dot{\rho }}_{B_t}ln\left\{\frac{{\rho }_{B_t}}{{\rho }_{*}}\right\}dx=-\int {\dot{\rho }}_{t}ln\left\{\frac{{\rho }_{B_t}}{{\rho }_{*}}\right\}dx\ge -\int {\dot{\rho }}_{t}ln\left\{\frac{{\rho }_t}{{\rho }_{*}}\right\}dx=\dot{S}\left({\rho }_t\right)\ge 0.$$

(40)

Here, we have used the fact that ${\rho }_{B_t}$ is box-wise constant as a function of x and the positivity of relative Shannon entropy. This argument now guarantees that $S\left({\rho }_{B_t}\right)$ is monotone non-decreasing as a function of the smearing parameter t. If $\overline{V}$ is held fixed (for example, if all boxes are the same size so that $\overline{V}=V$), this furthermore implies ${\dot{S}}_{B_t}\ge 0$. However, in general, because:

$$\frac{d{x}^{n\left(t\right)}}{dt}=\frac{d}{dt}exp(n\left(t\right)lnx)=\dot{n}lnx{x}^n,$$

(41)

we have:

$${\rho }_{*}\dot{\overline{V}}=\frac{d}{dt}\prod {\left({\rho }_{*}{V}_i\right)}^{p_i}=\sum {\dot{p}}_{i}ln\left({\rho }_{*}{V}_i\right)\prod {\left({\rho }_{*}{V}_i\right)}^{p_i}=\left\{\sum ({\dot{p}}_{i}ln\left({\rho }_{*}{V}_i\right))\right\}{\rho }_{*}\overline{V}.$$

(42)

Thus:

$$\dot{\overline{V}}=\overline{V}\left\{\sum {\dot{p}}_{i}ln\left({\rho }_{*}{V}_i\right)\right\},$$

(43)

which in general is nonzero (but independent of ${\rho }_{*}$) without imposing extra conditions. Nevertheless, what we do have is already quite powerful: we always have the result that $S\left({\rho }_{B_t}\right)$ is monotone non-decreasing, and whenever $\dot{\overline{V}}=0$, we even have $S_{B_t}$ monotone non-decreasing as a function of the smearing parameter t. We can then define the “hidden information” as:

$$I_{\mathrm{correlations}}=S_{B_t}-S({\rho }_t,{\rho }_{*})+ln\left[{\rho }_{*}{\overline{V}}_t\right]\ge 0.$$

(44)

This is both independent of ${\rho }_{*}$ and guaranteed non-negative. Physically, this represents the “hidden information” encoded in the now smearing-parameter-dependent density-density correlations inside each of the boxes $B_i$, correlations that we are simply agreeing not to look into. Note that box-averaging in this context thus yields a nice explicitly controllable entropy flow as a function of the coarse graining parameter t.

4.3. Discretium ⟶ Discretium: Aggregation/Averaging Forms of Coarse Graining

Suppose we have a denumerable set of states with probabilities $p_i$ for which we define:

$$S=-\sum p_{i}ln{p}_i.$$

(45)

(This could be done for instance, after breaking the universe up into a denumerable set of boxes.) Let us now consider two ways of “aggregating” these probabilities.

4.3.1. Naive Aggregation

Take any two states (any two boxes), say a and b, and simply average their probabilities, so that:

$$p_{a,\mathrm{new}}=p_{b,\mathrm{new}}=\overline{p}=\frac{p_a+p_b}{2}.$$

(46)

Note that this process leaves the number of states (boxes) fixed; it is inadvisable to merge the boxes, as that tends ultimately to decrease the entropy in a manner that is physically uninteresting. Then, set $p_a=\overline{p}\pm ϵ$ and $p_b=\overline{p}\mp ϵ$ (where we can always choose signs so that $ϵ\ge 0$). We have:

$$-p_{a}ln{p}_a-p_{b}ln{p}_b=-(\overline{p}\pm ϵ)ln(\overline{p}\pm ϵ)-(\overline{p}\mp ϵ)ln(\overline{p}\mp ϵ).$$

(47)

However,

$$\frac{d}{dϵ}\left[\phantom{|}-(\overline{p}\pm ϵ)ln(\overline{p}\pm ϵ)-(\overline{p}\mp ϵ)ln(\overline{p}\mp ϵ)\right]=\mp ln(\overline{p}\pm ϵ)\pm ln(\overline{p}\mp ϵ)=ln\left(\frac{\overline{p}-ϵ}{\overline{p}+ϵ}\right)\le 0.$$

(48)

Therefore:

$$-(\overline{p}\pm ϵ)ln(\overline{p}\pm ϵ)-(\overline{p}\mp ϵ)ln(\overline{p}\mp ϵ)\le {\left[-(\overline{p}\pm ϵ)ln(\overline{p}\pm ϵ)-(\overline{p}\mp ϵ)ln(\overline{p}\mp ϵ)\right]}_{ϵ=0}.$$

(49)

That is:

$$-p_{a}ln{p}_a-p_{b}ln{p}_b\le -2\overline{p}ln\overline{p}.$$

(50)

Thus, the result of aggregating (averaging) any two states in this manner is always to increase the Shannon entropy:

$$S_{\mathrm{aggregated}}\ge S_{\mathrm{unaggregated}}.$$

(51)

Note that the end point of this aggregation process occurs when all of the $p_i$ are equal, so $p_i=1/N$ and $S\to lnN$, which is its maximum possible value. We can now define the “hidden information” as:

$$I_{\mathrm{correlations}}=S_{\mathrm{aggregated}}-S_{\mathrm{unaggregated}}\ge 0.$$

(52)

These are now correlations between the states (boxes) that we lumped together in the averaging process (this is at this stage an “all-or-nothing” process with no tunable parameter).

4.3.2. Asymmetric Aggregation

A generalization of this notion of aggregation is to define an “asymmetric averaging”:

$$p_{a,\mathrm{new}}=\lambda p_a+(1-\lambda )p_b;p_{b,\mathrm{new}}=(1-\lambda )p_a+\lambda p_b;\lambda \in (0,1).$$

(53)

Then, ${\overline{p}}_{new}=\overline{p}$, the average is unaffected, whereas $|p_{a,\mathrm{new}}-p_{b,\mathrm{new}}|=|2\lambda -1\left|\right|p_a-p_b|<|p_a-p_b|$. Therefore, the new $p_a$ and $p_b$ move “closer together”.

Then, we can again write $p_a=\overline{p}\pm ϵ$ and $p_b=\overline{p}\mp ϵ$, and in addition, we have $p_{a,\mathrm{new}}=\overline{p}\pm {ϵ}_2$ and $p_{b,\mathrm{new}}=\overline{p}\mp {ϵ}_2$, with both $ϵ$ and ${ϵ}_2$ positive, and ${ϵ}_2<ϵ$. However, then, the same argument as used previously now gives:

$$-(\overline{p}\pm ϵ)ln(\overline{p}\pm ϵ)-(\overline{p}\mp ϵ)ln(\overline{p}\mp ϵ)\le {\left[-(\overline{p}\pm ϵ)ln(\overline{p}\pm ϵ)-(\overline{p}\mp ϵ)ln(\overline{p}\mp ϵ)\right]}_{ϵ={ϵ}_2}.$$

(54)

That is:

$$-p_{a}ln{p}_a-p_{b}ln{p}_b\le -p_{a,new}ln{p}_{a,new}-p_{b,new}ln{p}_{b,new}.$$

(55)

Thus, the result of aggregating (averaging) any two states in this manner is always to increase the Shannon entropy. The rest of the argument follows as previously; with the same conclusions (this argument can also be rephrased in terms of the concavity of the Shannon entropy as a function of the $p_i$). We can again define the “hidden information” as:

$$I_{\mathrm{correlations}}=S_{\mathrm{aggregated}}-S_{\mathrm{unaggregated}}\ge 0.$$

(56)

These are now correlations between the states (boxes) that we lumped together in the asymmetric averaging process.

4.4. Summary: Coarse Graining Classical Shannon Entropy

As we have just seen, it is possible to construct a number of reasonably well-behaved models for classical coarse graining of the Shannon entropy. Some of these models are “all-or-nothing”, without any tunable smearing parameter, whereas other models depend on a tunable coarse graining parameter (that we have chosen to designate t even when it might not represent physical time). The trickiest of these classical constructions is when one coarse-grains a continuum down to a discretium, since the normalizing parameter ${\rho }_{*}$ (which is essential to normalizing the continuum Shannon entropy) no longer appears in the discrete Shannon entropy. Which if any of these models is most relevant depends on the particular process under consideration; in all cases, these constructions provide a well-defined framework to use and allow explicit definition and determination of the hidden information.

5. Coarse Graining the Quantum Von Neumann Entropy

For quantum systems described by a normalized density matrix $\rho$, the von Neumann entropy is:

$$S=-\mathrm{tr}(\rho ln\rho );\mathrm{tr}\left(\rho \right)=1.$$

(57)

There is no continuum versus discretium distinction to worry about, and since the density matrix is automatically dimensionless, similarly there is never any need to introduce a “reference density” ${\rho }_{*}$. There are still at least three interesting and fundamental notions of coarse graining to consider:

• Maximal mixing.
• Partial traces (four sub-varieties).
• Decoherence (two sub-varieties).

In Section 7, we shall subsequently extend the constructions of this current section by the use of a variant of Hawking’s super-scattering operator to be introduced in Section 6.

5.1. Quantum Coarse Graining by Maximal Mixing

For simplicity, let the dimension of the Hilbert space be finite: N. Now, let $I_N$ denote the N-dimensional identity matrix. Then, $\frac{I_N}{N}$ is a properly normalized density matrix with entropy $lnN$. In fact, this is the maximally-mixed density matrix with maximum entropy given the dimensionality of the Hilbert space.

Now, consider this gedanken-process: Acting on the density matrix $\rho$, to keep things properly normalized, take:

$$\rho \to {\rho }_s=(1-s)\rho +s\frac{I_N}{N};s\in [0,1].$$

(58)

Here, one is using the smearing parameter s (which is certainly not physical time) to selectively ignore what would otherwise be physically accessible information; effectively one is “blurring” (that is, coarse graining) the system by throwing away (ignoring) position information in Hilbert space. (This is strictly a gedanken-process; we know of no physical way of implementing this particular mapping. Nevertheless, it is still appropriate to call this a (mathematical) coarse graining).

Now, use the fact that the von Neumann entropy is strictly concave in the density matrix:

$$S((1-s){\rho }_0+s{\rho }_1)\ge (1-s)S\left({\rho }_0\right)+sS\left({\rho }_1\right),$$

(59)

whence:

$$S\left({\rho }_s\right)\ge (1-s)S\left({\rho }_0\right)+slnN.$$

(60)

That is:

$$S\left({\rho }_s\right)-S\left({\rho }_0\right)\ge s\{lnN-S\left({\rho }_0\right)\}.$$

(61)

Then, we can formally define the hidden information by setting:

$$I_{\mathrm{correlations}}=S\left({\rho }_s\right)-S\left({\rho }_0\right)\ge s\{lnN-S\left({\rho }_0\right)\}\ge 0.$$

(62)

This represents the amount of information that is temporarily lost in the s-dependent blurring process, which in view of the gadanken-nature of the blurring process is recoverable simply by de-blurring one’s vision. In this construction, one also has a strict upper bound on the hidden information:

$$I_{\mathrm{correlations}}=S\left({\rho }_s\right)-S\left({\rho }_0\right)\le \{lnN-S\left({\rho }_0\right)\}.$$

(63)

It is important to note that this partial maximal mixing procedure involves some subtleties in its interpretation. Compared to other more physical coarse graining procedures, maximal mixing has a more mathematical flavour. Nevertheless, it is still appropriate to call it coarse graining due it continues to satisfy the crucial feature of selectively ignoring some physical aspects of the system, which in turn leads to an effective amount of hidden information.

5.2. Quantum Coarse Graining by Partial Traces

Many quantum information sources and texts hint at how taking partial traces is an example of “coarse graining”, but never quite seem to really get explicit about it. Below, we outline a few explicit constructions, some tunable, some “all-or-nothing”.

5.2.1. Non-Tunable Partial Traces

Suppose the Hilbert space factorizes $H_{AB}=H_A\otimes H_B$ (though the density matrix need not factorize), and define “simultaneous coarse grainings” onto the sub-spaces $H_A$ and $H_B$ by:

$${\rho }_{AB}\to {\rho }_{A:B}={\rho }_A\otimes {\rho }_B={\mathrm{tr}}_B\left({\rho }_{AB}\right)\otimes {\mathrm{tr}}_A\left({\rho }_{AB}\right).$$

(64)

Here, the notation A:Beffectively means this: slide a (unphysical, purely mathematical) barrier between the sub-spaces A and B, so there is no longer any cross-talk between the two sub-spaces (notationally, it is more common to see notation such as $S_{A;B}$, but this seems to us to unnecessarily suggest an asymmetry between the two factor sub-spaces). The idea is that people “living in” A agree not to look into B and vice versa. Then, it is relatively easy to prove the quite standard result that:

$$S_{A:B}=\mathrm{tr}\left(\phantom{|}[{\rho }_A\otimes {\rho }_B]ln[{\rho }_A\otimes {\rho }_B]\right)=\mathrm{tr}\left(\phantom{|}[{\rho }_A\otimes {\rho }_B]\{ln\left[{\rho }_A\right]\otimes I_B+I_A\otimes ln\left[{\rho }_B\right]\}\right)=S_A+S_B,$$

(65)

and furthermore, to derive the “triangle inequalities”:

$$|S_A-S_B|\le S_{AB}\le S_A+S_B.$$

(66)

Combining, these results, we see:

$$S_{A:B}=S_A+S_B\ge S_{AB}\ge |S_A-S_B|.$$

(67)

The hidden information can be defined by:

$$I_{\mathrm{correlations}}=S_{A:B}-S_{AB}=S_A+S_B-S_{AB}.$$

(68)

This is just the completely usual notion of mutual information. The triangle inequalities give:

$$I_{\mathrm{correlations}}=S_{A:B}-S_{AB}=S_A+S_B-S_{AB}\ge 0,$$

(69)

(so the hidden information is always non-negative), and furthermore:

$$I_{\mathrm{correlations}}=S_{A:B}-S_{AB}=S_A+S_B-S_{AB}\le S_A+S_B-|S_A-S_B|=2min\{S_A,S_B\},$$

(70)

so the hidden information is always bounded from above. Note that the current construction has no tunable parameter; the coarse graining is “all-or-nothing”, a feature that we shall now show how to evade.

5.2.2. Tunable Partial Traces

Let us now consider this model:

$${\rho }_{AB}\to {\rho }_s=(1-s){\rho }_{AB}+s{\rho }_A\otimes {\rho }_B;s\in [0,1].$$

(71)

This smoothly interpolates between the identity process at $s=0$ and the factorized subspace process described in the previous subsection at $s=1$ (the parameter s is not physical time; it merely characterizes the amount of coarse graining; so this is at this stage a gedanken-process). Concavity of the von Neumann entropy yields:

$$S\left({\rho }_s\right)=S\left((1-s){\rho }_{AB}+s{\rho }_A\otimes {\rho }_B\right)\ge (1-s)S\left({\rho }_{AB}\right)+sS\left({\rho }_A\otimes {\rho }_B\right),$$

(72)

whence:

$$S\left({\rho }_s\right)\ge (1-s)S_{AB}+s(S_A+S_B).$$

(73)

By the triangle inequality, this implies:

$$S\left({\rho }_s\right)\ge S_{AB}+s(S_A+S_B-S_{AB})\ge S_{AB}.$$

(74)

On the other hand, because partial trace applied to ${\rho }_s$ yields ${\rho }_s\to {\rho }_A\otimes {\rho }_B$ (independent of the value of s as long as it is in the range $s\in [0,1]$), we have:

$$S\left({\rho }_s\right)\le S({\rho }_A\otimes {\rho }_B)=S_A+S_B.$$

(75)

Now, consider the entropy flow under evolution of the parameter s:

$$\dot{S}\left({\rho }_s\right)=-\mathrm{tr}({\dot{\rho }}_{s}ln{\rho }_s)=-\mathrm{tr}([({\rho }_A\otimes {\rho }_B)-{\rho }_{AB}]ln{\rho }_s).$$

(76)

However, we can write this as:

$$\dot{S}\left({\rho }_s\right)=+\mathrm{tr}\left(\left[\frac{{\rho }_s-[{\rho }_A\otimes {\rho }_B]}{1-s}\right]ln{\rho }_s\right)=\left\{\frac{-\mathrm{tr}([{\rho }_A\otimes {\rho }_B]ln{\rho }_s)-S\left({\rho }_s\right)}{1-s}\right\}.$$

(77)

Now, applying the Klein inequality for quantum relative entropies, the positivity of relative von Neumann entropy, we have:

$$\mathrm{tr}([{\rho }_A\otimes {\rho }_B]ln{\rho }_s)\le \mathrm{tr}([{\rho }_A\otimes {\rho }_B]ln[{\rho }_A\otimes {\rho }_B])=-S\left({\rho }_A\right)-S\left({\rho }_B\right).$$

(78)

Therefore:

$$\dot{S}\left({\rho }_s\right)\ge \frac{1}{1-s}\{S\left({\rho }_A\right)+S\left({\rho }_B\right)-S\left({\rho }_s\right)\}\ge 0.$$

(79)

Thus, this tuned entropy is nicely monotonic (non-decreasing) in the tuning parameter s and is a good candidate for a (explicit and controllable) coarse graining process. We can then safely define the “hidden information” as:

$$I_{\mathrm{correlations}}=S\left({\rho }_s\right)-S_{AB}.$$

(80)

There is still a nice lower bound:

$$I_{\mathrm{correlations}}\ge s(S_A+S_B-S_{AB})\ge 0,$$

(81)

guaranteeing positivity of the hidden information. On the other hand, the best general upper bound seems to be relatively weak:

$$I_{\mathrm{correlations}}\le S\left({\rho }_s\right)-|S_A-S_B|\le S_A+S_B-|S_A-S_B|=2min\{S_A,S_B\}.$$

(82)

5.2.3. Asymmetric Partial Trace and Maximal Mixing

Here is a more brutal asymmetric combination of partial trace and maximal mixing:

$${\rho }_{AB}\to {\rho }_A\otimes \frac{I_B}{N_B}={\mathrm{tr}}_B\left({\rho }_{AB}\right)\otimes \frac{I_B}{N_B}.$$

(83)

Here, one is not just taking a partial trace over the subsystem B; one is also maximally mixing the subspace B. Therefore, this process is asymmetric in the way it treats Subsystems A and B (this is best thought of as a gedanken-process). To verify that entropy has increased in this “all-or-nothing” coarse graining process, we simply note that:

$$S\left({\rho }_{AB}\right)\le S({\rho }_A\otimes {\rho }_B)\le S\left({\rho }_A\otimes \frac{I_B}{N_B}\right);$$

(84)

where, specifically, it is easy to see that:

$$S\left({\rho }_A\otimes \frac{I_B}{N_B}\right)=S_A+ln{N}_B.$$

(85)

The hidden information can easily be defined by:

$$I_{\mathrm{correlations}}=S\left({\rho }_A\otimes \frac{I_B}{N_B}\right)-S_{AB}=S_A+ln{N}_B-S_{AB}\ge ln{N}_B-S_B\ge 0,$$

(86)

and is guaranteed non-negative. There is also an upper bound:

$$I_{\mathrm{correlations}}=S_A+ln{N}_B-S_{AB}\le S_A+ln{N}_B-|S_A-S_B|\le (ln{N}_B-S_B)+2min\{S_A,S_B\}.$$

(87)

This seems to be the best upper bound one can extract in this context.

5.2.4. Asymmetric Partial Trace Plus Maximal Mixing, with Tunable Parameter

Starting from the previous case, now consider the effect of adding a tunable parameter:

$${\rho }_{AB}\to {\rho }_s=(1-s){\rho }_{AB}+s{\rho }_A\otimes \frac{I_B}{N_B};s\in [0,1].$$

(88)

(This is again best thought of as a gedanken-process, rather than a physical coarse graining.) By concavity of the von Neumann entropy:

$$S\left({\rho }_s\right)=S\left((1-s){\rho }_{AB}+s{\rho }_A\otimes \frac{I_B}{N_B}\right)\ge (1-s)S\left({\rho }_{AB}\right)+sS\left({\rho }_A\otimes \frac{I_B}{N_B}\right),$$

(89)

so that:

$$S\left({\rho }_s\right)\ge (1-s)S_{AB}+s(S_A+ln{N}_B)=S_{AB}+s\{S_A+ln{N}_B-S_{AB}\}\ge S_{AB}.$$

(90)

To establish an upper bound, note that under the process envisaged in the previous Section 5.2.3 (asymmetric partial trace and mixing), we have (now for all s in the appropriate interval):

$${\rho }_s\to {\rho }_A\otimes \frac{I_B}{N_B}$$

(91)

so by the arguments in the previous Section 5.2.3, we have:

$$S\left({\rho }_s\right)\le S_A+ln{N}_B.$$

(92)

Checking monotonicity in the parameter s now requires that we investigate:

$$\dot{S}\left({\rho }_s\right)=-\mathrm{tr}({\dot{\rho }}_{s}ln{\rho }_s)=-\mathrm{tr}\left(\left[{\rho }_A\otimes \frac{I_B}{N_B}-{\rho }_{AB}\right]ln{\rho }_s\right)$$

(93)

To proceed, we adapt the discussion in Section 5.2.2 (tunable partial traces). We note we can write the above as:

$$\dot{S}\left({\rho }_s\right)=+\mathrm{tr}\left(\left[\frac{{\rho }_s-[{\rho }_A\otimes I_B/N_B]}{1-s}\right]ln{\rho }_s\right)=\left\{\frac{-\mathrm{tr}([{\rho }_A\otimes I_B/N_B]ln{\rho }_s)-S\left({\rho }_s\right)}{1-s}\right\}$$

(94)

Now, applying the Klein inequality, the positivity of relative von Neumann entropy, we have:

$$\mathrm{tr}([{\rho }_A\otimes I_B/N_B]ln{\rho }_s)\le \mathrm{tr}([{\rho }_A\otimes I_B/N_B]ln[{\rho }_A\otimes I_B/N_B])=-S\left({\rho }_A\right)-ln{N}_B.$$

(95)

Therefore:

$$\dot{S}\left({\rho }_s\right)\ge \frac{1}{1-s}\{S\left({\rho }_A\right)+ln{N}_B-S\left({\rho }_s\right)\}\ge 0.$$

(96)

Thus, this partial trace plus maximal mixing tunable entropy is nicely monotonic (non-decreasing) in the tuning parameter s and is another good candidate for a (explicit and controllable) coarse graining process. Then, for the “hidden information”, we can safely define:

$$I_{\mathrm{correlations}}=S\left({\rho }_s\right)-S_{AB}.$$

(97)

We note:

$$I_{\mathrm{correlations}}\ge s(S_A+ln{N}_B-S_{AB})\ge s(ln{N}_B-S_B)\ge 0,$$

(98)

thereby guaranteeing non-negativity and a nice lower bound. The best general upper bound in this context seems to be:

$$I_{\mathrm{correlations}}\le S\left({\rho }_s\right)-|S_A-S_B|\le S_A+ln{N}_B-|S_A-S_B|.$$

(99)

5.3. Quantum Coarse-Graining Defined by Decoherence

Choose any fixed, but arbitrary basis for the Hilbert space, and let $P_a$ be the corresponding one-dimensional projection operators onto these basis elements. Therefore, we have:

$$P_a{P}_b={\delta }_{ab}{P}_b;P_a^2=P_a;\sum _a{P}_a=I_N.$$

(100)

Now, use these projection operators to define decoherence (relative to the specified basis). Decoherence is typically viewed as a physical process associated with a semi-classical environment associated with the specified basis, but we could also think of it as a gedanken-process where we are just agreeing to ignore part of the density matrix (for a general background on decoherence, see for instance, [31,32,33,34,35,36,37]).

5.3.1. Full Decoherence

Define full decoherence as follows:

$$\rho \to {\rho }_D=\sum _a{P}_a\mathrm{tr}\left(P_a\rho \right).$$

(101)

(This preserves the $\mathrm{tr}\left(\rho \right)=1$ condition and kills off all non-diagonal elements in the density matrix.) When viewed as a gedanken-process, one is effectively agreeing to not look at the off-diagonal elements. Now, in this particular, case concavity does not tell one anything useful:

$$S\left({\rho }_D\right)=S\left(\sum _a{P}_a\mathrm{tr}\left(P_a\rho \right)\right)\ge \sum _a\mathrm{tr}\left(P_a\rho \right)S\left(P_a\right)=0.$$

(102)

One just obtains the trivial bound $S\left({\rho }_D\right)\ge 0$. Considerably more interesting in this context is the relative entropy inequality (the Klein inequality), which states that for any two density matrices $\rho$ and $\sigma$:

$$\mathrm{tr}\left(\rho \right[ln\rho -ln\sigma \left]\right)\ge 0.$$

(103)

In particular:

$$\mathrm{tr}\left(\rho [ln\rho -ln{\rho }_D]\right)\ge 0,$$

(104)

which can be rearranged to yield:

$$-\mathrm{tr}(\rho ln{\rho }_D)\ge -\mathrm{tr}(\rho ln\rho )=S\left(\rho \right).$$

(105)

Now, ${\rho }_D$ is diagonal, so $ln{\rho }_D$ is diagonal, so in the trace in the LHS above, only the diagonal elements of $\rho$ contribute:

$$-\mathrm{tr}(\rho ln{\rho }_D)=-\mathrm{tr}({\rho }_{D}ln{\rho }_D)=S\left({\rho }_D\right).$$

(106)

Therefore, overall, we see that $S\left({\rho }_D\right)\ge S\left(\rho \right)$; the decoherence process is entropy non-decreasing (as it should be). In a completely standard manner, we now define the hidden information as:

$$I_{\mathrm{correlations}}=S\left({\rho }_D\right)-S\left(\rho \right)\ge 0,$$

(107)

where the hidden information is hiding in the off-diagonal correlations that one has agreed to not look at in implementing the decoherence process. The best upper bound seems to be:

$$I_{\mathrm{correlations}}=S\left({\rho }_D\right)-S\left(\rho \right)\le lnN-S\left(\rho \right).$$

(108)

The full decoherence of the off-diagonal terms allows us to completely forget about quantum effects; the coarse-grained system is effectively classical (albeit a classical stochastic system).

5.3.2. Partial (Tunable) Decoherence

Now, define partial decoherence as follows:

$$\rho \to {\rho }_{D_s}=(1-s)\rho +s{\rho }_D=(1-s)\rho +s\left\{\sum _a{P}_a\mathrm{tr}\left(P_a\rho \right)\right\}.$$

(109)

(This preserves the $\mathrm{tr}\left(\rho \right)=1$ condition, but now only partially suppresses all non-diagonal elements in the density matrix.) We note that concavity implies:

$$S\left({\rho }_{Ds}\right)=S\left((1-s)\rho +s{\rho }_D\right)\ge (1-s)S\left(\rho \right)+sS\left({\rho }_D\right)=S\left(\rho \right)+s\{S\left({\rho }_D\right)-S\left(\rho \right)\}\ge S\left(\rho \right).$$

(110)

By using the Klein inequality:

$$\mathrm{tr}({\rho }_{D_s}ln{\rho }_{D_s})\le \mathrm{tr}({\rho }_{D_s}ln{\rho }_D)$$

(111)

However, since in the current situation, ${\rho }_D$ is diagonal (and so, $ln{\rho }_D$ is also diagonal) and since all of the diagonal elements of ${\rho }_{D_s}$ are by construction constants independent of s, this implies:

$$\mathrm{tr}({\rho }_{D_s}ln{\rho }_{D_s})\le \mathrm{tr}({\rho }_{D}ln{\rho }_D)$$

(112)

Thus, $S\left({\rho }_D\right)\ge S\left({\rho }_{D_s}\right)$, which has the nice interpretation that partial decoherence involves less entropy than full decoherence. Overall, we obtain the chain of inequalities:

$$S\left(\rho \right)=S\left({\rho }_{D_0}\right)\le S\left({\rho }_{D_s}\right)\le S\left({\rho }_{D_1}\right)=S\left({\rho }_D\right).$$

(113)

We can do even more and demonstrate that under partial decoherence, the entropy increases monotonically in the control parameter s. We note:

$$\begin{array}{ccc}\hfill \dot{S}\left({\rho }_{D_s}\right)& =& -\mathrm{tr}({\dot{\rho }}_{D_s}ln{\rho }_{D_s})\hfill \\ & =& -\mathrm{tr}([{\rho }_D-\rho ]ln{\rho }_{D_s})\hfill \\ & =& -\mathrm{tr}\left(\left[\frac{{\rho }_D-{\rho }_{D_s}}{1-s}\right]ln\left({\rho }_{D_s}\right)\right)\hfill \\ & =& \frac{1}{1-s}\left(-S\left({\rho }_{D_s}\right)-\mathrm{tr}({\rho }_{D}ln\left({\rho }_{D_s}\right))\right).\hfill \end{array}$$

(114)

Again appealing to the Klein inequality, we have $\mathrm{tr}({\rho }_{D}ln\left({\rho }_{D_s}\right))\le \mathrm{tr}({\rho }_{D}ln\left({\rho }_D\right))=-S\left({\rho }_D\right)$, whence:

$$\dot{S}\left({\rho }_{D_s}\right)\ge \frac{S\left({\rho }_D\right)-S\left({\rho }_{D_s}\right)}{1-s}\ge 0.$$

(115)

Therefore, again, we have a nice calculable monotonically non-decreasing explicit model for coarse graining. In a completely standard manner, we now define the hidden information as:

$$I_{\mathrm{correlations}}=S\left({\rho }_{D_s}\right)-S\left(\rho \right)\ge s\{S\left({\rho }_D\right)-S\left(\rho \right)\}\ge 0,$$

(116)

where the hidden information is hiding in the off-diagonal correlations that have been partially suppressed because one has agreed to not look at them too carefully in implementing the decoherence process. The best generic upper bound seems to be:

$$I_{\mathrm{correlations}}=S\left({\rho }_{D_s}\right)-S\left(\rho \right)\le lnN-S\left(\rho \right).$$

(117)

Thus, we have obtained the same upper bound as in the previous subsection, where we completely suppressed the off-diagonal terms.

5.4. Summary: Coarse Graining Quantum Von Neumann Entropy

In the current section, we have built three distinct explicit mathematical/physical models of quantum coarse graining of the von Neumann entropy. These models were based on maximal mixing; on partial traces (four sub-varieties); and on decoherence (two sub-varieties), seven sub-varieties in total. Some of these models were “all-or-nothing” (untunable) models; others have an explicit coarse graining parameter s, and for these tunable models, we carefully proved that the entropy was monotonic in the smearing parameter. This already gives one a much more concrete idea of what coarse graining in a quantum context should be taken to be. In the next two sections, we shall extend these ideas first by showing that these coarse graining processes can be viewed as specific examples of Hawking’s super-scattering operator [21] and then apply super-scattering to define a notion of diffusion in Hilbert space.

6. Hawking’s Super-Scattering Operator

In an attempt to develop a non-unitary version of quantum mechanics, Hawking introduced the notion of a super-scattering operator ${\displaystyle \$}$; see [21]. At its most basic, a super-scattering operator is taken to be a trace-preserving linear mapping from density matrices to density matrices: $\rho \to {\displaystyle \$}\rho$, with $\mathrm{tr}\left(\rho \right)=1$ implying $\mathrm{tr}\left({\displaystyle \$}\rho \right)=1$. A proper super-scattering operator is one that cannot simply be factorized into a tensor product of unitary scattering matrices $\mathbb{S}$. That is ${\displaystyle \$}\ne {\mathbb{S}}^{†}\otimes \mathbb{S}$, implying $\rho \to {\displaystyle \$}\rho \ne {\mathbb{S}}^{†}\rho \mathbb{S}$. While these ideas were originally mooted in a black hole context, there is nothing intrinsically black hole related in the underlying concept. It is sometimes implicitly demanded that super-scattering operators be entropy non-decreasing $S\left({\displaystyle \$}\rho \right)\ge S\left(\rho \right)$, but one could just as easily demand linearity and trace-preservation as the key defining characteristics and then divide super-scattering operators into entropy non-decreasing, entropy non-increasing, and other. This point of view has the advantage that the inverse of a super-scattering operator ${{\displaystyle \$}}^{-1}$, if it exists, is also a super-scattering operator.

To add to the confusion, depending on the sub-field, people often use different terminology for essentially the same concept, such as “trace-preserving (completely) positive operators”, or “quantum maps”, or “quantum process”, or “quantum channels”. Usage (and precise definition) within various sub-fields is (unfortunately) not entirely standardized. We will try to keep things simple with a minimal set of assumptions and with terminology that is likely to resonate widely with the larger sub-communities of interest.

Perhaps the simplest super-scattering process one can think of is this: pick a set of positive numbers $p_i$, such that ${\sum }_i{p}_i=1$ (which you might like to think of as classical probabilities) and a set of unitary matrices $U_i$. Now, consider the mapping:

$$\rho \to {{\displaystyle \$}}_{\{p,U\}}\rho ={\rho }_{\{p,U\}}=\sum _i{p}_i{U}_i^{†}\rho U_i.$$

(118)

This is not a unitary transformation; it is instead an “incoherent sum of unitary transformations”. However, note that the process is linear and trace preserving. Then, for this particular super-scattering process, we have:

$$S_{\{p,U\}}=\mathrm{tr}({\rho }_{\{p,U\}}ln{\rho }_{\{p,U\}})=\mathrm{tr}\left(\left[\sum _i{p}_i{U}_i^{†}\rho U_i\right]ln\left[\sum _i{p}_i{U}_i^{†}\rho U_i\right]\right).$$

(119)

By concavity of the von Neumann entropy:

$$S_{\{p,U\}}=\mathrm{tr}({\rho }_{\{p,U\}}ln{\rho }_{\{p,U\}})\ge \sum _i{p}_i\mathrm{tr}\left(\left[U_i^{†}\rho U_i\right]ln\left[U_i^{†}\rho U_i\right]\right)=\sum _i{p}_i\mathrm{tr}(\rho ln\rho )=S\left(\rho \right)=S.$$

(120)

That is, this particular super-scattering operator is entropy non-decreasing, $S_{\{p,U\}}\ge S$. Unfortunately, we do not have any nice monotonicity result describing how $S_{\{p,U\}}$ grows as a function of the $p_i$ and the $U_i$, this is again an “all-or-nothing scenario” (indeed, systematically finding constructions for which we are always guaranteed a nice monotonicity result is the topic of the next section).

As an aside, we point out that if one chooses the $p_i$ and $U_i$ from some classical stochastic ensemble, then one should work with (classical) expectation values:

$$\rho \to {{\displaystyle \$}}_{\langle p,U\rangle }\rho =〈\sum _i{p}_i{U}_i^{†}\rho U_i〉.$$

(121)

A suitable choice of the (classical) statistical ensemble for the $p_i$ and $U_i$ could then be used to build the decoherence operator $\rho \to {\rho }_D$ described in Section 5.3.1 above.

In a rather different direction, many of the coarse graining processes considered in Section 5 above can be reinterpreted as super-scattering processes. For instance, in addition to the construction given immediately above, the following are all entropy non-decreasing super-scattering processes:

Name	Definition	Condition
Maximal	${\displaystyle \phantom{|}\rho \to {{\displaystyle \$}}_M\rho =(1-s)\rho +s\frac{I_N}{N}}$	$s\in [0,1]$.
Partial/maximal	${\displaystyle \phantom{|}\rho \to {{\displaystyle \$}}_P\rho ={\rho }_A\otimes \frac{I_B}{N_B}}$	-
Partial/maximal/tunable	${\displaystyle \phantom{|}\rho \to {{\displaystyle \$}}_{P_s}\rho =(1-s)\rho +s{\rho }_A\otimes \frac{I_B}{N_B}}$	$s\in [0,1]$.
Full decoherence	${\displaystyle \phantom{|}\rho \to {{\displaystyle \$}}_D\rho =\sum _a{P}_a\mathrm{tr}\left(P_a\rho \right)}$	-
Partial decoherence	${\displaystyle \phantom{|}\rho \to {{\displaystyle \$}}_{D_s}\rho =(1-s)\rho +s\left\{\sum _a{P}_a\mathrm{tr}\left(P_a\rho \right)\right\}}$	$s\in [0,1]$.
Incoherent sum	${\displaystyle \phantom{|}\rho \to {{\displaystyle \$}}_{\{p,U\}}\rho =\sum _i{p}_i{U}_i^{†}\rho U_i}$	-

Oddly enough, it is the partial trace process ${\rho }_{AB}\to {\rho }_A\otimes {\rho }_B$ that is not a super-scattering process. The problem is that while the individual partial traces ${\rho }_{AB}\to {\rho }_A$ and ${\rho }_{AB}\to {\rho }_B$ are both linear functions, the direct-product reassembly process ${\rho }_A\otimes {\rho }_B$ is not linear. Therefore, ${\rho }_{AB}\to {\rho }_A\otimes {\rho }_B$ is not a linear mapping; it is not a super-scattering process. However, when viewed as a nonlinear process ${\rho }_{AB}\to \mathcal{N}{\rho }_{AB}={\rho }_A\otimes {\rho }_B$, the nonlinear operator $\mathcal{N}$ is idempotent, ${\mathcal{N}}^2=\mathcal{N}$. We will come back to this point later.

7. Diffusion on Hilbert Space

We shall now develop the notion of diffusion on Hilbert space, similar to what we did in physical space when considering continuum Shannon entropies. There are two routes to consider:

• Diffusion via super-scattering.
• Diffusion via partial traces.

7.1. Super-Scattering-Based Diffusion

Now, consider the class of super-scattering operators ${{\displaystyle \$}}_{*}$ that are entropy increasing:

$$S\left({{\displaystyle \$}}_{*}\rho \right)\ge S\left(\rho \right).$$

(122)

Certainly, the decoherence super-scattering operator ${{\displaystyle \$}}_D$ falls in this class; so does ${{\displaystyle \$}}_{D_s}$. Ditto for the other super-scattering operators (based on maximal mixing, partial traces and the like), considered above: ${{\displaystyle \$}}_M$, ${{\displaystyle \$}}_P$, ${{\displaystyle \$}}_{P_s}$ and ${{\displaystyle \$}}_{\{p,U\}}$. In particular, such entropy non-decreasing super-scattering operators must leave maximally mixed states maximally mixed:

$${{\displaystyle \$}}_{*}\left(\frac{I_N}{N}\right)=\frac{I_N}{N}.$$

(123)

For any one of these entropy non-decreasing ${{\displaystyle \$}}_{*}$ operators, we now define:

$${\rho }_t=e^{-t}{e}^{t{\$}_{*}}\rho ;\mathrm{so}\mathrm{that}{\dot{\rho }}_t={{\displaystyle \$}}_{*}{\rho }_t-{\rho }_t.$$

(124)

Then, for the von Neumann entropy, we have:

$$\begin{array}{ccc}\hfill {\dot{S}}_t& =& -\mathrm{tr}({\dot{\rho }}_{t}ln{\rho }_t)\hfill \\ & =& -\mathrm{tr}(\left[{{\displaystyle \$}}_{*}{\rho }_t\right]ln{\rho }_t)+\mathrm{tr}({\rho }_{t}ln{\rho }_t)\hfill \\ & =& -\mathrm{tr}(\left[{{\displaystyle \$}}_{*}{\rho }_t\right]ln\left[{{\displaystyle \$}}_{*}{\rho }_t\right])+\mathrm{tr}({\rho }_{t}ln{\rho }_t)-\{\mathrm{tr}(\left[{{\displaystyle \$}}_{*}{\rho }_t\right]ln{\rho }_t)-\mathrm{tr}(\left[{{\displaystyle \$}}_{*}{\rho }_t\right]ln\left[{{\displaystyle \$}}_{*}{\rho }_t\right])\}\hfill \\ & =& S\left({\$}_{*}{\rho }_t\right)-S\left({\rho }_t\right)+\{\mathrm{tr}(\left[{\$}_{*}{\rho }_t\right]ln\left[{\$}_{*}{\rho }_t\right])-\mathrm{tr}(\left[{\$}_{*}{\rho }_t\right]ln{\rho }_t)\}.\hfill \end{array}$$

(125)

However, the relative entropy inequality (the Klein inequality) is:

$$\mathrm{tr}\left(\rho \right[ln\rho -ln\sigma \left]\right)\ge 0,$$

(126)

so automatically:

$$\{\mathrm{tr}(\left[{{\displaystyle \$}}_{*}{\rho }_t\right]ln\left[{\$}_{*}{\rho }_t\right])-\mathrm{tr}(\left[{{\displaystyle \$}}_{*}{\rho }_t\right]ln{\rho }_t)\}\ge 0,$$

(127)

whence:

$${\dot{S}}_t\ge S\left({{\displaystyle \$}}_{*}{\rho }_t\right)-S\left({\rho }_t\right)\ge 0.$$

(128)

That is, as long as the super-scattering operator ${{\displaystyle \$}}_{*}$ is guaranteed entropy non-decreasing, then so is the Hilbert-space diffusion process $\rho \to {\rho }_t=e^{-t}{e}^{t{\$}_{*}}\rho$. Since the entropy is always non-decreasing, the only thing that ${\rho }_t$ can possibly converge on as $t\to \infty$ is the maximally mixed state $\frac{I_N}{N}$ with the maximal entropy $lnN$. That is, the compound object:

$$e^{-t}{e}^{t{\$}_{*}}=exp\left(t[{{\displaystyle \$}}_{*}-I_{N\otimes N}]\right)=exp(t\Delta )$$

(129)

is itself a super-scattering process that can be thought of as a diffusion operator on Hilbert space, corresponding to a “Laplacian”-like operator $\Delta ={{\displaystyle \$}}_{*}-I_{N\otimes N}$, which has the property $\mathrm{tr}(\Delta \rho )=0$ (in analogy with the property $\int \Delta \rho \left(x\right)d^{3}x=0$ holding in the continuum for classical probability distributions). As usual, we can define the hidden information as:

$$I_{\mathrm{correlations}}=S\left(e^{t\Delta }\rho \right)-S\left(\rho \right)\ge 0,$$

(130)

which is guaranteed positive and monotonic in the coarse graining parameter t. The t parameter may or may not represent physical time; we could view the entire construction as a gedanken-process in which case, t would just be a tuning parameter giving explicit control of the extent of coarse graining. The only generic upper bound on the hidden information in this context seems to be:

$$I_{\mathrm{correlations}}=S\left(e^{t\Delta }\rho \right)-S\left(\rho \right)\le lnN-S\left(\rho \right).$$

(131)

7.2. Partial Trace Based Diffusion

Recall that the partial trace process can be viewed as a nonlinear process ${\rho }_{AB}\to \mathcal{N}{\rho }_{AB}={\rho }_A\otimes {\rho }_B$. The nonlinear operator $\mathcal{N}$ is idempotent, ${\mathcal{N}}^2=\mathcal{N}$. Now, consider:

$$\rho \to {\rho }_t=e^{-t}{e}^{t\mathcal{N}}\rho =e^{-t}\{\rho +(e^t-1)\mathcal{N}\rho \}=e^{-t}\rho +(1-e^{-t})\mathcal{N}\rho .$$

(132)

More explicitly:

$${\rho }_t=e^{-t}{e}^{tN}{\rho }_{AB}=e^{-t}{\rho }_{AB}+(1-e^{-t}){\rho }_A\otimes {\rho }_B=(1-s){\rho }_{AB}+s{\rho }_A\otimes {\rho }_B.$$

(133)

Here, we have set $s=1-e^{-t}\in [0,1]$ as $t\in [0,\infty )$. That is, a diffusion-like process based on the nonlinear operator $\mathcal{N}$ recovers the process of quantum coarse graining by tuning the partial traces that we had previously considered; see Equation (71). That is, the compound object:

$$e^{-t}{e}^{t\mathcal{N}}=exp\left(t[\mathcal{N}-I_{N\otimes N}]\right)=e^{-t}{I}_{N\otimes N}+(1-e^{-t})\mathcal{N};$$

(134)

while it represents a nonlinear process when acting on density matrices, nevertheless it has formal similarities to a diffusion process. As usual, we can define the hidden information as:

$$I_{\mathrm{correlations}}=S\left(e^{t\{\mathcal{N}-I_{N\otimes N}\}}\rho \right)-S\left(\rho \right)\ge 0,$$

(135)

which is guaranteed positive and monotonic in the coarse graining parameter t. Again, the t parameter may or may not represent physical time; we could view the entire construction as a gedanken-process in which case t would just be a tuning parameter giving explicit control of the extent of coarse graining. The only generic upper bound on the hidden information in this context seems to be:

$$I_{\mathrm{correlations}}=S\left(e^{t\{\mathcal{N}-I_{N\otimes N}\}}\rho \right)-S\left(\rho \right)\le lnN-S\left(\rho \right).$$

(136)

8. Discussion

We see that we can model the coarse graining procedure, in a quantifiable and controllable manner, by following different methods depending on what aspect of the system one is taking into account. Classically, when considering the Shannon entropy, we have studied coarse graining allowing the system change (or not) between a continuum and discretium character and also differentiating situations where we introduce a tunable smearing parameter to track the change from continuum to discretium. We have studied coarse graining that involves diffusion, density-density correlations in a box (depending on a smearing parameter or not) and correlation between the states of two boxes, obtaining in every cases a controllable and monotonic flux of entropy.

On the other hand, we have considered the quantum von Neumann entropy, coarse-grained by maximal mixing or by partial traces, tunable or not, asymmetric in the treatment and also combined with maximal mixing. That corresponds to blurring process depending on a smearing parameter and the choice of separate subsystems that “decide” not to look at each other. In all cases, we have found a positivity of entropy flow and a generic upper bound.

We have also studied the coarse graining induced by decoherence, either partial or complete, allowing us to either remove all off-diagonal terms or control the extent to which we partially suppressed them. We have obtained also a consistent definition of the hidden information and an upper bound for it, one that corresponds (as it should) to complete decoherence.

We have introduced an analogy with the Hawking’s super-scattering operator that allows us to define a generic class of quantum coarse grainings. Finally, we have developed a notion of diffusion in Hilbert space, defined by means of super-scattering or partial traces, again finding a consistent and controlled definition of the entropy flow.

It is important to note that all of these models give an explicit and calculable idea of how the entropy flow increases and can be controlled when we coarse grain a system. These models can be useful tools in order to study some physical processes concerning information issues; such as the well-known information puzzle in black hole physics [21,22,23,24,25,26,27,28,29], the ontological status of Bekenstein’s horizon entropy [51,52,53,54,55,56] and the very existence of horizons themselves [57,58,59].