On Equivalence of Nonequilibrium Thermodynamic and Statistical Entropies

Abstract

We review the concept of nonequilibrium thermodynamic entropy and observables and internal variables as state variables, introduced recently by us, and provide a simple first principle derivation of additive statistical entropy, applicable to all nonequilibrium states by treating thermodynamics as an experimental science. We establish their numerical equivalence in several cases, which includes the most important case when the thermodynamic entropy is a state function. We discuss various interesting aspects of the two entropies and show that the number of microstates in the Boltzmann entropy includes all possible microstates of non-zero probabilities even if the system is trapped in a disjoint component of the microstate space. We show that negative thermodynamic entropy can appear from nonnegative statistical entropy.

1. Introduction

Although the concept of entropy plays important roles in diverse fields ranging from classical thermodynamics of Clausius [1–9], statistical thermodynamics of Boltzmann and Gibbs [4,10–13], dissipation theorems and stochastic thermodynamics [14–16], quantum mechanics and uncertainty [17–19], quantum Carnot engine [20–22], black holes [23,24], and coding and computation [25–27], to information technology [28–30], it does not seem to have a standard definition; see for example [30–34]. Here, we are interested in its application to nonequilibrium classical thermodynamics [1–7,9] and its statistical formulation for a macroscopic system shown in Figure 1 as developed recently by us [35–39]. The system is contained within a clearly defined boundary (real or imaginary); the latter specifies a volume V associated with the system, which may change in time. In thermodynamics, we also need other physical quantities such as the energy E, number of particles N, etc. to specify its (thermodynamic) state. As is the case in classical thermodynamics, these quantities are taken to be extensive1. For this we require the interactions to be tempered (such as screened Coulomb forces) 2 and the system to be stable [40] (pp. 320–321) and references therein. In the words of Lieb and Lebowitz, “That thermodynamics is applicable to systems with Coulomb forces is a fact of common experience, …. ” They also establish that the thermodynamic properties in most cases of interest, including Coulombic interactions, are not sensitive to the shape of the boundary of the system. We are mostly interested in nonequilibrium states (NEQSs) of a macroscopic system; we will not always identify states as nonequilibrium from now on unless clarity is needed to distinguish it from equilibrium states (EQSs).

Without a proper identification of a NEQS, there is no hope of identifying its entropy. In the absence of a precisely defined nonequilibrium entropy, the law of increase of entropy S₀ as formulated by Clausius [1] for an isolated system, see Σ₀ in Figure 1a,b, is also deprived of its power as the ultimate law of Nature according to which (see also Section 3)

$$d{S}_0/dt\ge 0.$$

(1)

For an interacting system, such as Σ in Figure 1b, the law of increase of entropy reduces to the law of decrease of a thermodynamic potential such as the Gibbs free energy [4] with time; see also Section 3.9. It is known that the entropy for an EQS in classical thermodynamics is a thermodynamic quantity without any mechanical analog as opposed to mechanical quantities such as E, V, etc. that are also defined for a mechanical system. It can be determined calorimetrically by integrating the heat capacity and following rules of equilibrium thermodynamics. There is some doubt whether nonequilibrium thermodynamic entropy has any meaning in classical thermodynamics, despite the fact that its existence follows directly from Equation (1). Indeed, classical thermodynamics is not restricted to the study of EQSs only [1,4,6] since the first law of thermodynamics applies equally well to all (equilibrium and nonequilibrium) processes. To see how nonequilibrium entropy appears, consider a classic example, treated in almost all standard textbooks, in which two bodies at different temperatures in thermal contact form an isolated system. The latter is obviously out of equilibrium, but its entropy S₀ is uniquely specified as the sum of the entropies of the two bodies [4] (p. 35); see footnote 1. The entropy must have explicit time dependence as seen in Equation (1). The example is invariably used to prove that the energy flows from a hotter to a colder body.

We quote Landau and Lifshitz [4] (p. 29), which leaves no doubt that the entropy in the second law in classical thermodynamics applies equally well to nonequilibrium states:

“… if a closed system is at some instant in a non-equilibrium macroscopic state, the most probable consequence at later instants is a steady increase in the entropy of the system. This is the law of increase of entropy or second law of thermodynamics, discovered by R. Clausius (1865); its statistical explanation was given by L. Boltzmann in the 1870s.”

We will, therefore, follow Clausius and take the view here that the thermodynamic entropy is a well-defined notion for any irreversible process going on within a system for which he writes [1](p. 214).

$$T_{0}dS>d_{\mathrm{e}}Q;$$

(2)

here we have used the notation introduced in Figure 1: d_eQ is the exchange heat (traditionally written as dQ) from the medium that is added to the system and T₀ is the temperature of the medium.3,4 Despite the inequality, the entropy change in an irreversible cyclic process in which the entropy has the same value as the initial value after the completion of the cycle, we have

$${\displaystyle \oint dS=0}$$

(3)

in terms of entropy change dS for intermediate nonequilibrium states [6] (Equation (3.3.10)). As we are dealing with an inequality, the evaluation of S for nonequilibrium processes is not so straightforward because of the irreversible contribution d_iS, which cannot be directly measured as it occurs within the body. On the other hand, d_eQ is directly measurable. Let us focus on an interacting system in a medium at temperature T₀ and pressure P₀. In terms of the exchange work d_eW = P₀dV (traditionally written as dW) against P₀, the first law which is merely a statement of the conservation of energy is written as dE ≡ d_eE = d_eQ − d_eW in terms of exchange heat and work5. Applying the same first law to internal processes for which d_iE = d_iQ − d_iW ≡ 0, we have

$$d_{\mathrm{i}}Q=d_{\mathrm{i}}W,$$

(4)

which is always valid. Combining the two formulations, we have dE ≡ dQ − dW in terms of dQ = d_eQ + d_iQ and dW = d_eW + d_iW. The internal work is d_iW = (P − P₀)dV = d_iQ > 0 in terms of the pressure P ≠ P₀ of the system so that dW = PdV. The irreversible entropy $d_{\mathrm{i}}^{(\mathrm{Q})}S$ due to heat exchange and $d_{\mathrm{i}}^{(\mathcal{W})}S$ due to work exchange are given by $T{d}_{\mathrm{i}}^{(\mathrm{Q})}S=(T_0-T)d_{\mathrm{e}}S$ and $T{d}_{\mathrm{i}}^{(\mathcal{W})}S=(P-P_0)dV$, respectively, [6,35]. Adding them together, we have

$$T{d}_{\mathrm{i}}S=(T_0-T)d_{\mathrm{e}}S+(P-P_0)dV=d_{\mathrm{e}}Q-T{d}_{\mathrm{e}}S+d_{\mathrm{i}}W,$$

(5)

which using Equation (4) can be written as an extremely useful identity in the form of an equality [35–39]

$$dQ(t)=T(t)dS(t)$$

(6)

valid whenever T and P for the system can be defined; see caption for Figure 1. As we will see later in Section 3.6, it a very general result and replaces the Clausius inequality in Equation (2). The equality also shows that it is dS that determines the new heat quantity dQ even though knowing d_iS is still important as its non-zero value indicates the presence of irreversibility in the system. The new concept of work dW in our formulation now represents the isentropic change in dE, while dQ is determined by the change dS so that the splitting of dE into dQ and dW only involves the variables of the system. In contrast, dE ≡ d_eQ − d_eW involves variables of the medium.

In our opinion, it is the notion of entropy as a thermodynamic quantity that differentiates a thermodynamic system from a purely mechanical system. This then explains the motivation for this review. As S is the central and the new concept, determining dS is the goal in thermodynamics; see Equation (6). There are thus two possible approaches. In one of these approaches, we develop means to determine d_iS so that dS can be computed by supplementing the directly measured d_eS. This is how the traditional nonequilibrium thermodynamics has developed historically, ; see for example [2,3]. The other approach requires directly dealing with dS; in this approach, the determination of d_iS is no longer central, although it can be determined using dS − d_eS. In this approach, it is important to identify the nonequilibrium entropy so that we can pursue its physical significance.6 There are several possible definitions of entropy in the literature, not all of them equivalent [5,9,30–33,38,39,41–49] to name a few7. The entropy in our view is a macroscopic concept, for reasons to be given later in Section 3.8, and does not require any association with system’s microstates and their probabilities, while modern approach in statistical thermodynamics, primarily due to Boltzmann and Gibbs, requires a probabilistic approach in terms of microstates and provides a microscopic interpretation of entropy. Because of its microscopic connection, the statistical interpretation of entropy provides a more detailed description of a thermodynamic system. However, it is not obvious that there is any connection between entropies in the two approaches under all conditions. For example, the connection between heat or the “jiggling motion of particles” [50] and the statistical notion of entropy is not obvious at all; see however Ref. [38,39]. For this reason, we will carefully distinguish the two kinds of entropies, and call the former the thermodynamic entropy and the latter the statistical entropy in this review. We will briefly review these concepts in the next section for the benefit of the reader. For an illuminating discussion of equilibrium thermodynamic entropy, we refer the reader to an excellent review by Lieb and Yngvason [31]8 and to Zanchini and Beretta [51].

The viewpoint we take here is the following. As the state of a nonequilibrium system continuously changes with time, one cannot make repeated measurements at different times on the same system to obtain average properties of the system at a given moment. In equilibrium, this poses no problem as properties are time independent. The only possibility for nonequilibrium states is to consider measurements on independent samples (also called replicas) of the system at the same time t under identical macroscopic conditions. With this in mind, we have have recently proposed the Fundamental Axiom, see Section 3.3 in [52], which we produce below with slight modification for completeness:

Fundamental Axiom The thermodynamic behavior of a body is not the behavior of a single sample, but the average behavior of a large number of independent samples, prepared identically under the same macroscopic conditions at time t.

The axiom represents the fact that thermodynamic is an experimental science so that several measurements performed at the same time t on the samples are required to obtain reliable average results at t. Thus, dynamics is not taken into consideration in identifying the instantaneous quantities such as the entropy. The samples are completely independent of each other so that different measurements do not influence each other. A sample represents one of the instantaneous state (microstates) of the body, and the collection of these instantaneous states change with time for a body out of equilibrium. The average of any thermodynamic or mechanical quantity over the collection of these samples then determines the thermodynamic property of the system at that instant. The sample approach will play a central role in our statistical formalism.

We consider thermodynamic entropy S to be the physical entropy of the body alone, even though we recognize that it can only be determined experimentally using calorimetry for equilibrium states and possibly with some effort for states in internal equilibrium introduced previously [35–37] and to be explained below in Section 3.4. The entropy S, being an intrinsic property of the system alone, depends explicitly on the current state of the body but not that of the medium. It must possess certain properties; the most important for this review are the following:

• SP1 S exists as a unique quantity for all states of the system at each instant t; consequently, it does not depend on the state of the medium;
• SP2 S is extensive, continuous and at least twice differentiable;
• SP3 S is additive over quasi-independent subsystems;
• SP4 S satisfies the second law; and
• SP5 S is nonnegative.

The above requirements must be satisfied for all possible states. The first requirement asserts, following Clausius, the existence of a unique quantity called the entropy S for any state and which may depend explicitly on time t. The explicit dependence on t, which is never considered a state variable, implies that S is not always a state function (of state variables); see Section 3.1. The entropy also cannot explicitly depend on the current state of the medium, although it implicitly depends through the system’s state variables that are affected by the medium. The second requirement is in accordance with classical thermodynamics, which the current approach is an extension of [31]. The third requirement is to ensure the additivity property of S [31], and follows from quasi-independence of subsystems9 that act almost independently; this will be further explained in Section 3.8, but also see footnote 1. The fourth requirement is to ensure the fulfillment of the second law. The last requirement follows from the third law for equilibrium states since the entropy of a NEQS cannot be less than that of EQS, and the latter is determined by the third law to vanish at absolute zero or be nonnegative. The requirement is also consistent with the existence of the residual entropy as we will discuss later. A careful reader will have noticed that no convexity restrictions on the entropy are imposed to allow for unstable nonequilibrium states.

Any statistical notion of entropy to be acceptable must reproduce the physical entropy in all cases and for all kinds of systems including glasses that are ubiquitous and are a prime example of nonequilibrium states. There are several statistical quantities in the literature that have been identified as candidates for entropy; for example, see [30,33] and Section 2 where we briefly review these quantities. The fact that these statistical entropies also exhibit a nondecreasing tendency is by itself does not prove that these entropies are equal to the thermodynamic entropy in general. To identify one of these with thermodynamic entropy requires showing their numerical equivalence in all cases. Demonstrating this equivalence or its absence beyond equilibrium has motivated this review. Here, we give a general derivation of the statistical entropy, to be denoted by S, and show their equivalence when S is a state function of nonequilibrium state variables to be introduced below and a few other cases of importance to be explained later.

It is well known that the thermodynamic entropy S is not different from the statistical entropies proposed by Boltzmann and Gibbs for a system in equilibrium.10 However, there is at present no consensus about their equivalence when the system is out of equilibrium. Indeed, not everyone agrees that they can be applied to a body out of equilibrium. We find the answer to be partially affirmative. The statistical entropy S formally appears to be a generalization of the Gibbs formulation and contains the generalization of the Boltzmann formulation under an important condition to be specified later, and which is satisfied in most cases of interest. Thus, in this case, they give the same results; in general, they give different results. This is important as the literature is not very clear on this issue [42,43,53–56]. We then extend the result to other cases. However, there are some hurdles that we encounter in achieving the goal. Our first hurdle is about measuring or computing S for nonequilibrium states. We have shown earlier [35–39] that the thermodynamic entropy can be determined or measured for a special class of nonequilibrium states, which we have recently introduced as being in internal equilibrium. These states are of considerable interest in this work and are revisited in Section 3.4. We immediately meet with another serious hurdle. In classical thermodynamics, S is known to become negative such as for an ideal gas, even though the Boltzmann or Gibbs statistical entropy is non-negative. We wish to understand the origin of the negative entropy. Using a non-trivial example, we demonstrate how it emerges from a non-negative statistical entropy. The third hurdle we encounter is with Boltzmann’s concept of the number of microstates when dealing with systems such as glasses [57,58] where we encounter component confinement over a long period of time; see footnote 10 and Section 7. Component confinement occurs at low temperatures where equilibration time τ_eq becomes larger than the observation or measurement time τ_obs. This was not an issue for Boltzmann who was primarily interested in the kinetic theory of gases for which τ_eq << τ_obs. Does the number of microstates refers to the microstates that the system may be able to dynamically explore in accordance with its equations of motion over the time period τ_obs with non-zero probabilities or refers to all microstates consistent with the macrostate regardless of the dynamics? In the former scenario, we need to consider only the microstates restricted to one of the components in which the system finds itself trapped, while in the latter scenario, all microstates in all components will have to be considered. Evidently, the two scenarios will yield widely different results in general. To the best of our knowledge, this issue has not received much attention, which we hope to settle here. In this work, we will take our cue from the behavior of the thermodynamic entropy in vitrification and the resulting residual entropy to settle this important issue. The result is summarized in Claim 6 in Section 7.

A system may be isolated, see Σ₀ in Figure 1a or may be surrounded by a medium, see Σ surrounded by the medium $\tilde{\mathrm{\Sigma }}$ in Figure 1b. In the latter case, we will call the system to be interacting (with the medium). For us, the medium represents a heat source, a working source, a particle source, etc. or a combination thereof. We will take it to be extensively large compared to the system Σ, which itself is macroscopically large. We will refer to a system in both situations as a body in this work. As we will consider both kinds of systems, it is advantageous to introduce some notations. Quantities for the isolated system will carry a suffix 0 while those for Σ will carry no suffix. All extensive quantities for the medium $\tilde{\mathrm{\Sigma }}$ will carry a tilde. Because of the excessive size of the medium (see also the caption for Figure 1), there is no need to distinguish its fields from those of Σ₀. Accordingly, we will use the suffix 0 and not a tilde to denote the fields of $\tilde{\mathrm{\Sigma }}$. Quantities without the suffix will refer to a body, whether isolated or interacting. The body will always be denoted by Σ.

The extensive observables11 the energy E₀, the volume V₀, the number of particles N₀, etc. for Σ₀ must remain constant even if the system is out of equilibrium.12 While this will simplify our discussion to some extent, it will also create a problem as we will discuss later in Section 3.1 and force us to consider internal variables for a complete description. We briefly review the notion of a thermodynamic state or simply a state and entropy in equilibrium thermodynamics and several statistical entropies in the next section. We discuss the concept of a NEQS, state variables and state functions in Section 3 using the approach developed recently by us [35–39]. We briefly review the vitrification process and the residual entropy in Section 4. We introduce the statistical entropy formulation in Sections 5–7 where we also show its equivalence with thermodynamic nonequilibrium entropy when the latter is a state function and in some other specific cases. This allows us to make Conjecture 1 about their equivalence in all cases in Section 6. In Section 8, we briefly review the semi-classical approximation and show that one of the standard formulation of nonequilibrium entropy (cf. Equation (22)) does not really measure the entropy. In Section 9, we consider a 1-d lattice model appropriate for Tonks gas in continuum and take the continuum limit to contrast the behavior of the lattice and continuum entropies and to shed light on how a negative entropy emerges. The last section contains a brief summary and some important discussion.

2. Brief Review

2.1. Equilibrium Thermodynamic Entropy

In equilibrium thermodynamics, a body is specified by the set X formed by its independent extensive observables (E,V,N,etc.); the set also serves the purpose of specifying the thermodynamic state (also known as the macrostate) $ℳ$(X) of the system. It is implicit that the body has a well-defined boundary ∂V that specifies V but is not part of X as we are not interested in surface thermodynamics. The equilibrium entropy S_eq(X) is a state functions and is supposed to be at least twice differentiable except possibly at phase transitions, which we will not consider in this review. It satisfies the Gibbs fundamental relation

$$d{S}_{\mathrm{eq}}(\mathbf{X})\equiv (\partial S_{\mathrm{eq}}/\partial \mathbf{X})\cdot d\mathbf{X}\equiv {\displaystyle {\sum }_p(\partial S_{\mathrm{eq}}\partial X_p)d{X}_p=(dE+PdV-\mu dN+\dots )/T,}$$

(7)

where we have shown only the terms related to E, V and N in the last equation. The missing terms refer to the remaining variables in X ≡ {X_p}, and T, P, μ, etc. have their standard meaning in equilibrium

$$(\partial S_{\mathrm{eq}}/\partial E)=1/T,(\partial S_{\mathrm{eq}}/\partial V)=P/T,(\partial S_{\mathrm{eq}}/\partial N)=-\mu /T,\cdots .$$

(8)

Let us consider a thermodynamic process ${\mathcal{P}}_{12}$ taking the system from an EQS $ℳ$_eq1(X₁) to another EQS $ℳ$_eq2(X₂), shown as 1 and 2 in Figure 2, respectively; here X is taken to contain only two elements X₁ and X₂ for simplicity. The process, which is shown by a red curve connecting $ℳ$_eq1, $ℳ$_eq2 in Figure 2, does not have to be an equilibrium process. The X₁-X₂ plane corresponding to ξ = 0 contains all EQSs of the system, shown schematically by the region enclosed by the blue curve in that plane. As explained later, the ξ-axis with ξ ≠ 0 is used schematically here to denote out-of-equilibrium states. The dotted curve γ₁₂ in the equilibrium plane ξ = 0 denotes one of many possible equilibrium processes connecting the states 1 and 2: all states on γ₁₂ are EQSs for which Equation (7) holds. The change

$$\mathrm{\Delta }{S}_{\mathrm{eq}}({ℳ}_{\mathrm{eq}1}\to {ℳ}_{\mathrm{eq}2})\equiv S_{\mathrm{eq}}({\mathbf{X}}_2)-S_{\mathrm{eq}}({\mathbf{X}}_1)\equiv {\displaystyle {\int }_{\gamma 12}d{S}_{\mathrm{eq}}(\mathbf{X})}$$

(9)

in S in the process ${\mathcal{P}}_{12}$ is independent of the actual process, but depends only on the end macrostates. Numerically, it can be determined by integrating the Gibbs fundamental relation along γ₁₂. This is the calorimetric evaluation of dS_eq(X) between two equilibrium states. The result is independent of all possible paths γ₁₂ or ${\mathcal{P}}_{12}$; moreover, these paths may have nothing to do with the actual process ${\mathcal{P}}_{12}$.

As an example, consider a process at constant V and N. Then, ΔS_eq is obtained calorimetrically by integrating the heat capacity C_V,N = (∂E/∂T)

$$\mathrm{\Delta }{S}_{\mathrm{eq}}({ℳ}_{\mathrm{eq}1}\to {ℳ}_{\mathrm{eq}2})={\displaystyle {\int }_{{\mathrm{T}}_1}^{{\mathrm{T}}_2}\frac{{\mathcal{C}}_{V,N}}{T}dT,}$$

(10)

where we have assumed in the last equation that there is no discontinuous phase transition as the system moves from X₁ to X₂ along γ₁₂, and where T₁ and T₂ are the equilibrium temperatures associated with $ℳ$_eq1 and $ℳ$_eq2, respectively. If the process ${\mathcal{P}}_{12}$ is irreversible, then during this process there will be irreversible entropy generation, see footnote 3,

$${\mathrm{\Delta }}_{\mathrm{i}}{S}_{{\mathcal{P}}_{12}}={\displaystyle {\int }_{{\mathcal{P}}_{12}}{d}_{\mathrm{i}}S>0,}$$

(11)

which cannot be measured. The reversible entropy change

$${\mathrm{\Delta }}_{\mathrm{e}}{S}_{{\mathcal{P}}_{12}}=\mathrm{\Delta }{S}_{\mathrm{eq}}-{\mathrm{\Delta }}_{\mathrm{i}}{S}_{{\mathcal{P}}_{12}}={\displaystyle {\int }_{{\mathcal{P}}_{12}}{d}_{\mathrm{e}}S={\displaystyle {\int }_{{\mathcal{P}}_{12}}\frac{d_{\mathrm{e}}Q}{T_0}}}$$

(12)

is the entropy exchange with the medium during ${\mathcal{P}}_{12}$ and can be of any sign. The values of ${\mathrm{\Delta }}_{\mathrm{e}}{S}_{{\mathcal{P}}_{12}}$ and ${\mathrm{\Delta }}_{\mathrm{i}}{S}_{{\mathcal{P}}_{12}}$, but not of ΔS_eq, depend on the process ${\mathcal{P}}_{12}$. We now see the origin of the Clausius inequality

$$\mathrm{\Delta }{S}_{\mathrm{eq}}({ℳ}_{\mathrm{eq}1}\to {ℳ}_{\mathrm{eq}2})\ge {\displaystyle {\int }_{{\mathcal{P}}_{12}}\frac{d_{\mathrm{e}}Q}{T_0};}$$

(13)

see Equation (2). For a reversible process (d_iS = 0) Δ_eS = ΔS_eq($ℳ$_eq1 → $ℳ$_eq2). In this case, Δ_eS is the same for all reversible processes $ℳ$_eq1 → $ℳ$_eq2.

The choice of the set X and the number of elements in it depend on the level and kind of description of the body [13]. For example, if the magnetization is not part of the description, and we only use E, V and N to specify the macrostate, then the entropy will be expressed as S_eq(E, V, N). If on the other hand magnetization M is also required to specify the macrostate, the equilibrium entropy S_eq(E, V, N, M) will be different from S_eq(E, V, N). In order to obtain the latter, we must remove a constraint (of having a given magnetization M) on the system. Hence,

$$S_{\mathrm{eq}}(E,V,N)\ge S_{\mathrm{eq}}(E,V,N,\mathrm{M}).$$

(14)

The above discussion is equally valid for an interacting system or an isolated system. For the latter, we need to append a subscript 0 to all the quantities. In this case, the equilibrium entropy S_0eq is a state function of the observables E₀, V₀, N₀, etc. The corresponding macrostate $ℳ$_0eq is shown in Figure 2 in the ξ = 0 plane. The functional dependence results in the Gibbs fundamental relation

$$d{S}_{0\mathrm{eq}}={\displaystyle {\sum }_p(\partial S_{0\mathrm{eq}}/\partial X_{0p})d{X}_{0p}}$$

(15)

in terms of the observables {X_0p}, with the partial derivatives related to the equilibrium fields of the system:

$$(\partial S_0/\partial E_0)=1/T_0,(\partial S_0/\partial V_0)=P_0/T_0,\cdots$$

(16)

However, as the observables remain constant (dX_0p ≡ 0) for the isolated system, we have

$$d{S}_{0\mathrm{eq}}\equiv 0.$$

(17)

Thus, the equilibrium entropy of the isolated system Σ₀ cannot change.

2.2. Equilibrium Statistical Entropies

The statistical approach is used to provide a more detailed description of a body in thermodynamics at the microscopic level. In this approach, a body is represented by a collection Γ of microstates α, each with its own probability p_α. A microstate represents the instantaneous state of the body at the microscopic level and evolves in time following equations of motion. The macrostate $ℳ$ is treated as representing the collection of those microstates and their probabilities. All microstates are characterized by the same set of observables X; some of the observables may be constants. The set of these microstates is denoted by Γ(X). While temporal evolution is not our primary interest in this work, we still need to remember the importance of temporal evolution in thermodynamics. We will say that two microstates belonging to the microstate subspace Γ(X) are “connected” (in time) if one evolves from the other after some time τ < ∞ with a non-zero probability. Before this time, they will be treated as “disconnected” (in time). A component is a collection of all microstates that are connected with each other but with no connection with any microstate outside the collection. This makes all components disjoint from each other. Let τ_max denote the maximum τ over all pairs of microstates. The space Γ(X) becomes a single component for all times longer than τ_max in that each microstate can evolve into another microstate ∈ Γ(X) in due time with non-zero probability. For t < τ_max, the space Γ(X) will consist of disjoint components, an issue that neither Boltzmann nor Gibbs considered to the best of our knowledge. However, the issue, which we consider later in Sections 4 and 7, becomes important for nonequilibrium states.

Let us consider a body Σ having a single component Γ(X) and assume equal probability of various microstates following Boltzmann. It is clear that τ_max ≤ τ_eq. Boltzmann identifies the statistical entropy in terms of the number of microstates [4,60–62] W (X) in Γ (X) as follows13

$$S_{\mathrm{B}}(\mathrm{X})\equiv \ln W(\mathrm{X})\ge 0;$$

(18)

we will set the Boltzmann constant to be unity throughout the work so that the entropy will always be a pure number. It is easy to show that the above entropy is no different than the equilibrium thermodynamic entropy. To show this, consider the body at inverse temperature β ≡ 1/T = 1/T₀ in contact with a medium at T₀. The corresponding canonical partition function is given by

$$Z(\beta )={\displaystyle {\sum }_{E}W(E)e^{-\beta E}={\displaystyle {\sum }_E{e}^{-\beta (E-T{S}_{\mathrm{B}})},}}$$

where we do not exhibit fixed observables V, N, etc. Introducing F_B(T) = −T ln Z(β), we see immediately that the average energy $\overline{E}=-\partial (\beta F_{\mathrm{B}})/\partial \beta$ and $\partial F_{\mathrm{B}}/\partial T=\beta (F_{\mathrm{B}}-\overline{E})=-S_{\mathrm{B}}(T)$. We have used here the identity $F_{\mathrm{B}}(T)=\overline{E}-T{S}_{\mathrm{B}}(\overline{E})$ for a macroscopic system, obtained by replacing the sum by the dominant term. Let the dominant term occur for E = E_d, which must be a function of T, so that F_B(T) = E_d(T) − TS_B(T), where we have written S_B(T) for S_B(E_d). We immediately observe that $\overline{E}=E_{\mathrm{d}}$. Furthermore, calculating explicitly the derivative ∂F_B/∂T and equating it to −S_B(T), we conclude that $\partial \overline{E}/\partial T=T\partial S_{\mathrm{B}}/\partial T$. Thus,

$$S_{\mathrm{B}}(T)=S_{\mathrm{B}}(T=0)+{\displaystyle {\int }_0^T(1/T)\partial \overline{E}/\partial TdT.}$$

(19)

We now turn to the thermodynamic definition of the equilibrium Helmholtz free energy F_eq(T) = E(T) − TS_eq(T) for the same system; here we have written S_eq(T) for S_eq(E(T)). In order to be able to relate F_eq(T) and F_B(T), we require that $E(T)=\overline{E}$. Recognizing that $\mathcal{C}\equiv \partial E/\partial T\equiv \partial \overline{E}/\partial T$ is the heat capacity of the system, we see from Equations (10) and (19) that S_B(T) = S_eq(T), provided we ensure that S_B(T = 0) = S_eq(T = 0) ≥ 0; see SP4. This also results in the equality F_eq(T) = F_B(T). The above derivation can be easily extended to other equilibrium ensembles and to an isolated system, but the equality works only for macroscopic bodies. Thus, S_B = S_eq holds in general for S_B in Equation (18).

Gibbs, also using the probabilistic approach, gives the following formula for the entropy in a canonical ensemble [4,10]14

$$S_{\mathrm{G}}\equiv -{\displaystyle \sum _{\alpha \in {\mathrm{\Gamma }}_{\mathrm{E}}}{p}_{\alpha }\ln p_{\alpha };}{\displaystyle \sum _{\alpha \in {\mathrm{\Gamma }}_{\mathrm{E}}}{p}_{\alpha }=1,}$$

(20)

where p_α > 0 is the probability of the αth microstate of the system, and the sum is over all microstates corresponding to all possible energies E (with other elements in X held fixed); their set is denoted by Γ_E in the above sum.15 We refer the reader to some of the reviews [11,30,52,54–56] for a comparison between the Boltzmann and Gibbs formulations. From p_α = exp[−β(E_α − F_eq)], we obtain $S_{\mathrm{G}}(T)=\beta (\overline{E}-F_{\mathrm{eq}})$. Using the same arguments as above, we easily establish that S_G(T) = S_eq(T) = S_B(T), a result that can be extended to other ensembles. In particular, in the microcanonical ensemble, S_G = S_B if we assume p = 1/W (X) for each microstate.

It should be noted, see footnote 13, that Boltzmann [61] actually provides the following expression for the entropy [61,65] in terms of a single particle probability $p_i^{(1)}$ for the particle to be in the ith state:

$$S_{\mathrm{B}}^{(1)}=-N{\displaystyle {\sum }_i{p}_i^{(1)}\ln p_i^{(1)},}$$

(21)

instead of S_B in Equation (18). While $S_{\mathrm{B}}^{(1)}$ has the formal appearance of S_G, the two are different. We should also not confuse $S_{\mathrm{B}}^{(1)}$ with S_B, even when $p_i^{(1)}$’s are equally probable: $p_i^{(1)}=1/w$, where w is the number of possible states of a single particle. In this case, $S_{\mathrm{B},\max }^{(1)}=N\ln w$. In general, particles are not independent due to the presence of inter-particle interactions and the number of possible states W < w^N. Accordingly, the maximum Gibbs entropy S_G,max is less than the corresponding equiprobable entropy $S_{\mathrm{B},\max }^{(1)}$. However, Jaynes [64] gives a much stronger result $S<S_{\mathrm{B}}^{(1)}$, see his Equation (5). The equality occurs if and only if there are no interactions between the particles. We will no longer consider $S_{\mathrm{B}}^{(1)}$. There is another statistical formulation of entropy, heavily used in the literature, in terms of the phase space distribution function f(x, t), which follows from Boltzmann’s celebrated H-theorem and is given as:

$$S_f(t)=-{\displaystyle \int f(x,t)\ln f(x,t)dx;}$$

(22)

here, x denotes a point in the phase space. This quantity is not only not dimensionless but, as we will show later, is not the correct formulation in general; see Equation (72a). It should be noted that the H-theorem uses a single particle distribution function and not the phase space distribution function f.

In quantum mechanics, the entropy is given by the von Neumann entropy formulation [17,18] in terms of the density matrix ρ:

$$S_{\mathrm{vN}}=-Tr(\rho \ln \rho ).$$

(23)

The entropy formulation S_I in the information theory [28,29] has a form identical to that of S_G even though the temperature and other fields, and internal variables (see the next section) have no significance in the information theory. Therefore, we will no longer consider S_I.

3. The Second Law and Nonequilibrium States

3.1. The Second Law, non-State Entropy and Internal Variables

The second law due to Clausius [1] states that d_iS in any infinitesimal physical process satisfies

$$d_{\mathrm{i}}S\ge 0;$$

(24)

the equality occurs for a reversible process and the inequality for an irreversible or spontaneous process. For an isolated system Σ₀, we have d_eS₀ = 0 so that dS₀ (= d_eS₀ + d_iS₀) satisfies

$$d{S}_0=d_{\mathrm{i}}{S}_0\ge 0.$$

(25)

The law refers to the thermodynamic entropy. It is not a state function in the conventional sense as we cannot express it as S₀(X₀) of X₀ when Σ₀ is not in equilibrium. The reason is simple: the mechanical observables E₀,V₀,N₀, etc. forming X₀ must remain constant even if Σ₀ is out of equilibrium. This results in a constant S₀(X₀). To allow for a changing entropy, we must write it as S₀(X₀, t) as a non-state function with an explicit time-dependence. The explicit time dependence shows that X₀ is not sufficient to specify the macrostate of Σ₀. The situation is schematically shown in Figure 2, where $ℳ$₀ represents a nonequilibrium macrostate of Σ₀ at time t = 0 with ξ ≠ 0. We have assumed here that only one additional independent variable ξ, known as the internal variable (also called a hidden variable or a progress variable), is required along with X₀ to specify $ℳ$₀; the latter evolves in time at constant X₀ along the vertical red arrow until it equilibrates to $ℳ$_0eq for t ≥ τ_0eq, the equilibration time, during which $ℳ$₀(t) ≡ $ℳ$₀ (ξ(t)| X₀) continues to change as ξ(t) changes towards its equilibrium value ξ_eq = 0 at t = τ_0eq. Accordingly, $ℳ$₀(t) requires X₀ and ξ for its specification for t < τ_0eq. In general, there will be a set ξ of internal variables. Once, X₀ and ξ are specified, the macrostate $ℳ$₀(t) ≡ $ℳ$₀ (ξ(t)| X₀) is also uniquely specified. In contrast, $ℳ$_0eq requires only X₀, which means that in equilibrium, ξ_eq is no longer independent of X₀. We will again come back to this issue.

The above discussion also applies to the interacting system Σ in Figure 1b for which Equation (24) holds. However, the application of Equation (25) to Σ₀ provides additional information about the behavior of the system Σ in an arbitrary nonequilibrium state (ANEQS), a NEQS with no additional conditions imposed on it. It is briefly reviewed in Section 3.9.

3.2. Number of Required Internal Variables

The choice of how many internal variables for a body are needed will depend on experimental time scales and requires careful consideration of the relaxation behavior of each internal variable. We refer the reader to a very clear discussion of this issue in Ch. 8 of the monogram [9]. The conclusion is that the number of internal variables depends on the experimental time scale for observation. A shorter time scale will require greater number of internal variables. Many of these would come to equilibrium for a longer time scale choice so that we then need a smaller number. This can be understood by considering the equilibration time ${\tau }_{\mathrm{eq}}^{(k)}$ of the kth internal variable ξ_k and comparing with the observation time τ_obs. All internal variables with ${\tau }_{\mathrm{eq}}^{(k)}<{\tau }_{\mathrm{obs}}$ have been equilibrated and need not be considered. On the other hand, all internal variables with ${\tau }_{\mathrm{eq}}^{(k)}>{\tau }_{\mathrm{obs}}$ are out of equilibrium and their temporal variation will result in irreversible entropy generation and must be included in Z(t). We refer the reader to a recent work by Pokrovskii [66] for temporal variation of internal variables.

3.3. Nonequilibrium Processes

Instead of considering the nonequilibrium process ${\mathcal{P}}_{12}$ between two equilibrium macrostates 1 and 2, we now consider a process ${\mathcal{P}}_{12}^{\prime }$ between two NEQSs ${ℳ}_1^{″}$ and ${ℳ}_2^{″}$; the latter are shown by 1′ and 2′ in Figure 2, respectively. The process also includes a process from an EQS to a NEQS or vice versa, not shown explicitly in the figure but is easy to envision, by considering a point on ${\mathcal{P}}_{12}$ as a representative NEQS and a process between this point and 1 or 2. The change $\mathrm{\Delta }{S}_{{\mathcal{P}}_{12}^{\prime }}$ along ${\mathcal{P}}_{12}^{\prime }$ is

$$\mathrm{\Delta }{S}_{{\mathcal{P}}_{12}^{\prime }}\equiv \mathrm{\Delta }S({ℳ}_1\stackrel{{\mathcal{P}}_{12}^{\prime }}{\to }{ℳ}_2)={\displaystyle \underset{{\mathcal{P}}_{12}^{\prime }}{\int }{d}_{\mathrm{e}}S+{\displaystyle \underset{{\mathcal{P}}_{12}^{\prime }}{\int }{d}_{\mathrm{i}}S}.}$$

(26)

The value of ΔS and of each integral depends on the process ${\mathcal{P}}_{12}^{\prime }$. As noted in footnote 3, d_eS = CdT₀/T₀, C ≡ d_eQ/dT₀, and can be used to evaluate the first integral ${\mathrm{\Delta }}_{\mathrm{e}}{S}_{{\mathcal{P}}_{12}^{\prime }}$ along ${\mathcal{P}}_{12}^{\prime }$.16 As far as the second integral ${\mathrm{\Delta }}_{\mathrm{i}}{S}_{{\mathcal{P}}_{12}^{\prime }}$, which is no different in formalism than ${\mathrm{\Delta }}_{\mathrm{i}}{S}_{{\mathcal{P}}_{12}}$ in Equation (11), is concerned, all that we know is that it is positive. Its value can be indirectly determined by subtracting the first integral from $\mathrm{\Delta }{S}_{{\mathcal{P}}_{12}^{\prime }}$:

$${\mathrm{\Delta }}_{\mathrm{i}}{S}_{{\mathcal{P}}_{12}}\equiv \mathrm{\Delta }{S}_{{\mathcal{P}}_{12}^{\prime }}-{\mathrm{\Delta }}_{\mathrm{e}}{S}_{{\mathcal{P}}_{12}^{\prime }}>0.$$

It contains, in addition to the contribution from the irreversible exchanges such as heat transfer with the medium, contributions from all sorts of viscous dissipation going on within the system [2,3,6]. For a process that is conventionally identified as an adiabatic process (con. adia. d_eS ≡ 0, d_eQ ≡ 0) for which

$${\mathrm{\Delta }}_{\mathrm{e}}^{(\mathrm{con}.\mathrm{adia})}{S}_{P_{12}}=0,{\mathrm{\Delta }}_{\mathrm{i}}^{(\mathrm{con}.\mathrm{adia})}{S}_{{\mathcal{P}}_{12}}\equiv \mathrm{\Delta }{S}_{{\mathcal{P}}_{12}^{\prime }}>0,$$

(27)

still requires ξ ≠ 0. However, as the terminal states in the process are NEQSs, we either need to determine $\mathrm{\Delta }{S}_{{\mathcal{P}}_{12}^{\prime }}$ or ${\mathrm{\Delta }}_{\mathrm{i}}{S}_{{\mathcal{P}}_{12}}$ directly. The direct evaluation of $\mathrm{\Delta }{S}_{{\mathcal{P}}_{12}^{\prime }}$ as a difference S($ℳ$₂) − S($ℳ$₁) will require identifying and specifying NEQS $ℳ$ and the entropy as a state function to which we now turn. We will exploit the established practice of using internal variables to describe a non-equilibrium state [3,9,13,36,45] to ensure that S($ℳ$) becomes a state function. Some examples of internal variables include the relative velocities of macroscopically large regions of a system, degree of advancement in chemical reactions, internal parameters of state required during creep or relaxation in a solid under strain or stress, respectively, the area swept out by dislocation lines in plastic materials, the conformation tensor in polymeric fluids, etc.

3.4. Concept of a Nonequilibrium State (NEQS) and of an Internal Equilibrium State (IEQS)

To motivate the use of internal variables, let us again focus on an isolated system Σ₀. If it is in equilibrium, its macrostate $ℳ$_0eq, specified uniquely by the set X₀ of state variables, remains the same unless it is disturbed. Then, as discussed above, its equilibrium entropy S_0eq(X₀) is a state function. This conclusion is most certainly not valid if Σ₀ is out of equilibrium. Its NEQS $ℳ$₀(t) will continuously change, see Figure 2, and as concluded above this will require expressing its entropy as S₀(X₀, t) with an explicit time-dependence. It is postulated in the theory involving internal variables that the changes in the entropy and the macrostate come from the variations of the additional variables ξ₀, independent of the observables, such as ξ in Figure 2 and [3,9,13,36,45].17 The internal variables ξ₀ cannot be manipulated by the observer, although it is possible in some cases to measure them indirectly [9]. Because of this, identifying relevant internal variables becomes difficult in many cases. As they are associated with dissipative processes within the system, a better understanding of the physics behind dissipation helps in identifying these variables in most cases. We denote X₀ and ξ₀(t) collectively by Z₀(t) in the following and refer to Z₀(t) as the set of (nonequilibrium) state variables. For a body, the set of state variables will be denoted by Z(t).

Once the body has come to equilibrium, its entropy S₀(X₀, t) has no explicit time-dependence and becomes a state function of X₀ only. As is well known, the entropy in this state has its maximum possible value for given X₀. This can be summarized in the following claim, which will play an important role below.

Claim 1. When the entropy becomes a state function, it achieves the maximum possible value for the given set of state variables (such as X₀ for an equilibrium state).

The concept of a NEQS must be such that it allows an interacting body to be in the same state at different times such as when it goes through a cyclic process;18 see also [32]. Such a cyclic process is considered in Equation (3). Another possible cyclic process for an interacting system is shown by the combined process ${\mathcal{P}}_{12}^{\prime }$ followed by ${\mathcal{P}}_{21}^{″}$ in which the system starts in 1′ at time t = 0 and after passing through 2′ along ${\mathcal{P}}_{12}^{\prime }$ returns to 1′ along ${\mathcal{P}}_{21}^{″}$ at some arbitrarily chosen later time t = τ_C, the time to complete the cycle. As the body is in the same state 1′ at different times (t = 0 and t = τ_C), we conclude that

Claim 2. For the state to recur, the state must be uniquely characterized by state variables but not time t.

Thus, the specification of the states 1′ but not of 2′ must be unique in terms of variables Z(t) (such as those used in the figure) used to specify it. We say that Z(t) provides a complete or equivalently unique description of the macrostate of the body. It is quite possible that intermediate states such as 2′ require a different set than Z(t) for their unique specification. We can now extend the cycle in Equation (3) to NEQSs. We only require that the cycle originates and returns to the same state such as 1′ above. If it happens that all state variables and functions including the entropy repeat themselves in time after any arbitrary cycle time τ_C, then

$$\mathbf{Z}(t+{\tau }_{\mathcal{C}})=\mathbf{Z}(t),ℳ(t+{\tau }_{\mathcal{C}})=ℳ(t),S(\mathbf{Z}(t+{\tau }_{\mathcal{C}}))=S(\mathbf{Z}(t)).$$

This ensures Δ_cS ≡ S(Z(t + τ_C)) − S(Z(t)) = 0, which then provides the desired extension of Equation (3) to NEQSs. All that is required for Equation (3) for the cyclic process to hold is that the body must start and end in the same NEQS such as 1′ for which the entropy is a state function; however, during the remainder of the cycle such as along ${\mathcal{P}}_{12}^{\prime }$ and ${\mathcal{P}}_{21}^{″}$ in Figure 2, the entropy of the body need not be a state function. It is also possible that Z(t) is not sufficient to uniquely specify the interior states.19

The most general form of the entropy of a body in a NEQS specified by Z(t) can be written as S(t) ≡ S(Z(t), t). In this general form, the entropy is not a state function. From the discussion of a cyclic process, it follows that we can classify NEQSs into two distinct classes. Those NEQSs for which the entropy is a state function S(t) ≡ S(Z(t)) [35–39] play an important role in this review and are called internal equilibrium states (IEQSs). The remaining NEQSs for which the entropy is a non-state function S(t) ≡ S(Z(t), t) with an explicit time dependence have been called arbitrary nonequilibrium states (ANEQSs).

We now extend the notion of IEQSs to an isolated system. Let us consider some IEQS uniquely specified by Z₀(t), i.e., its macrostate $ℳ$₀(t) is uniquely specified by Z₀(t). Its entropy is a state function S₀(Z₀(t)). These states must be homogeneous as we discuss in Section 3.6. The situation is now almost identical to that of an isolated body in equilibrium: The state $ℳ$₀(t) and its entropy are uniquely specified by Z₀(t). The significance of this statement becomes evident from a result proven elsewhere [36] (see Theorem 4 there), which can be paraphrased as follows: the state function S₀(Z₀(t)) can be expressed only in terms of the observables by having an explicit time dependence as S₀(X₀, t). The result can be easily extended to the case when Z₀(t) is not sufficient to provide a complete description of the macrostate. In this case, it is possible that S₀(Z₀(t), t) can be converted to a state function $S_0({\overline{\mathbf{Z}}}_0(t))$ by adding additional independent internal variables ${\mathit{\xi }}_0^{\prime }(t)$ not present in Z₀(t) (we collectively denote all the state variables by the set ${\overline{\mathbf{Z}}}_0(t))$) that uniquely specifies the macrostate provided the system is homogeneous. The proof is straight forward and will not be given here.

The state function S₀(Z₀(t)) for an IEQS, however, allows us to extend the Gibbs fundamental Equation (15) to

$$d{S}_0={\displaystyle \sum {}_p}\left(\partial S_0/\partial Z_{0p}\right)d{Z}_{0p}$$

(28)

in which the partial derivatives are related to the fields of Σ₀. The (possibly t-dependent) fields associated with the observables X₀ are

$$\left(\partial S_0/\partial E_0\right)=1/T_0,\left(\partial S_0/\partial V_0\right)=P_0/T_0,\cdots .$$

(29)

The fields associated with ξ₀(t) are related to affinities A₀ [2,3] according to

$$\left(\partial S_0/\partial {\mathit{\xi }}_0\right)=A_0/T_0.$$

(30)

As dX_0p ≡ 0, we have

$$d{S}_0={\displaystyle \sum {}_P}(\partial S_0/\partial {\mathit{\xi }}_{0p})d{\mathit{\xi }}_{0p}={\displaystyle \sum {}_P}{A}_0(t)d{\mathit{\xi }}_{0p}/T_0(t)\ge 0,$$

(31)

in accordance with the second law. Each summand must be non-negative for the law to be valid:

$$A_0(t)d{\mathit{\xi }}_{0p}/T_0(t)\ge 0.$$

(32)

The fields will continue to change until Σ₀ achieves equilibrium, when the affinities vanish

$${\mathrm{A}}_{0\mathrm{eq}}=0,$$

(33)

since ξ₀ are no longer independent. In this case, dS_0eq = 0 as we have already seen earlier in Equation (17).

From the Gibbs fundamental equation in Equation (28), we can compute the difference ΔS₀ between two internal equilibrium macrostates by integrating the equation. It should be obvious that an IEQS is very different from an EQS because the latter only has dS_0eq = 0 in contrast to dS₀ in Equation (31). As ξ₀(t) changes in time, $ℳ$₀(t) changes, but at each instant the nonequilibrium entropy for an IEQS has the maximum possible value for given Z₀(t) even though $ℳ$₀(t) ≠ $ℳ$_0eq; see Claim 1. The concept of IEQS contains the concept of EQS as a limiting case.

3.5. Arbitrary States as Incompletely Specified States

The entropy of an ANEQS S₀(Z₀(t), t) is not a state function, even though the state is still specified by Z₀(t). This may be because the state is inhomogeneous. This case will be considered in Section 3.8. Here we will consider the state to be homogeneous. In this case, the state is an incompletely specified state mentioned above for which the derivatives in Equation (29) cannot be identified as state variables like, temperature, pressure, etc. As said above, we can enlarge the state space to ${\overline{\mathbf{Z}}}_0(t)$ so that the entropy becomes a state function $S_0({\overline{\mathbf{Z}}}_0(t))$ of ${\overline{\mathbf{Z}}}_0(t)$. As the latter description is obtained by specifying the new internal variables ${\overline{{\mathit{\xi }}^{\prime }}}_0(t)$, which were left unspecified in the earlier description, it follows from the second law20 and from the nature of internal equilibrium that the entropy of an ANEQS obeys the inequalities

$$S_0({\mathbf{Z}}_0(t))\ge S_0({\mathbf{Z}}_0(t),t)\ge S_0({\overline{\mathbf{Z}}}_0(t)).$$

(34)

This is easily seen by considering the specification of ${\overline{\mathit{\xi }}}_0^{\prime }(t)$ as imposing restrictions on the system, which always lowers the entropy. Thus, we are able to make the following

Claim 3. The incomplete description for an arbitrary homogeneous state in terms of Z₀(t) can be converted to an IEQS description by a proper identification of the additional independent set${\overline{\mathit{\xi }}}_0^{\prime }(t)$ and its incorporation in${\overline{\mathbf{Z}}}_0(t)$. The description of the state in terms of Z₀(t) is not unique, while it becomes unique in terms of${\overline{\mathbf{Z}}}_0(t)$.

It is important to again make the following observation about a homogeneous state at this point. Let us assume that ξ used for the IEQS contains n internal variables. We have shown elsewhere [36] and summarized in Claim 3 that if we use only n′ < n of the internal variables to be denoted by ξ′, then the entropy will contain an explicit time dependence and will be expressed as S(X, ξ′, t). Thus, the same body will no longer be in IEQS with respect to the reduced set Z′ = {X, ξ′}, even though the body is uniform. Thus, being uniform is not sufficient for the body to be described as in IEQS unless we use the right number of internal variables for a given τ_obs in its description. The situation is no different for equilibrium states of a body as was discussed above in reference to the use of magnetization as an observable. The inequality $S(\mathbf{Z}(t)\ge S(\overline{\mathbf{Z}}(t))$ for internal equilibrium is no different in spirit from the inequality in Equation (14) for equilibrium states.

For a body that is not uniform, the explicit time dependence cannot be avoided. In this case, we may still consider subsystems of the body that remain in IEQSs as discussed in the following section. However, as there is no Gibbs fundamental relation for S0(Z0(t), t) due to the presence of explicit t, there is no way to determine differences in the entropy by integration. Thus, entropy difference between two ANEQSs cannot be determined unless the time-dependent path γ(t) connecting them is also specified.

The above discussion is easily extended to any interacting body with the conclusion that we need to include internal variables to describe its nonequilibrium states. For a body in internal equilibrium, the corresponding fields are given by derivatives in Equations (29) and (30) without the subscript 0

$$(\partial S/\partial E)=1/T,(\partial S/\partial V)=P/T,\cdots ,(\partial S/\partial \mathit{\xi })=\mathrm{A}/T;$$

(35)

these fields change with time until the body comes to equilibrium. However, as the observables for an interacting body are not necessarily constant, we need to include all terms in Equation (28) with the result that

$$dS={\displaystyle \sum {}_p(\partial S/\partial Z_p)d{Z}_p.}$$

(36)

Recall, see footnote 11, that we must set dX_f = 0 above. There is a similar analog of Equation (34).

3.6. Internal Equilibrium versus Equilibrium

For a body in IEQS, many results of equilibrium thermodynamics hold, except that we need a larger state space of Z(t). One such result has to do with the homogeneity of the body in IEQS [4,36], according to which a body in IEQS must be homogeneous and can only execute uniform translation and rotation. The Gibbs fundamental relation in the laboratory frame in this case has an additional contribution due to linear and angular momenta P(t) and M(t) [36]:

$$dE(t)=T(t)dS(t)-P(t)dV(t)+\mathrm{V}(t)·d\mathbf{P}(t)+\Omega (t)·d\mathrm{M}(t)-\mathrm{A}(t)·d\mathit{\xi }(t);$$

(37)

here, V(t) and Ω(t) are the linear and angular velocities in the lab frame; we have also shown internal variable for completeness. The additional contribution due to dP(t) is due to the velocity of the system as a whole and is important to include in the lab frame. For example, such a contribution is needed to describe the flow of a superfluid in which the normal and superfluid components have different velocities so that the superfluid cannot be considered at rest in any frame, see Equation (130.9) in [67]. We will need to allow for this possibility when we apply our approach of nonequilibrium thermodynamics to inhomogeneous systems below where different subsystems will undergo relative motion. The addition of translation and rotation in the description of the body provides the description at the hydrodynamic level, while the description is at the standard thermodynamic level in their absence [13]. This should be contrasted with the description at the Boltzmann level, which is kinetic in spirit in that it deals with collisions at the molecular level. Such a description is valid both for equilibrium and nonequilibrium processes [13]. It follows form Equation (37) that the drift velocity of the center of mass and the angular velocity of the system are given by

$${(\partial E(t)/\partial \mathbf{P}(t))}_{S,V,N,M,\mathit{\xi }}=\mathrm{V}(t),{(\partial E(t)/\partial \mathrm{M}(t))}_{S,V,N,\mathbf{P},\mathit{\xi }}=\Omega (t).$$

We observe from Equation (37) that the entropy in the lab frame is a function of the additional extensive conjugate quantities P(t) and M(t) rather than the external parameters V and Ω.

Let us consider the term V(t) · dP(t) in Equation (37), which can be written asV(t) · F(t)dt, where F(t) ≡ dP(t)/dt is the force acting on body. Thus, this term represents the internal work d_iW done by this force in time dt. Similarly, the work [P(t) − P₀]dV(t) is the internal work done by the body due to the pressure difference P(t) − P₀. Indeed, all the terms except the first one in Equation (37) involve mechanical variables and represent the isentropic change in dE; see the discussion following Equation (6). Accordingly, the negative of these terms can be identified with the work done dW(t) by the system so that dE(t) = T(t)dS(t) − dW(t). As the medium is characterized by T₀, P₀, V₀, Ω₀, and A₀ = 0, we have d_eW(t) = P₀dV(t)−V₀·dP(t)−Ω₀·dM(t). The difference d_iW(t) = dW(t)−d_eW(t) dissipates within the system in the form of the internal heat d_iQ(t) ≡ T(t)dS(t) − d_eQ(t), which is in accordance with the general result dQ(t) ≡ T(t)dS(t) derived earlier in Equation (6) for a system in IEQS; recall that d_iQ(t) ≡ d_iW(t).

It is now clear that the entropy difference along ${\mathcal{P}}_{12}^{\prime }$ is obtained by a trivial extension of Equation (9)

$$\mathrm{\Delta }S\left({ℳ}_1^{″}\to {ℳ}_2^{″}\right)\equiv S\left({\mathrm{Z}}_2\right)-S\left({\mathrm{Z}}_1\right)\equiv {\displaystyle {\int }_{{\gamma }_{12}^{\prime }}dS(\mathrm{Z})},$$

(38)

where the integration is over any path ${\gamma }_{12}^{\prime }$ along which all states are IEQSs; again, ${\gamma }_{12}^{\prime }$ may have nothing to do with ${\mathcal{P}}_{12}^{\prime }$. Entropy being a state function, the integral is independent of the actual path or the process. Thus, the change in the entropy in a nonequilibrium process ${\mathcal{P}}_{12}^{\prime }$ between two terminal IEQSs can also be computed just as we are able to compute the entropy change in a nonequilibrium process ${\mathcal{P}}_{12}$ connecting two equilibrium states. The states along ${\mathcal{P}}_{12}^{\prime }$ do not need to be in internal equilibrium. All that is required is for the two terminal states to be in internal equilibrium. If it happens that some of the states along ${\mathcal{P}}_{12}^{\prime }$ are in internal equilibrium in a larger state space, see Claim 3, then we can again compute the entropy difference between these states.

3.7. Internal Equilibrium Concept is Commonly Used

It may appear to a reader that the concept of IEQS for which the entropy is a state function is very restrictive. This is not the case as this concept is always implicit or explicit in the literature where the relationship of the thermodynamic entropy with state variables is investigated. As we will see later, the IEQS entropy of a body is given by the Boltzmann formula

$$S(\mathbf{Z}(t))=\ln W(\mathbf{Z}(t)),$$

(39)

in terms of the number of microstates W(Z(t)) corresponding to Z(t). In classical nonequilibrium thermodynamics [3], the entropy is always taken to be a state function, but only of the observables X(t). Gyftopoulos and Baretta [5] also take the entropy to be a state function of X(t). In the Edwards approach [68] for granular materials, all microstates are equally probable as is required for the above Boltzmann formula. Bouchbinder and Langer [69] assume that the nonequilibrium entropy is given by Equation (39). Lebowitz [55,56] also takes ln W(X(t)) for his definition of the nonequilibrium entropy. As a matter of fact, we are not aware of any work dealing with entropy computation that does not assume the nonequilibrium entropy to be a state function. This does not, of course, mean that all states of a system are IEQSs.

3.8. Inhomogeneous Body and Quasi-Independence

A body Σ that is not homogeneous cannot be in IEquations However, it is still possible in some cases to consider its various subsystems σ_k that happen to be individually in some IEQS at some instant t. This issue was discussed elsewhere [36], to which we refer the reader for more details. They need not all have the same temperature, pressure, etc. The issue of additivity of various extensive observables over σ_k naturally arises. We take σ_k, each of volume υ_k, to be spatially disjoint and make up the entire body so that $V={\displaystyle \sum _k{u}_k}$. The interfacial regions between various subsystems are defined so that they contain no particles. Therefore quantities such as particle number, linear and angular momenta, magnetization, polarization, etc. that are defined as a sum over particles are always additive: $N(t)\equiv {\displaystyle \sum {}_k}{n}_k(t),P(t)\equiv {\displaystyle \sum {}_k}{p}_k(t),M(t)\equiv {\displaystyle \sum {}_k{m}_k(t),\cdots }$. The problem arises with state variables such as E whose value in almost all cases of interest is determined by at least pairs of particles so its additivity is not trivial. In general, we can write $E={\displaystyle \sum {}_k{e}_k(t)+E^{(\mathrm{int})}}$, where e_k(t) is the energy of σ_k and E^(int) denotes the interaction energy between subsystems. Thus, E is in general nonadditive. As noted in footnote 1 and elsewhere [36], we restrict ourselves to tempered interactions (short range interactions or screened long range interactions) of some range R. Then, it is easy to show that the sizes of these subsystems, which we denote by λ_R, should be macroscopically large, e.g., [4], so that we can neglect E^(int) compared to ${\displaystyle \sum {}_k}{e}_k(t)$ to yield 21 the additivity of the energy, a mechanical quantity. We will say that various subsystems are almost noninteracting (weakly interacting) or mechanically independent [4].

The additivity of entropy requires a more subtle analysis. To clarify this point, we consider various homogeneous and macroscopic bodies Σ_k, k = 1, 2, 3⋯, each characterized by the state variable Z_k(t). We consider Λ identical spatially disjoint subsystems (each denoted by σ_k) of the body Σ_k; the energy and entropy of each subsystem are denoted by e_k ≠ 0 and s_k. In general, we can write $E_k=\mathrm{\Lambda }{e}_k+E_k^{(\mathrm{int})}$ and $S_k=\mathrm{\Lambda }{s}_k+S_k^{(\mathrm{int})}$, where the notation is obvious. Again, to restore additivity for E_k, we need to ensure that $\left|E_k^{(\mathrm{int})}/\mathrm{\Lambda }{e}_k\right|\ll 1$. Let ${\lambda }_R^{(k)}$ be the corresponding linear size of the macroscopically large σ_k. There is also a thermodynamic length, the correlatiion length${\lambda }_k^{\prime }$,22 which plays an important role in that two particles whose separation is larger than ${\lambda }_k^{\prime }$ are almost uncorrelated. Thus, the two subsystems, each of size larger than ${\lambda }_k^{\prime }$ act almost independently; we say that the two subsystems are quasi-independent [36].23 We can now neglect $S_k^{(\mathrm{int})}$ compared to Λs_k to make the entropy also additive; see SP3. Let λ_k denote the larger of ${\lambda }_R^{(k)}$ and ${\lambda }_k^{\prime }$. This is an important point as at low temperatures, it is possible for ${\lambda }_k^{\prime }$ to be larger than ${\lambda }_R^{(k)}$. We take the size of σ_k to be λ_k, so that various mechanically and thermodynamically independent σ_k’s are described by the state variable z_k(t) = Z_k(t)/Λ. By construction, E_k and S_k are now additive:

$$E_k=\mathrm{\Lambda }{e}_k,S_k=\mathrm{\Lambda }{s}_k.$$

As we have selected the largest length to ensure both kinds of independence, the internal variables are also additive over various σ_k. Under this condition, the entropy for the homogeneous body becomes a homogeneous function of order 1 of state variables. Hence, we conclude

Claim 4. Our concept of (nonequilibrium and equilibrium) entropy is in general a macroscopic concept as stated earlier.

We now imagine Σ to consist of various spatially disjoint σ_k as shown in Figure 3. The new body is inhomogeneous even though its various parts are homogeneous. The construction ensures that various subsystems are both mechanically and thermodynamically independent so that

$$E(t)\simeq {\displaystyle \sum {}_k{e}_k(t),S(t)\simeq {\displaystyle \sum {}_k{s}_k(t);\cdots .}}$$

(40)

We assume the subsystems are in IEQSs so that then s_k(t) ≡ s_k(z_k(t)). This allows us to define the entropy S(t) ≡ S(Z(t), t) of the body Σ as the sum

$$S(E(t),V(t),N,\mathcal{P}(t),\mathrm{M}(t),\mathit{\xi }(t),t)={\displaystyle \sum {}_k{s}_k(z_k(t));}$$

(41)

we have taken N to be a constant per footnote 12. For the k th subsystem, the Gibbs fundamental relation takes the following form [36],

$$d{e}_k=T_k(t)d{s}_k(t)-P_k(t)d{v}_k(t)+{\mu }_k(t)d{n}_k(t)+v_k(t)·d{\mathrm{p}}_k(t)+{\Omega }_k(t)·d{\mathrm{m}}_k(t)-{\mathrm{A}}^{\left(k\right)}(t)·d{\mathit{\xi }}_k(t),$$

(42)

with the drift and angular velocities given as the derivatives of s_k(e_k, υ_k, n_k, p_k, m_k, ξ_k):

$$\frac{{\mathrm{v}}_k(t)}{T_k(t)}=-\frac{\partial s_k(t)}{\partial {\mathrm{p}}_k(t)},\frac{{\mathrm{\Omega }}_k(t)}{T_k(t)}=-\frac{\partial s_k(t)}{\partial {\mathrm{m}}_k(t)}.$$

Observables not included above can be added when needed.

The computation of S(Z(t), t) in terms of s_k(z_k(t)) proceeds from the additivity of the entropy in Equation (41). We assume that N^(a)(t) ≥ 0 subsystems at time t correspond to the same macrostate specified by z^(a)(t). Let the set {z^(a)(t)} denote all distinct macrostates of all the subsystems. We write $S(\mathbf{Z}(t),t)={\displaystyle \sum {}_a}{N}^{(a)}(t)s({\mathrm{z}}^{(a)}(t))$. At some later time t′ = t + Δt, we can again divide the body in various subsystems, each in IEQS, to find the new sets $\{N^{(b)}(t^{\prime })\}$ and $\{{z^{\prime }}^{(b)}(t^{\prime })\}$ so that

$$S({\mathbf{Z}}^{\prime }(t^{\prime }),t^{\prime })=N^{(b)}(t^{\prime })s({{\mathrm{z}}^{\prime }}^{(b)}(t^{\prime })).$$

Here, we allow for the possibility of different Z(t) and Z′ (t′). We can now determine uniquely the change in the entropy between two inhomogeneous states by the difference of the above two entropies.

3.9. NEQS Thermodynamic Potential for an Interacting System Σ

We consider situation shown in Figure 1b. We simplify the discussion by not allowing for the possibility of relative translation and rotation between Σ and $\tilde{\mathrm{\Sigma }}$, and assume them to be at rest. Obviously, Σ₀ is not in IEquations We assume the medium to be in equilibrium and express its entropy as $\tilde{S}\left(\tilde{\mathbf{Z}}\left(t\right)\right)$.24 The system, which is out of equilibrium with the medium, is not assumed to be necessarily in eane IEQS so that we write S(Z(t), t) for its entropy. Assuming additivity of entropy, we have

$$S_0({\mathbf{Z}}_0(t),t)=\tilde{S}(\tilde{\mathbf{Z}}(t))+S(\mathbf{Z}(t),t).$$

(43)

We expand $\tilde{S}\left(\tilde{Z}\left(t\right)\right)$ in terms of Z(t) around ${\mathbf{Z}}_0(t){\mathbf{Z}}_0(t)=\tilde{\mathbf{Z}}(t)+\mathbf{Z}(t))$ by treating Z(t) as a small quantity compared to Z₀(t); see Refs. [35,36] for details. The difference $G^{\mathbf{X}}(t)=-T_0[S_0({\mathbf{Z}}_0(t),t)–\tilde{S}({\mathbf{Z}}_0(t))]$ is found to be the thermodynamic potential [35,36]; see in particular Equation (62) in [36].

$$G^{\mathbf{X}}(t)=E(t)-T_{0}S(t)+P_{0}V(t)+{\mu }_0^{\prime }·{\mathbf{X}}^{\prime }(t),$$

(44)

where X′(t) stands for all other observables besides E(t), V(t) and X_f ≡ N.25 Here, T₀, P₀ and $\mu {\prime }_0\equiv -T_0(\partial \tilde{S}/\partial \tilde{X}\prime )$ are the fields associated with the medium. Note that there are no terms involving internal variables as the corresponding affinities ${\mathrm{A}}_0\equiv -T_0(\partial \tilde{S}/\partial \tilde{\mathit{\xi }})$ vanish in equilibrium; see Equation (33).

To conclude that G^X (t) is a thermodynamic potential with the property that it decreases spontaneously as Σ₀ relaxes towards equilibrium, we proceed as follows. It follows from A0 = 0 and the constancy of X0 that $d\tilde{S}\left({\mathbf{Z}}_0(t)\right)/dt=0$. Accordingly, we conclude

$$\frac{d{G}^{\mathbf{X}}(t)}{dt}=-T_0\frac{d{S}_0(t)}{dt}\le 0,$$

which proves that G^X(t) is a time-dependent nonequilibrium thermodynamic potential. All terms except the second one in Equation (44) are mechanical quantities and can be experimentally determined. Despite this, G^X(t) cannot be experimentally determined unless we can experimentally determine S(t) in the NEQS. It follows from the above discussion that the entropy, being a thermodynamic quantity, cannot in general be related to purely mechanical quantities. In the absence of X′, the thermodynamic potential reduces to the conventional Gibbs free energy G(T₀, P₀, t) = E(t) − T₀S(t) + P₀V (t).

4. Vitrification and the Residual Entropy

As an application, we apply the above approach to vitrification, which is a prime example of an irreversible process going on at low temperatures. It is here one encounters the residual entropy, which will prove extremely useful in clarifying the confusion regarding which microstates are included in W in the Boltzmann entropy in Equation (39). Because of its importance, we briefly review vitrification. Normally, the process is carried out at some (not necessarily a fixed) cooling rate as follows but keeping the pressure P₀ of the medium fixed. The temperature of the medium is isobarically lowered by some small but fixed ΔT₀ from the current value to the new value, and we wait for (not necessarily fixed) time τ_obs at the new temperature to make an instantaneous measurement on the system before changing the temperature again. The choice of τ_obs ensures that at temperatures above the melting temperature T_0M, the system is a liquid in equilibrium, which can be supercooled below T_0M to give rise to a supercooled liquid. At some lower temperature T_0g, see Figure 4, the equilibration time τ_eq, which continuously increases as the temperature is lowered, becomes equal to τ_obs. Just below T_0g, the entropy of the system falls out of constrained equilibrium26 and follows the dotted curve, but the structures in the supercooled liquid are not yet frozen. They “freeze” at a lower temperature T_0G (not too far from T_0g) to form an amorphous solid with a viscosity close to 10¹³ poise.

This solid is identified as a glass (GL). The location of both temperatures depends on the rate of cooling, i.e., on τ_obs. Compared to the glass, the supercooled liquid (SCL) is in constrained equilibrium [70,71], and its entropy is shown by the continuous curve in Figure 4). Over the glass transition region between T_0G and T_0g, the nonequilibrium liquid (dotted curve) gradually turns from SCL at or above T_0g into a glass at or below T_0G [58]. Over this region, thermodynamic quantities such as the volume or the enthalpy change continuously but slowly during cooling. In addition, they also undergo relaxation towards SCL even if the temperature of the medium is held fixed; see the two downward vertical arrows. It is commonly believed that S_SCL vanishes at absolute zero (S_SCL(0) ≡ 0).

We bring the glass forming system Σ, which is at time t₁ in a medium at temperature T₀₁ (we will not explicitly exhibit P₀ in this section) in a macrostate $ℳ$₁, into a medium at a lower temperature T₀₂ at time t₂, where it happens to be macrostate $ℳ$₂ via an irreversible vitrification process ${\mathcal{P}}_{12}^{\prime }$. We have along ${\mathcal{P}}_{12}^{\prime }$

$$S{({ℳ}_2)}^{\underset{=}{{\mathcal{P}}_{12}^{\prime }}}S({ℳ}_1)+{\displaystyle \underset{{\mathcal{P}}_{12}^{\prime }}{\int }{d}_{e}S+}{\displaystyle \underset{{\mathcal{P}}_{12}^{\prime }}{\int }{d}_{i}S,}$$

(45)

which follows from Equation (26). A discontinuous change in the entropy is ruled out from the continuity of the Gibbs free energy G and the enthalpy H in vitrification proved elsewhere [35]. Since the second integral in Equation (45) is always positive, we conclude that

$$S({ℳ}_2)>S({ℳ}_1)+{\displaystyle \underset{T_{01}}{\overset{T_{02}}{\int }}{\mathcal{C}}_{P}d{T}_0/T_0.}$$

(46)

For an equilibrium macrostate $ℳ$₁ such as a liquid at or above T₀ = T_0M, S($ℳ$₁) = S_eq($ℳ$_eq1). The residual entropy S_R of the glass, which is the entropy S(0) at absolute zero (T₀₂ = 0), can be expressed as

$$S_{\mathrm{R}}\equiv S(0)>S_{\mathrm{expt}}(0)\equiv S(T_0)+{\displaystyle \underset{T_0}{\overset{0}{\int }}{\mathcal{C}}_{P}d{T}_0/T_0.}$$

(47)

It follows from this that S_R must be strictly larger than the measured27 S_expt(0) in the vitrification process; note that we have put no restriction on the nature of the states along the dotted curve in the above derivation. The difference S_R− S_expt(0) would be larger, more irreversible the process is. The quantity S_expt(0) can be determined calorimetrically by performing a cooling experiment. We take T₀₁ to be the melting temperature T_0M, and uniquely determine the entropy of the supercooled liquid at T_0M by adding the entropy of melting (fusion) to the crystal (CR) entropy S_CR(T_0M) at T_0M. The latter is obtained in a unique manner by integration along a reversible path from T₀ = 0 to T₀ = T_0M:

$$S_{\mathrm{CR}}(T_{0\mathrm{M}})=S_{\mathrm{CR}}(0)+{\displaystyle {\int }_0^{T_{0\mathrm{M}}}{\mathcal{C}}_{P,\mathrm{CR}}},d{T}_0/T_0,$$

here, S_CR(0) is the entropy of the crystal at absolute zero, which is traditionally taken to be zero in accordance with the third law and which we assume here, and C_P,CR(T₀) is the equilibrium isobaric heat capacity of the crystal. This then uniquely determines the entropy of the liquid to be used on the right hand side in Equation (47).

Experiment evidence for a non-zero value of S_expt(0) is abundant as discussed by several authors [72,73,75–82] among others; various textbooks [70,71] also discuss this issue. The strict inequality proves immediately that the residual entropy cannot vanish for glasses.

5. Statistical Entropy for an Isolated System with a Single Component Sample Space

The discussion in the last two sections has been about the thermodynamic entropy with no mention of microstates, component confinement in microstate space, etc., even though we have discussed vitrification in which a system can find itself among many disjoint components. We now turn to these topics as we provide a very general formulation of the statistical entropy.

5.1. General Derivation of the Statistical Entropy

The formulation is completely independent of the nature of dynamics in the system. The use of classical Newtonian dynamics at the microscopic level in identifying entropy was first initiated by Boltzmann [61,62] and has been a very active approach since then;28 see for example, Lebowitz [55,56]. The use of quantum dynamics was initiated by von Neumann [18] and has also been a very active approach, e.g., [33,42,43,64]. Despite this, it is important to investigate if it is possible to identify statistical entropy without the use of any dynamics. This is highly desirable as the dynamics, when extremely slow such as at low temperatures or in glasses, can result in the microstate space to consist of disjoint components in the system gets confined over the duration of τ_obs. The dynamics requires the temporal average of all the quantities over a period of time → ∞ (really larger than τ_eq). During component confinement, the dynamics does not allow the system to explore microstates belonging to disjoint components. Under this condition, if all physical quantities including the entropy are given by their temporal averages, then they should be governed by the particular component in which the system is found to be trapped; see the discussion in Section 1 and the footnote 10.

In contrast, Gibbs [10] introduced the ensemble average at each instant. The approach does not require any temporal average or dynamics. This issue was discussed elsewhere [72], where it was concluded that the temporal average is not suitable when τ_obs between successive measurements is not very large (τ_obs < τ_eq). The ensemble is found to represent all microstates even in the presence of confinement. It was also concluded that the ensemble average is always applicable. We will review this discussion and provide a clear evidence of the above conclusion when we discuss the issue of the residual thermodynamic entropy and its connection with the statistical entropy in Section 7. In addition, not bringing dynamics into consideration provides an opportunity to settle the dispute regarding the meaning of microstates counted in W; see the footnote 10. The correct interpretation will be found to be consistent with the ensemble average of Gibbs. This then reveals the true meaning of microstates counted in W.

Our approach will also demonstrate the statistical entropy to be a statistical average over microstates. Thus, it is not defined for a single microstate in our approach as is commonly done in stochastic thermodynamics [14–16]. Our derivation is based on the Fundamental Axiom presented in Section 1, and requires (mentally) preparing a large number of samples at a given instant t, all specified by Z(t) [52]. How this can be done even though the internal variables are not controllable by the observer will become clear below. The samples represent microstates with nonzero probabilities. It is clear that even for a glass, there is no restriction that the samples all refer to the same component of the microstate space. Indeed, different sample will belong to different components and the collection of samples will represent all components as discussed above.

We introduce some useful notations to simplify the derivation. We consider a macrostate $ℳ$(t) ≡ $ℳ$(Z(t)) of a body Σ at a given instant t. In the following, we suppress t unless necessary. The macrostate $ℳ$(Z) refers to the set of microstates m = {m_k} and their probabilities p = {p_k}.29 For the computation of combinatorics, the probabilities are handled in the following abstract way. We consider a large number $\mathcal{N}=\mathcal{C}W(\mathbf{Z})$ of independent samples of Σ, with $\mathcal{C}$ some large constant integer and W(Z) the number of distinct microstates m_α that occur in $ℳ$(Z) with non-zero probabilities. We will use Γ(Z) to denote the microstate space spanned by these m_ks, which is assumed to be a single component in this section. Because of the restriction p_α(t) > 0, the set m is determined by Z and will be denoted by m(Z) if clarity is needed.

For the isolated system Σ₀, we add a suffix 0 to the various system quantities introduced above. Let ${\mathcal{N}}_{0\alpha }(t)$ denote the number of samples in the m_0α-microstate so that

$$0<p_{\alpha }(t)={\mathcal{N}}_{0\alpha }(t)/{\mathcal{N}}_0\le 1;{\displaystyle \sum _{\alpha =1}^{W_0({\mathbf{Z}}_0)}{\mathcal{N}}_{0\alpha }(t)={\mathcal{N}}_0.}$$

(48)

The above sample space is a generalization of the ensemble introduced by Gibbs, except that the latter is restricted to an equilibrium system, whereas our sample space refers to the system in any arbitrary state so that p_α may be time-dependent. The (sample or ensemble) average of some quantity Q₀ over these samples is given by

$${\overline{Q}}_0\equiv {\displaystyle \sum _{\alpha =1}^{W_0({\mathbf{Z}}_0)}{p}_{\alpha }(t)Q_{0\alpha },}{\displaystyle \sum _{\alpha =1}^{W_0({\mathbf{Z}}_0)}{p}_{\alpha }(t)}\equiv 1,$$

(49)

where Q_0α is the value of Q₀ in m_0α. As the samples are independent of each other, we can treat the samples to be the outcomes of some random variable, the macrostate $ℳ$₀(t). This independence property of the outcomes is crucial in the following, and does not imply that they are equiprobable. The number of ways ${\mathcal{W}}_0$ to arrange the ${\mathcal{N}}_0$ samples into W₀(Z₀) distinct microstates is

$${\mathcal{W}}_0\equiv {\mathcal{N}}_0!/{\displaystyle \prod _{\alpha }{\mathcal{N}}_{0\alpha }(t)!.}$$

(50)

Taking its natural log to obtain an additive quantity per sample

$${\mathcal{S}}_0\equiv \ln {\mathcal{W}}_0/{\mathcal{N}}_0,$$

(51)

and using Stirling’s approximation, we see easily that ${\mathcal{S}}_0$, which we hope to identify later with the entropy S₀(t), can be written as the average of the negative of η(t) ≡ ln p(t), what Gibbs [10] calls the index of probability:

$${\mathcal{S}}_0({\mathbf{Z}}_0(t),t)\equiv -\overline{\eta (t)}\equiv -{\displaystyle \sum _{\alpha =1}^{W_0({\mathbf{Z}}_0)}{p}_{\alpha }(t)\ln p_{\alpha }(t),}$$

(52)

where we have also shown an explicit time-dependence for the reason that will become clear below.30 The above derivation is based on fundamental principles and does not require the system to be in equilibrium; therefore, it is always applicable. To the best of our knowledge, even though such an expression has been extensively used in the literature, it has been used without any derivation; one simply appeals to this form by invoking it as the information entropy; however, see Section 10.

Because of its similarity in form with $S_{\mathrm{G}}^{(\mathrm{c})}$ in Equation (20), we will refer to ${\mathcal{S}}_0({\mathbf{Z}}_0(t),t)$ as the Gibbs statistical entropy from now on. As the entropy for a process in which the system is always in IEQS can be determined by integrating the Gibbs fundamental relation in Equation (28), we can compare it with the statistical entropy introduced above. However, such an integration is not possible for a process involving states that are arbitrary (not in internal equilibrium). Therefore, it is not possible to compare ${\mathcal{S}}_0(t)$ with the corresponding thermodynamic entropy only because the value of the latter cannot be determined.

We refer the reader to several proofs in the literature [11,12,52,64,83,84] establishing that ${\mathcal{S}}_0(t)$ satisfies Equation (25). The maximum ${\mathcal{S}}_0(t)$ for given ${\mathbf{Z}}_0(t)$ occurs when m_0α are equally probable:

$$p_{\alpha }(t)\to p_{\alpha }{}_{,\mathrm{int}.\mathrm{eq}}\equiv 1/W_0({\mathbf{Z}}_0),\forall m_{\alpha }\in {\mathrm{\Gamma }}_0({\mathbf{Z}}_0).$$

In this case, the explicit time dependence in ${\mathcal{S}}_0(t)$ disappears and we have

$${\mathcal{S}}_{0,\max }({\mathbf{Z}}_0(t),t)={\mathcal{S}}_0({\mathbf{Z}}_0(t))=\ln W_0({\mathbf{Z}}_0(t)),$$

(53)

which is identical in form to the Boltzmann entropy in Equation (18) for an isolated body in equilibrium, except that the current formulation has been extended to an isolated body out of equilibrium; see also Equation (39). The statistical entropy in this case becomes a state function. The only requirement is that all microstates in m₀(Z₀) be equally probable. This is the extension of the maximum entropy principle to IEQSs first proposed by Gibbs [10] for EQSs and extensively studied later by Jaynes [85–87].

Let us consider a macrostate specified by some X₀ and the set ${\overline{m}}_0$ consisting of microstates $\left\{{\overline{\mathrm{m}}}_{\alpha }\right\}$. Let $\{{\overline{p}}_{\alpha }(t)\}$ denote the set of probabilities for ${\overline{\mathrm{m}}}_0$. Applying the above formulation to this macrostate, we obtain

$${\mathcal{S}}_0({\mathbf{X}}_0,t)\equiv -{\displaystyle \sum _{\alpha =1}^{W_0({\mathbf{X}}_0)}{\overline{p}}_{\alpha }(t)\ln }{\overline{p}}_{\alpha }(t),{\displaystyle \sum _{\alpha =1}^{W_0({\mathbf{X}}_0)}{\overline{p}}_{\alpha }(t)\equiv 1,}$$

(54)

as the statistical entropy of this macrostate, where W₀(X₀) is the number of distinct microstates should be obvious that

$$W_0({\mathbf{X}}_0)\equiv {\displaystyle \sum {}_{{\mathit{\xi }}_0(t)}}{W}_0({\mathbf{Z}}_0(t)).$$

Again, under the equiprobability assumption

$${\overline{p}}_{\alpha }(t)\to p_{\alpha ,\mathrm{eq}}=1/W_0({\mathbf{X}}_0),\forall {\overline{m}}_{\alpha }\in {\mathrm{\Gamma }}_0({\mathbf{X}}_0),$$

Γ₀(X₀) denoting the space spanned by microstates $\left\{{\overline{m}}_{\alpha }\right\}$, ${\mathcal{S}}_0({\mathbf{X}}_0,t)$ takes its maximum possible value

$${\mathcal{S}}_{0,\max }({\mathbf{X}}_0,t)={\mathcal{S}}_0({\mathbf{X}}_0)=\ln W_0({\mathbf{X}}_0),$$

(55)

which from Section 2.2 is identical the thermodynamic entropy S_0eq for an isolated body in equilibrium. The maximum value occurs at t ≥ τ_0eq. It is evident that

$${\mathcal{S}}_0[{\mathbf{Z}}_0(t),t]\le {\mathcal{S}}_0[{\mathbf{Z}}_0(t)]\le {\mathcal{S}}_0({\mathbf{X}}_0);$$

(56)

compare with Equation (34) by replacing ${\overline{\mathbf{Z}}}_0$ with Z₀ and Z₀ with X₀ to obtain the above inequalities.

The anticipated identification of nonequilibrium thermodynamic entropy with ${\mathcal{S}}_0$ under some restrictions allows us to identify ${\mathcal{S}}_0$ as the statistical entropy formulation of the thermodynamic entropy. From now on, we will refer to the general entropy in Equation (52) in terms of microstate probabilities as the time-dependent Gibbs formulation of the entropy or simply the Gibbs entropy, and will not make any distinction between the statistical and thermodynamic entropies. Accordingly, we will now use the standard symbol S₀ for ${\mathcal{S}}_0$ for the rest of the work, unless clarity is needed.

We will refer to S₀(Z₀(t)) in terms of W₀(Z₀(t)) in Equation (53) as the time-dependent Boltzmann formulation of the entropy or simply the Boltzmann entropy [55,56], whereas S₀(X₀) in Equation (55) represents the equilibrium (Boltzmann) entropy. It is evident that the Gibbs formulation in Equations (52) and (54) supersedes the Boltzmann formulation in Equations (53) and (55), respectively, as the former contains the latter as a special limit. However, it should be also noted that there are competing views on which entropy is more general [42,43,55,56]. The continuity of S₀(Z₀, t) (except in the thermodynamic limit) follows directly from the continuity of p_α(t). The existence of the statistical entropy S₀(Z₀, t) follows from the observation that it is bounded above by ln W₀(Z₀) and bounded below by 0, see Equation (53).

It should be stressed that $\mathcal{W}$ is not the number of microstates of the $\mathcal{N}$ replicas; the latter is given by ${[W_0({\mathbf{Z}}_0)]}^{\mathcal{N}}$. Thus, the entropy in Equation (51) should not be confused with the Boltzmann entropy, which would be given by $\mathcal{N}$. It should be mentioned at this point that Boltzmann uses the combinatorial argument to obtain the entropy of a gas, see Equation (21), resulting in an expression similar to that of the Gibbs entropy in Equation (20) except that the probabilities appearing in his formulation represents the probability of various discrete states of a particle.

6. Statistical Entropy for an Interacting System with a Single Component Sample Space

6.1. Statistical Entropies of the System and Medium

The system must be treated as interacting (with a medium) if any measurements have to be performed on it. Using the above formulation of S₀(Z₀(t), t), see Equation (52), we have determined [36] the statistical formulation of the entropy for a system Σ, which is a small but macroscopically large part of Σ₀; see Figure 1b and Equation (43). It is assumed that the system and the medium are quasi-independent so that S₀(t) ≡ S₀(Z₀(t),t) can be expressed as a sum of entropies S(t) ≡ S(Z(t),t) and $\tilde{S}(t)\equiv \tilde{S}(\tilde{\mathbf{Z}}(t))$ of the system and the medium, respectively; see [36] for the derivation, which we omit here as it is straightforward. The two statistical entropies are given by identical formulations

$$S(t)=-{\displaystyle \sum _k{p}_k(t)\ln p_k(t),}\tilde{S}(t)=-{\displaystyle \sum _k{\tilde{p}}_k(t)\ln {\tilde{p}}_k(t)},$$

(57)

respectively. Here, m_k with probability p_k > 0 denotes a microstate of Σ and ${\tilde{m}}_k$ with probability ${\tilde{p}}_k>0$ that of the medium. In the derivation [36], we neither assume the medium nor the system to be in internal equilibrium. Thus, the above formulations of statistical entropies are valid under all conditions. However, the formulation will not remain valid if the two systems are not quasi-independent; in that case, we must account for the entropy contribution due to their mutual interactions. The same will also be true of the thermodynamic entropies.

6.2. Statistical Entropy as a State Function: IEQS

For a system in IEQS, its statistical entropy S(t) must be maximized under the constraints imposed by the medium; compare with Gibbs [10] and Jaynes [85–87] for EQSs with no internal variables. The constraints we need to satisfy are on the fixed average values of the state variables:

$$\mathbf{Z}(t)={\displaystyle \sum _k{\mathbf{Z}}_k{p}_k,}$$

where Z_k is the value of Z in m_k. The condition for an IEQS is obtained by varying p_k. Using the Lagrange multiplier technique, it is easy to see that the condition for this in terms of the Lagrange multipliers whose definitions are obvious is

$$-\ln p_k=\lambda +\mathit{\lambda }·{\mathbf{Z}}_k;$$

(58)

the scalar product is over the elements in the set Z_k. It now follows that

$$S=\lambda +\mathit{\lambda }·\mathbf{Z}(t),$$

(59)

using the above scalar product. We identify the Lagrange multipliers by observing that

$$dS=\mathit{\lambda }·d\mathbf{Z}.$$

(60)

Comparing this relation with the Gibbs fundamental relation for the system in Equation (36), we find

$${\lambda }_p\equiv (\partial S/\partial Z_p).$$

Accepting this identification now allows us to conclude that the statistical entropy S(t) of the interacting system in Equation (57) is no different than thermodynamic entropy of the same system in IEQS up to a constant. The constant can be determined by the use of the Nernst’s postulate for both entropies to ensure that they are numerically identical and proves the following claim:

Claim 5. The two entropies are identical everywhere for an interacting system in IEquations

The significance of λ is quite obvious. In IEQS, it is given by

$$\lambda =S-(\partial S/\partial \mathbf{Z})·\mathbf{Z}(t).$$

(61)

In equilibrium, the Lagrange multipliers associated with the internal variables vanish and Equation (60) reduces to

$$d{S}_{\mathrm{eq}}={\mathit{\lambda }}_{\mathrm{eq}}·d\mathbf{X}.$$

(62)

A special case of such a system is the (equilibrium) canonical ensemble of Gibbs.

6.3. Statistical Entropy not a State Function: ANEQS

The thermodynamic entropy for an ANEQS cannot be measured or computed. Thus, while the statistical entropy can be computed in principle in all cases, there is no way to compare its value with the thermodynamic entropy in all cases. Thus, no comment can be made about their relationship in general. However, as discussed in Section 3.8, we are able to express the non-state thermodynamic entropy of an inhomogeneous body in terms of the state entropies of its quasi-independent subsystem. It is easy to see that both entropies are identical. Furthermore, when the system is in a homogeneous ANEQS, the inequalities in Equations (56) and (34) also shows the deep connection between the two entropies. However, as arbitrary states occur because of an incomplete specification of the NEQS, the problem is more of an incomplete specification. We can find an extended set $\overline{\mathbf{Z}}$ for a complete specification of the macrostate so that the entropy becomes a state function. Thus, we conjecture the following:

Conjecture 1. As the two entropies are the same when the homogeneous body is in IEQS or when it is in an inhomogeneous ANEQS for which various quasi-independent subsystems are in IEQSs, the thermodynamic entropy is no different from its statistical analog for any NEQS.

The conjecture allows a computational method to determine the thermodynamic entropy for any NEQS.

7. Statistical Entropy with Component Confinement

The consideration of dynamics when the sample (also known as the microstate or phase) space has a single component has played a pivotal role in developing the kinetic theory of gases [55,56,62], where the interest is at high temperatures. As dynamics is very fast here, it is well known that the ensemble entropy agrees with its temporal formulation. However, at low temperatures, where dynamics becomes sluggish as in a glass [57,58], the system can be confined into disjoint components. This occurs when the observational time scale τ_obs becomes shorter than the equilibration time τ_eq; see also Section 4.

Confinement at a phase transition such as a liquid-gas transition is well known in equilibrium statistical mechanics [4,52,72]. It also occurs when the system undergoes symmetry breaking such as during magnetic transitions, crystallizations, etc. However, confinement can also occur under nonequilibrium conditions. The issue has been recently considered by us [52], where only energy as an observable was considered. We again focus on an isolated system Σ₀. The discussion is easily extended to the present case when confinement occurs for whatever reasons into one of the thermodynamically significant number31 of disjoint components Γ_0λ, λ = 1, 2, 3⋯, ${\mathcal{C}}_0$, each component corresponding to the same set Z₀ (we suppress the dependence for simplicity), respectively. Such a situation arises, for example, in Ising magnets at the ferromagnetic transition., where the system is either confined to Γ₀₊ with positive magnetization or Γ₀₋ with negative magnetization. Even a weak external magnetic field |H| → 0, that we can control as an observer, will allow the system to make a choice between the two parts of Γ₀. It just happens that in this case $\mathcal{C}=2$ and is thermodynamically insignificant.

The situation with glasses or other amorphous materials is very different [57]. We again focus on an isolated system Σ₀. In the first place, Γ₀ is a union of thermodynamically significant number ${\mathcal{C}}_0\sim e^{c0{\mathcal{N}}_0},c_0>0$, disjoint components. In the second place, there is no analog of a symmetry breaking field. Therefore, there is no way to prepare a sample in a given component Γ_0λ. Thus, the samples will be found in all different components. As there is no symmetry breaking field, there is no harm in treating the glass as an isolated system. Taking into consideration disjointness of the components generalizes the number of configurations in Equation (50) to

$${\mathcal{W}}_0\equiv {\mathcal{N}}_0!/{\displaystyle \prod _{\lambda ,\alpha \lambda }{\mathcal{N}}_{0{\alpha }_{\lambda }}(t)!,}$$

(63)

where ${\mathcal{N}}_{0\alpha \lambda }$ denotes the number of sample in the microstate m_0αλ in the λ-th component. In terms of $p_{\alpha \lambda }={\mathcal{N}}_{0\alpha \lambda }(t)/{\mathcal{N}}_0$, this combination immediately leads to

$${\mathcal{S}}_0(t)\equiv \ln {\mathcal{W}}_0/{\mathcal{N}}_0=-{\displaystyle \sum {}_{\lambda }{\displaystyle \sum {}_{{\alpha }_{\lambda }}{p}_{{\alpha }_{\lambda }}(t)\ln p_{{\alpha }_{\lambda }}(t)},}$$

(64)

for the statistical entropy of the system and has already been used earlier [52] by us; see Section 4.3.3 there. From what has been said above, this statistical entropy is also the thermodynamic entropy of a IEQS under component confinement for which both entropies are state functions of Z₀. Therefore, as before, we take ${\mathcal{S}}_0$ to be the general expression of the nonequilibrium thermodynamic entropy and use S₀ in place of ${\mathcal{S}}_0$.

It should be stressed that the number of microstates that appear in ${\mathcal{S}}_0(t)$ has an explicit sum over all components, and not just a single component that would be the case if there were a symmetry breaking field to prepare the body in a particular component. This observation will prove important below when making connection with glasses.

In terms of $p_{\lambda }(t)\equiv {\displaystyle \sum {{}_{\alpha }}_{{}_{\lambda }}}{p}_{{\alpha }_{\lambda }}(t)$, it is easy to see [52] that

$$S_0(t)={\displaystyle \sum {}_{\lambda }}{p}_{\lambda }(t)S_{0\lambda }(t)+S_{0\mathcal{C}}(t).$$

(65)

The entropy S_0λ(t) of the component Γ_0λ in terms of the reduced microstate probability ${\widehat{p}}_{{\alpha }_{\lambda }}\equiv p_{{\alpha }_{\lambda }}/p_{\lambda }$ is

$$S_{0\lambda }(t)=-{\displaystyle \sum {}_{{\alpha }_{\lambda }}{\widehat{p}}_{{\alpha }_{\lambda }}(t)\ln }{\widehat{p}}_{{\alpha }_{\lambda }}(t)$$

(66)

so that it is its average over all components. The second term represents the component entropy and is given by

$$S_{0\mathcal{C}}(t)=-{\displaystyle \sum {}_{\lambda }{p}_{\lambda }(t)\ln p_{\lambda }(t),}$$

(67)

The same calculation for an interacting system will result in an identical formulation for the entropy S(t) as in Equation (65) and other quantities. For S(t), we have

$$S(t)=-{\displaystyle \sum {}_{\mu }{\displaystyle \sum {}_{k_{\mu }}{p}_{k_{\mu }}(t)\ln p_{k_{\mu }}(t)},}$$

(68)

where the sum over μ is over disjoint components $T{d}_{\mathrm{i}}^{(\mathrm{Q})}S=(T_0-T)d_{\mathrm{e}}S$, for the system, and k_μ represents a microstate in the μth component. The component entropy $S_{\mathcal{C}}(t)$ is given by

$$S_{\mathcal{C}}(t)=-{\displaystyle \sum {}_{\mu }{p}_{\mu }(t)\ln p_{\mu }(t)},$$

(69)

with the sum over disjoint components μ for the system. We now apply this formulation to a glass, which is now treated as an interacting system. As noted above, the number of microstates that appear in S(t) belong to all $\mathcal{C}$ components. At absolute zero, only the lowest lying (in energy) microstate in each component will be occupied, which we take to be non-degenerate. In this case, the analog ${\widehat{p}}_{k\mu }$ of ${\widehat{p}}_{\alpha \lambda }$ for the μth component is ${\widehat{p}}_{k\mu }=1$. Thus, the analog S_μ(t) of S_0λ(t) is strictly zero: S_μ(t) = 0. It then follows that at absolute zero, $S(t)=S_{\mathcal{C}}(t)$ determines the residual entropy of any glassy NEQS.

If it happens that S(t) in Equation (68) is restricted to a sum over a single component, then S_C(t) = 0 and the residual entropy will always vanish for all NEQSs. This means that the number of microstates that appears in the Boltzmann formulation will correspond to W_μ(Z), the number of microstates in the μth component in which the glass appears to confined. As the glass cannot visit microstates belonging to other components in the duration τ_obs, this scenario suggests that the not all microstates determine the entropy of the system. However, as the experimental evidence is quite strong that the residual entropy is not zero as discussed in Section 4, we must conclude that it is not W_μ(Z) but all microstates in all components determines the residual entropy, which is given by the absolute zero value of S_C(t) in Equation (69). If all components are equally probable, then $p_{\mu }(t)=1/\mathcal{C}(0)$, with $\mathcal{C}(0)$ representing the number of components at absolute zero, then the residual entropy of a glass in IEQS is given by

$$S_{\mathrm{R}}=\ln \mathcal{C}(0)>0.$$

Finally, the calorimetrically measured (see footnote 27) non-zero entropy S_expt(0), which is a lower bound on S_R, see Equation (47), allows us to finally settle the meaning of W that appears in the Boltzmann formulation.

Claim 6. W that appears in the Boltzmann formulation represents all microstates with non-zero probabilities regardless of whether they can be accessed by the system during τ_obs.

8. Semi-Classical Approximation in Phase Space for S

The analog of a quantum microstate in classical statistical mechanics is normally obtained by recognizing that in the adiabatic approximation,32 each small phase space cell of volume element dx_c(= dpdq in terms of generalized coordinates q and momenta p) of size h^s corresponds to a microstate [4,12], where s is the degrees of freedom of the system. The latter follows from the Bohr-Sommerfeld quantization rule for a periodic motion. The adiabatic approximation requires the parameters characterizing the system to vary extremely slowly. We will assume one such parameter λ (such as the volume V) so that [90] τλ << λ, where τ is the period of oscillation for constant λ; the system would be isolated in the latter case. In the above approximation, the energy of the system will vary very slowly and can be taken to be constant over a period of oscillation. The action taken over the closed path for the constant value of λ and the energy is quantized [17]: ∫ pdq = (n + 1/2)h. This observation is the justification of the above cell size of a classical microstate. Thus, the number of “classical” microstates is given by W = ΔΓ/h^s, where ΔΓ is the phase space volume corresponding to the system. This allows us to divide the phase space into W cells, index by k, of volume dx_k = dx_c and “centered” at x_k at some time which we call the initial time t = 0. We will denote the evolution of the cell at time t by the location of its “center” x_k(t) and its volume element dx_k(t). In terms of the distribution function f(x, ξ, t) in the phase space, which may depend on the internal variables, the kth microstate probability therefore is given by

$$p_k(t)\equiv f\left(x_k,{\mathit{\xi }}_k,t\right)d{x}_k(t).$$

(70)

Evidently,

$${\displaystyle \sum {}_{k}f(x_k,{\mathit{\xi }}_k,t)d{x}_k(t)=1}$$

(71)

at all times. The entropy and the average of any observable O(x, t) of the system are given by

$$S(t)\equiv -{\displaystyle \sum {}_k}f(x_k,{\mathit{\xi }}_k,t)d{x}_k(t)\ln [f(x_k,{\mathit{\xi }}_k,t)d{x}_k(t)],$$

(72a)

$$\overline{O}(t)\equiv {\displaystyle \sum {}_k}O(x_k,t)f(x_k,{\mathit{\xi }}_k,t)d{x}_k(t),$$

(72b)

the sum being over all W microstates. While it is easy to see that continuum analogs for Equations (71) and (72b) are easily identified, this is not so for the entropy in Equation (72a) [88,89]. However, it should be obvious that S_f in Equation (22) cannot be a candidate for the statistical entropy S(t).

It is well known [17] that the system in the adiabatic limit remains in the same quantum state. For Hamiltonian dynamics, the conservation of the phase space cell volume under evolution ensures dx_k = dx_c for each cell so that $\dot{x}=0$. This results in the incompressibility of the phase space. In equilibrium, f(x_k, t) is not only uniform but also constant, and we conclude from Equation (71) that this value is f = 1/ΔΓ. Accordingly, p_k = 1/W in equilibrium as expected and we obtain the equilibrium entropy S_eq = ln W.

The situation is far from clear for nonequilibrium states. The system is no longer restricted to be in the same microstate, which means that the microstate energy is no longer a constant and the phase space trajectory is no longer closed. Thus, the suitability of the Bohr-Sommerfeld quantization is questionable, and care must be exercised to identify the microstates. We will adopt the following prescription. We consider some EQS (uniform f(x)) of the isolated system to identify the cell volume dx_c = h^s. Once the identification has been done, we will no longer worry about its relationship with h^s, and only deal with the cells. We then follow the evolution of each cell in time in a nonequilibrium process during which $\dot{x}=0$ may or may not hold. Thus the volume of each cell may no longer be constant. The process may also result in changes in f [91]. Indeed, it is quite possible that f diverges at the same time that dx_c vanishes [92]. However, their product, which determines the microstate probability, must remain strictly bounded and ≤ 1. In particular, as the cell volume shrinks to zero, f must diverge to keep the product bounded. Thus, the divergence [93,94] of f alone does not imply that the entropy diverges to negative infinity.

This is easily seen by the following 1-d damped oscillator, the standard prototype of a dissipative system [90]: $\ddot{x}+2\kappa \dot{x}+{\omega }_0^{2}x=0$ with a (positive) damping coefficient, which is chosen such that κ > ω₀ just for simplicity. We have the case of aperiodic damping. We will only consider the long time behavior. It is easy to see that in this limit $({\kappa }^{\prime }=\kappa +\sqrt{{\kappa }^2-{\omega }_0^2})$

$$d{x}_c\sim \exp (-2{\kappa }^{\prime }t),f\sim \exp (+2{\kappa }^{\prime }t),$$

and their product remains bounded, as expected.

9. 1-d Tonks Gas: A simple Continuum Model

A careful reader would have realized by this time that the proposed entropy form in Equation (52) is not at all the same as the standard classical formulation of entropy, such as for the ideal gas, which can be negative at low temperatures or at high pressures. The issue has been discussed elsewhere [95] but with a very different perspective. Here, we visit the same issue from yet another perspective that allows us to investigate if and how the entropy in continuum models is related to the proposed entropy in this work. For this, we turn to a very simple continuum model in classical statistical mechanics: the Tonks gas [96,97], which is an athermal model and contains the ideal gas as a limiting case when the rod length l vanishes. We will simplify the discussion by considering the Tonks gas in one dimension. The gas consists of r impenetrable rods, each of length l lying along a line of length L. We will assume r to be fixed, but allow l and L to change with the state of the system determined by its temperature, pressure, etc. The configurational entropy per rod determined by the configurational partition function is found to be [97]

$$s_c=\ln [e(v-l)a],$$

(73)

where υ is the “volume” available per rod L/r and a is a constant that has the dimension of length to make the entropy and the corresponding configurational partition function33

$$Z_c=\frac{1}{a^r}{\displaystyle \int {\displaystyle \int \cdots {\displaystyle \int \exp }}}(-\varphi (x_1,x_2,\cdots x_r)/T)d{x}_{1}d{x}_2\cdots d{x}_r$$

dimensionless. The function ϕ represents the overlap potential, which is zero when no rods overlap and becomes infinitely large whenever there is an overlap [97]. The temperature of the gas is denoted by T, which is determined by the distribution of momenta of various rods.34 As we will be relating Tonks gas to a lattice model, we will only consider the configurational entropy in the rest of the section. The configurational entropy is obtained by dividing T ln Z_c by T. Even though s_c is derived for an equilibrium Tonks gas, it is easy to see from the nature of the microstate probabilities p_k in Equation (58) that the same result also applies for the gas in internal equilibrium whose temperature is T. The set Z of state variables includes E, L, r, l and ξ. All other extensive quantities will in general be functions of Z.

The entropy vanishes when υ = l + 1/ae and becomes negative for all l < υ < l + 1/ae. Indeed, it diverges to −∞ in the incompressible limit υ = l. This is contrary to the statistical entropy in the Boltzmann approach (cf. Equation (18)) or the Gibbs approach (cf. Equation (52)), +which can never be negative. Can we reconcile the contradiction between the continuum entropy and the current statistical formulation?

We now demonstrate that the above entropy for the Tonks gas is derivable from the current statistical approach under some approximation, to be noted below, by first considering a lattice model for the Tonks gas and then taking its continuum limit. It is in the lattice model can we determine the number of microstates. In a continuum, this number is always unbounded (see below also). We consider a 1-d lattice Λ_f with N_f sites; the lattice spacing, the distance between two consecutive sites, is given by δ. We take N_f >> 1 so that L_f = L − rl = (N_f − 1)δ ≈ N_fδ is the length of the the lattice Λ_f. Evidently, L_f is a function of observables L, r, and l, it does not depend on ξ. Despite this, there is no reason to assume that N_f and δ individually do not depend on E and ξ; only their product is independent. In general, we expect N_f and δ to depend on Z.

We randomly select r sites out of N_f. The number of ways, which then represents the number of configurational microstates, is given by

$$W_c={\mathcal{N}}_{\mathrm{f}}!/r!\left({\mathcal{N}}_{\mathrm{f}}-r\right)!.$$

After the choice is made, we replace each selected site by λ + 1 consecutive sites, each site representing an atoms in a rod, to give rise to a rod of length l ≡ λδ. The number of sites in the resulting lattice Λ is

$$\mathcal{N}={\mathcal{N}}_{\mathrm{f}}+r\lambda$$

so that the length of Λ is given by L = L_f + rl = (N − 1)δ ≈ Nδ since N >> 1. Again, there is no reason to assume that N and δ individually do not depend on Z as above.

We introduce the number densities φ_f = r/N_f, ρ_f = r/N_fδ ≈ r/L_f and ρ = r/Nδ ≈ r/L. A simple calculation shows that S = ln W_c is given by

$$S=-{\mathcal{N}}_{\mathrm{f}}[{\rho }_{\mathrm{f}}\delta \ln {\rho }_{\mathrm{f}}\delta +(1-{\rho }_{\mathrm{f}}\delta )\ln (1-{\rho }_{\mathrm{f}}\delta )].$$

This result can also be obtained by taking the athermal entropy for a monodisperse polymer solution a Bethe lattice [98] by setting the coordination number q to be q = 2. We now take the continuum limit δ → 0 for fixed ρ_f and ρ, that is fixed L_f and L, respectively. In this limit, (1 − ρ_fδ) ln(1 − ρ_fδ) ≈ −ρ_fδ, and we find

$$S=r\ln (e/{\rho }_{\mathrm{f}}\delta )\to \infty .$$

(74)

The continuum limit of S using the Boltzmann approach has resulted in a diverging entropy regardless of the value of ρ_f [95], a well known result. Using the arbitrary constant a of Equation (73) results in

$$S/r=\ln (e/{\rho }_{f}a)-\ln (\delta /a),$$

(75)

in which the first term remains finite in the continuum limit, and the second term contains the divergence.35 The diverging part, although explicitly independent of ρ_f, still depends on the state of the gas through δ, and cannot be treated as a constant unless we assume δ to be independent of the state of the gas. It is a common practice to approximate the lattice spacing δ as a constant. In that case, the diverging term represents a constant that can be subtracted from S/r. Recognizing that 1/ρ_f = υ − l, we see that the first term in Equation (75) is nothing but the entropy of the Tonks gas in Equation (73). The equivalence only holds in the state independent constant δ approximation.

As the second term above has been discarded, the continuum entropy s_c also has no simple relationship with the number (≥ 1) of microstates in the continuum limit, which means that the continuum entropy cannot be identified as the Boltzmann entropy in Equation (55). To see this more clearly, let us focus on the centers of mass of the rods; they can be thought of as representing the r sites that were selected in Λ_f. Each of the r sites x_k, k = 1, 2,⋯, r, is free to move over L_f. The adimensional volume |Γ_f|, also called the probability and denoted by Z (not to be confused with the the partition function) by Boltzmann [55,56,63], of the corresponding phase space Γ_f is $L_{\mathrm{f}}^r/a^r$. However, contrary to the conventional wisdom [55,56], ln |Γ_f| does not yield s_c. The correct expression is given by the Gibbs-modified adimensional volume |Γ_f|/r!, i.e.,

$$\frac{1}{r!a^r}{L}_{\mathrm{f}}^r.$$

The presence of r! is required to restrict the volume due to indistinguishability of the rods à la Gibbs. For large r, this quantity correctly gives the entropy s_c. However, this quantity is not only not an integer, it also cannot be always larger than or equal to unity, as noted above.

10. Summary and Discussion

The current work is motivated by a desire to understand the relationship between thermodynamic and statistical entropies. We have briefly reviewed the thermodynamic entropy and several important statistical entropies in Section 2. Of the latter, we pay close attention to the Boltzmann and Gibbs entropies that have played a major role in contemporary statistical thermodynamics. While these entropies and their equivalence with thermodynamic entropies are well understood in equilibrium for ergodic systems, it is not so obvious that the equivalence also holds off-equilibrium. Extension to nonequilibrium states gives rise to new and important technical problems, requires clarification of some confusion in the field, but also results in establishing the Clausius equality dQ = TdS which gives rise to some simplification of extreme importance; see point (6) below.

• We need to formulate the concept of a state, state variables and state functions. We discuss these issues and the behavior of nonequilibrium thermodynamic entropy in Section 3, where we follow the recent approach initiated by us [35–39]. In particular, we discuss the role of internal variables that are needed to specify the (macro)state of a NEQS. Without internal variables, we cannot introduce the concept of a macrostate for an isolated system (see Section 3.1 and an interacting system (see Section 3.4). Thus, state variables include extensive observables (see footnote 11) and internal variables (see footnote 17) to specify a macrostate. The thermodynamic entropy is a state function for IEQS but a non-state function for ANEQS. The concept of internal equilibrium to characterize IEQSs [35–39] is based on the fact that state variables uniquely specify homogeneous macrostates. We point out its close similarity with the concept of equilibrium; see Section 3.6. The relationship of ANEQSs to IEQSs is similar to that of NEQSs to EQSs. In particular, we show how the entropy can be calculated for any arbitrary nonequilibrium process whose end states are IEQSs. We discuss various aspects of internal equilibrium and show how the entropy of an inhomogeneous system in ANEQS can be determined provided each of its subsystems are in internal equilibrium; see Section 3.8. A uniform body not in internal equilibrium with respect to a smaller set Z(t) of state variables can be made in internal equilibrium by extending the set of state variables. This justifies our claim in footnote 8. The vitrification process is reviewed in Section 4 where we introduce the concept of the residual thermodynamic entropy S_R.
• We provide a first principles statistical formulation of nonequilibrium thermodynamic entropy for an isolated system in terms of microstate probabilities without the use of any dynamical laws36 in Section 5. The formulation is based on the Fundamental Axiom and is valid for any state of the body. We use a formal approach (frequentist interpretation of probability) by extending the equilibrium ensemble of Gibbs to a nonequilibrium ensemble, which is nothing but a large number ${\mathcal{N}}_0$ of (mental) samples of the body under consideration; we refer the ensemble as a sample space. The formal approach enables us to evaluate the combinatorics for a given set of microstate probabilities. The resulting statistical entropy is independent of the number of samples and depends only on the probabilities as is seen from Equations (52) and (64). Thus, the use of a large number of samples is merely a mathematical formality and is not required in practice. We have shown that in equilibrium, the statistical entropy is the same as the equilibrium thermodynamic entropy: ${\mathcal{S}}_{\mathrm{eq}}(\mathbf{X})=S_{\mathrm{eq}}(\mathbf{X})$. However, we have also shown that the statistical entropy is equal to the thermodynamic entropy in IEQS described by the state variables $\mathbf{Z}(t):{\mathcal{S}}_{\mathrm{IEQS}}(\mathbf{Z}(t))=S_{\mathrm{IEQS}}(\mathbf{Z}(t))$. The IEQSs correspond to homogeneous states of the body. The conclusion also remains valid for an inhomogeneous system in ANEQS whose subsystems are individually homogeneous and in different IEQSs. We cannot make any comment about the relationship between $\mathcal{S}(\mathbf{Z}(t),t)$ and S(Z(t),t) for the simple reason that there is no way to measure or calculate a non-state function S(Z(t),t). Despite this, when the system is in a homogeneous ANEQS, the inequalities in Equations (56) and (34) still show the deep connection between the two entropies. Therefore, the statistical entropy in this case can be used to give the value of the corresponding thermodynamic entropy. This then finally allows us to make Conjecture 1 in Section 6. The discussion is extended to the case where state space is multi-component such as in glasses in Section 7.
• The second problem relates to the meaning of the number of microstates W that appears in the Boltzmann entropy. We have concluded in previous work [52,58] that temporal averages may not be relevant at low temperatures where component confinement occurs in of thermodynamically significant several disjoined components. The glass is formed when the system gets trapped in one of these components. Does W refer to the number of microstates within this component or to all microstates in all the components even though the glass is not allowed to probe these components over the observation period τ_obs < τ_eq? If dynamics is vital for the thermodynamics of the system, then it would appear that latter alternative should be unacceptable. Various reliable estimates for S_R show that it is positive. Its positive value plays a very important role in clarifying the meaning of W. The answer is that W must contain all the microstates with non-zero probabilities in all disjoint components; see the Claim 6 in Section 7.
• In Section 8, we briefly review the semi-classical approximation for the entropy by introducing cells in phase space, whose volume is h^s under adiabatic approximation. Using the cell construction, we show that S_f in Equation (22) cannot represent the entropy of the system. We argue that for nonequilibrium states, f may diverge as the cell volume dx_c → 0. However, the product fdx_c must be strictly ≤ 1. We consider a simple example of a 1-d damped oscillator for which we show that the above conclusion is indeed valid. Thus, contrary to what is commonly stated, e.g., [92,94], the divergence of f does not mean that the entropy also diverges.
• Thermodynamic entropy can become negative, but this is not possible for the statistical entropy. To clarify the issue, we consider a simple lattice model in Section 9, which in the continuum limit becomes the noninteracting Tonks gas. The latter model has been solved exactly and shows negative entropy at high coverage; see Equation (73). The entropy of the original lattice model is non-negative at all coverages and reduces to zero only when the coverage is full. It is found that one must subtract a term containing δ, the lattice spacing. By a proper subtraction, the lattice entropy has been shown to reproduce the continuum entropy in Equation (73) as δ → 0. It is a common practice to treat δ a constant, in which case the subtraction is harmless. However, there is no reason to assume δ to be state independent.
• One encounters inequalities such as in Equation (2), which creates problems in determining the entropy calorimetrically; the calorimetric determination of d_eQ and T₀ is not sufficient to obtain dS. In our approach based on internal variables [38,39], see in particular the discussion starting below Equation (3) and leading to Equation (6), we have dQ = d_eQ + d_iQ = T (t)dS(t), dW = d_eW + d_iW = P(t)dV(t) +⋯+ A(t) · dξ(t) so that the inequality dS > d_eQ/T₀ is replaced by an equality dS = dQ/T (t). Thus, both dQ and dW are expressed in terms of the fields of the body defined in Equation (35). This provides for tremendous simplification as internal equilibrium is very similar in many respect to equilibrium; see Section 3.6 and Ref. [38,39].
• For an isolated system in internal equilibrium (p_α = p for ∀m_α), just a single sample (or microstate) will suffice to determine the entropy as samples are unbiased. The entropy in this case is no different than the “entropy” − ln p of a single sample [52,55,56]:

  $$S(t)=(-p\ln p)W_0=-\ln p=\ln W_0,$$

  where W₀ represents W₀(Z₀(t)) or W₀(X₀). The entropy of the “single” microstate in this case will always satisfy the second law for W₀ continues to increase as the system relaxes.
• However, this simplicity is lost as soon as the system is not in internal equilibrium. Here, one must consider averaging over all microstates to evaluate the statistical entropy, which must satisfy the second law. In contrast, the negative of the index of probability −η(t) used in Equation (52), which has also been identified as the microstate entropy in modern literature [14–16], may increase or decrease with time. This is merely a consequence of probability conservation (see the second equation in Equation (49)), and is not related to second law, which refers to the behavior of the negative of the average index of probability.
• Some readers may think that our statistical formulation is no different than that used in the information theory. We disagree. For one, there is no concept of internal variables ξ₀ in the latter theory. Because of this, our approach allows us to consider three levels of description so that we can consider three different entropies S₀(Z₀(t),t), S₀(Z₀(t)) and S₀(X₀) satisfying the inequalities in Equation (56). The information theory can only deal with two levels of entropies. There is also no possibility of a residual entropy in the latter.
• We should remark here that the standard approach to calculate nonequilibrium entropy is to use the classical nonequilibrium thermodynamics [3] or its variant, which treats the entropy at the local level as an equilibrium state function. Moreover, it can only treat an inhomogeneous system in a NEQS but not a homogeneous NEQS. Furthermore, the correlation length plays no role in determining the “elemental volume” over which local equilibrium holds [6] (p. 335). As no internal variables are required, it is different from our approach which exploits internal variables. Our approach is also different from those approaches in which the entropy is treated as a function of fluxes [9]. As fluxes depend strongly on how the system relates to the medium, they cannot serve as state variables for the system in our approach; see SP1 and the discussion following it.

We suggest that our statistical Gibbs formulation can be applied to any NEQS. However, as there does not appear at present a way to experimentally measure the thermodynamic entropy in an arbitrary NEQS, there is no hope to compare the two entropies in all cases. Therefore, our final conclusion is presented as a conjecture.