Relaxation Processes and the Maximum Entropy Production Principle

Abstract

Spontaneous transitions of an isolated system from one macroscopic state to another (relaxation processes) are accompanied by a change of entropy. Following Jaynes’ MaxEnt formalism, it is shown that practically all the possible microscopic developments of a system, within a fixed time interval, are accompanied by the maximum possible entropy change. In other words relaxation processes are accompanied by maximum entropy production.

1. Introduction

E.T. Jaynes, claiming that our knowledge of the initial microscopic state of a system is not a matter of physics but of information theory, introduced information entropy to describe the behavior of macroscopic systems [1,2]. Keeping in mind that the macroscopic development of a system depends on its constraints rather than on the initial microscopic state, Jaynes used such constraints as the pillars of his theory. He proposed that the probabilities of states, under constraints, make the information entropy the maximum possible. This procedure is known as the MaxEnt formalism [3].

MaxEnt can be equally applied to equilibrium and nonequilibrium processes. It was applied by a number of authors [4,5,6] to nonequilibrium processes in order to determine the physical principle relevant for nonequilibrium thermodynamics. We single out Dewar’s [4] and Niven’s [5] approaches.

Dewar considered the ensemble of the trajectories (paths) in phase space. He maximized the information entropy over paths with macroscopic quantities as constraints (MaxEnt) and found that path probability is proportional to the exponential function of entropy production, when entropy production is expressed as the sum of products of fluxes and conjugated forces. Thus, the most probable evolution of a system is accompanied by the maximum possible entropy production. Some criticism of Dewar’s work can be found in references [7,8,9].

Niven [5] applied MaxEnt to stationary processes in a different way than Dewar. Values of fluxes are the main elements of his approach. He considered the mean values of fluxes as constraints. Assuming local equilibrium, he found, under certain additional assumptions, that entropy production must achieve maximal value.

Both Dewar and Niven assumed the existence of a stationary state, local equilibrium and entropy production defined as the sum of the products of thermodynamic forces and conjugated fluxes.

We noted in paper II that in a stationary nonequilibrium state the subject must interfere with a system in order to maintain such a state. In this paper, entitled paper III, we apply the MaxEnt formalism to relaxation processes. Our approach is free of the above-mentioned assumptions. Assuming that changes of entropy, within a fixed interval of time, are distributed so as to achieve maximum information entropy, we find that the evolution of an isolated system is accompanied by the maximum possible entropy production.

The paper is organized in the following manner. Section II is devoted to the Jaynes’ principle of maximum information entropy. In Section III, we apply this principle to the change of entropy of an isolated system. We find that the development of an isolated system (relaxation) is accompanied by the maximum production of entropy. In the Conclusions section we summarize our results.

2. Information Entropy and MaxEnt Formalism

E.T. Jaynes [1,2], having in mind that initial microscopic states cannot be determined, proposed that the statistics of any microscopic physical quantity, describing the time evolution of a system, can be determined by making use of information entropy [10]:

$$S_I=-\sum _i{w}_{i}ln{w}_i$$

(1)

Here, $w_i$ is the probability of the $i-th$ value of the considered microscopic quantity. Taking into account a definite number of constraints (conservation laws and/or the mean values of measured quantities ), the probability distribution is obtained by the maximization of entropy $S_I$. The constrained maximization of entropy $S_I$ is well known as the MaxEnt formalism. This maximum-entropy inference gives probability distribution as a measure of the amount of uncertainty. Jaynes [1,2] has pointed out that the very existence of definite macroscopic properties (states) is essentially related to sharp distributions: the overwhelming majority of microscopic evolutions correspond to the same macroscopic (deterministic) evolution of microscopic states.

We note that information entropy is a property of any distribution of probabilities and has, in principle, nothing to do with thermodynamic entropy that is a property of state of macroscopic system (see page 351 in reference [10]).

3. Relaxation Processes, Information Entropy and the MEP Principle

The relaxation process is the spontaneous approach of an isolated system toward its equilibrium state.

The initial problem in applying MaxEnt is the choice of events of interest. In the case of an isolated system, the initial microscopic state of the system defines the “phase space trajectories” (paths). Assuming that the initial microscopic states do not share a common path there is a one-to-one correspondence between initial states and paths.

Time evolution of the macroscopic state of a system is defined with the set $\left\{Q_i\right\}$ of mutually independent variables. To be more specific, these variables are fluxes, the main characteristic of non-equilibrium processes, or quantities related to fluxes.

We can define the entropy of an isolated system by means of microcanonical ensemble using either Boltzmann’s , $S=k_{B}lnW$, or Gibb’s, $S=-k_B{\sum }_i{p}_{i}ln{p}_i$ definition of the entropy. Here, $k_B$, W and $p_i$ are Boltzmann’s constant, statistical weight of the macroscopic state and probability of $i_{th}$ microscopic state, respectively. The unsatisfactory feature of both definitions, in the case of isolated systems, comes from the Louville’s theorem, which states that the number of microscopic states do not change in course of the time. In particular, one initial microstate will give just one final microstate. i.e., there is no increase of the number of microscopic states. Yet, according to the second law the number of states should increase in the course of time.

In order to resolve this paradox, Zurek [11] introduced the notion of the algorithmic entropy (randomness) as an objective property of the microscopic state. Algorithmic entropy is equal to the number of digits s of the shortest program that generates output that contains sufficient information, within required accuracy, about the microscopic state of the system. Momenta and positions of all particles are examples of such output in the case of systems described within classical mechanics. When s is interpreted as a binary representation of the integer, algorithmic entropy is $K={log}_{2}s$. Zurek has shown that the old problem of statistical mechanics, fundamental incompatibility of invariance of the physical laws on the change of the sign of time (reversibility) and second law of thermodynamics could be overcome by means of algorithmic entropy. However, since algorithmic entropy is tightly connected with determination of the microscopic state of the system of classical particles, it does not include the principle of the particle indistinguishability. Zurek was forced to modify the starting algorithm (see Appendix A in [11]) in order to get Sackur-Tetrode relation, the correct expression for free gas entropy. It is interesting to note that in order to find correct expression for entropy of ideal gases, Sackur and Tetrode were forced to use the quantum-mechanical concepts - the principle of particle indistiguishabilty and definition of the elementary volume ${\hslash }^3$ in phase space - in 1912, long before the formulation of quantum mechanics in 1925. In standard textbooks [12,13], one exploits uncertainty relation to define ${\hslash }^3$ as the elementary volume of the phase space. This process, known as coarse graining, enables one to cross from the continuous to the discrete microscopic states. This introduction of the discrete microscopic states enables one to define ensembles and corresponding statistics.

Each measuring process is defined by the measuring device and system, whose state is determined with the value of measured quantity. A measurement of any physical quantity is accompanied with uncertainty. The resulting uncertainty is a superposition of the uncertainties introduced by the measuring device and system. The uncertainty introduced due to the measuring instrument is within the resolution of the measuring instrument. We too exploit the coarse grain approach, but in a different way from the above described. We take the resolution of the measuring devices $\left\{\delta Q_i\right\}$ as the quantization step of the variable $\left\{Q_i\right\}$. The macroscopic state is defined with values of independent physical quantities in intervals $\{Q_i,Q_i+\delta Q_i\}$. We follow the Boltzmann definition of the entropy. The number of microstates W that have values of independent physical quantities within the intervals $\{Q_i,Q_i+\delta Q_i\}$ are the statistical weight of state defined with $\left\{Q_i\right\}$. The entropy of this state is $S=k_{B}lnW$. We assign this entropy to each microscopic state characterized with values of physical quantities in interval $\{Q_i,Q_i+\delta Q_i\}$. In the following we will write $\left\{Q_i\right\}$ for $\{Q_i,Q_i+\delta Q_i\}$.

The initial state is defined by a subensemble with fixed values $\left\{Q_i\left(t\right)\right\}$. During small fixed macroscopic interval of time $\Delta t$, the state $\left\{Q_i\right\}$ evolves into the state $\left\{Q_i(t+\Delta t)\right\}$, where $\left\{Q_i(t+\Delta t)\right\}$ are just discrete values of variables $\left\{Q_i\right\}$ . There is an essential difference between initial and final states. The microscopic states of the initial macroscopic state are elements of the subensemble with the variables $\left\{Q_i\right\}$ in the interval $\{Q_i\left(t\right)<Q_i<Q_i\left(t\right)+\delta Q_i\}$. On the other hand, there is an additional uncertainty of the variables $\left\{Q_i\right\}$ in the final state due to the different evolutions (paths) of the elements of the initial subensemble. The uncertainty of physical variables in the final state can be larger than the resolution of the measuring device. We assign this excess of uncertainty to the fluctuations of the measuring quantities $\left\{Q_i\left(t\right)\right\}$. To be more specific, the elements of the starting subensemble defined by $\{Q_i\left(t\right),Q_i\left(t\right)+\delta Q_i\}$ can end in different subensembles $\{Q_i(t+\Delta t)+j\delta Q_i,Q_i+(j+1)\delta Q_i\}$. Here, j assumes integer values. This additional uncertainty $j\ne 0$ is due to the different time evolution of the elements of the initial subensemble and can be in principle detected by the measuring instrument. Note that there is no fluctuation of entropy due to the uncertainty introduced by the measuring device.

The mean value of entropy is the result of averaging its value over subensembles defined with $\left\{Q_i\left(t\right)\right\}$. If the relaxation is close to equilibrium, the mean value of entropy change is equal to the change of locally defined thermodynamic entropy. The latter is defined in same way as in equilibrium thermodynamics [14,15].

We assume that changes of entropy satisfy the MaxEnt distribution. Then the measured (mean) value of the change of entropy is the only constraint in addition to the normalization of probabilities:

$$\overline{\Delta S}=\overline{S(t+\Delta t)}-S\left(t\right)$$

(2)

Here $S\left(t\right)$ is the entropy of the microscopic or macroscopic initial state, while $\overline{S(t+\Delta t)}$ and $\overline{\Delta S}$ are mean entropies of the final state of a system and the mean change of entropy, respectively. Due to the evolution of a system, entropy in final microscopic states can acquire different values, although entropy of initial microscopic states has same value.

Information entropy in Eq. (1) becomes

$$S_I=-\sum _{{(\Delta S)}_{min}}^{{(\Delta S)}_{max}}w(\Delta S)ln\left[w(\Delta S)\right]$$

(3)

Here $w(\Delta S)$ is the probability of the entropy change $\Delta S$, and ${(\Delta S)}_{min}$ and ${(\Delta S)}_{max}$ are the minimum and the maximum possible entropy changes, respectively. These extreme values are imposed by the dynamics of the system.

The constrained maximization of information entropy (3), with respect to $w(\Delta S)$, is performed according to standard procedure [16] by introducing Lagrange’s multipliers and looking for the maximum of the Lagrangian function

$$F=-\sum w(\Delta S)ln\left[w(\Delta S)\right]+\lambda \left(\sum w(\Delta S)-1\right)+\mu \sum w(\Delta S)\left[\Delta S-S(t+\Delta t)+S\left(t\right)\right]$$

(4)

The outcome of the maximization of F is

$$w(\Delta S)=C^{-1}exp(\mu \Delta S)$$

(5)

This result can be put into relationship with Einstein’s formula for a probability of fluctuations [17,18]. He proposed that the probability of fluctuation from equilibrium, in the isolated system, is $w(\Delta S)=C^{-1}exp(\Delta S/k_B)$. Here, $\Delta S<0$ is the decrease of the entropy from its maximum value. Comparing Einstein’s formula with Eq. (5) we get $\mu =1/k_B$.

We return back to the relaxation of the system. It comes from the foregoing expression that the maximum possible change of entropy is the most probable one. For a very short interval of time $\Delta t$, many different developments of the system are possible with likely probabilities. But as the time interval increases, the ratio $exp(\Delta S/k_B)$ becomes extremely sharp function at the value of maximum possible entropy change. This result is in accordance with the fluctuation theorem [19,20]. In the case of large time intervals we can say that, apart from relatively small fluctuations, the relaxation of the system is accompanied by maximum possible change of entropy. In other words, the most probable time evolution of the isolated system is accompanied by the maximum entropy production (MEP). Martyushev and Seleznev [7] provide a plausible derivation of the MEP principle in the case of an isolated system. They argue that an isolated system reaches a state of maximum entropy within relaxation time (or within the order of relaxation time) according to the second law of thermodynamics. Therefore, the change of entropy within this time interval is the maximum possible. The MaxEnt formalism applied to the change of entropy of an isolated system supports the plausible derivation of the MEP principle by Martyushev and Seleznev.

In short, relaxation processes in isolated systems are in accordance with the MEP principle. An isolated system will develop in such a way so as to produce the maximum possible entropy. This is an addition of the second law of thermodynamics. While the second law states only that an isolated system will approach the state of largest entropy, the MEP principle states that the system will approach that state with the maximum possible speed [4,5,7,19,21,22].

4. Conclusions

The basic assumption of this paper is that an entropy change in relaxation processes obeys the MaxEnt formalism. Therefore, the largest possible increase in entropy is the most probable increase of entropy. This means that a system evolves with maximum entropy production. In this way, the MEP principle becomes an addition to the second law of thermodynamics. While the second law requires that an isolated system will find itself in the state of maximum entropy, the MEP principle asserts that the system reaches this state as quickly as possible.