System Entropy Measurement of Stochastic Partial Differential Systems

Abstract

System entropy describes the dispersal of a system’s energy and is an indication of the disorder of a physical system. Several system entropy measurement methods have been developed for dynamic systems. However, most real physical systems are always modeled using stochastic partial differential dynamic equations in the spatio-temporal domain. No efficient method currently exists that can calculate the system entropy of stochastic partial differential systems (SPDSs) in consideration of the effects of intrinsic random fluctuation and compartment diffusion. In this study, a novel indirect measurement method is proposed for calculating of system entropy of SPDSs using a Hamilton–Jacobi integral inequality (HJII)-constrained optimization method. In other words, we solve a nonlinear HJII-constrained optimization problem for measuring the system entropy of nonlinear stochastic partial differential systems (NSPDSs). To simplify the system entropy measurement of NSPDSs, the global linearization technique and finite difference scheme were employed to approximate the nonlinear stochastic spatial state space system. This allows the nonlinear HJII-constrained optimization problem for the system entropy measurement to be transformed to an equivalent linear matrix inequalities (LMIs)-constrained optimization problem, which can be easily solved using the MATLAB LMI-toolbox (MATLAB R2014a, version 8.3). Finally, several examples are presented to illustrate the system entropy measurement of SPDSs.

1. Introduction

Information entropy is considered a measure of uncertainty and its maximization guarantees the best solutions for the maximal uncertainty [1,2,3,4,5]. Information entropy characterizes uncertainty caused by random parameters of a random system and measurement noise in the environment [6]. Entropy has been used for information retrieval such as systemic parametric and nonparametric estimation based on real data, which is an important topic in advanced scientific disciplines such as econometrics [1,2], financial mathematics [4], mathematical statistics [3,4,6], control theory [5,7,8], signal processing [9], and mechanical engineering [10,11]. Methods developed within this framework consider model parameters as random quantities and employ the informational entropy maximization principle to estimate these model parameters [6,9].

System entropy describes disorder or uncertainty of a physical system and can be considered to be a significant system property [12]. Real physical systems are always modeled using stochastic partial differential dynamic equation in the spatio-temporal domain [12,13,14,15,16,17]. The entropy of thermodynamic systems has been discussed in [18,19,20]. The maximum entropy generation of irreversible open systems was discussed in [20,21,22]. The entropy of living systems was discussed in [19,23]. The system entropy of stochastic partial differential systems (SPDSs) can be measured as the logarithm of system randomness, which can be obtained as the ratio of output signal randomness to input signal randomness from the entropic point of view. Therefore, if system randomness can be measured, the system entropy can be easily obtained from its logarithm. The system entropy of biological systems modeled using ordinary differential equations was discussed in [24]. However, since many real physical and biological systems are modeled using partial differential dynamic equations, in this study, we will discuss the system entropy of SPDSs. In general, we can measure the system entropy from the system characteristics of a system without measuring the system signal or input noise. For example, a low-pass filter, which is a system characteristic, can be determined from its transfer function or system’s frequency response without measuring its input/output signal. Hence, in this study, we will measure the system entropy of SPDSs from the system’s characteristics. Actually, many real physical and biological systems are only nonlinear, such as the large-scale systems [25,26,27,28], the multiple time-delay interconnected systems [29], the tunnel diode circuit systems [30,31], and the single-link rigid robot systems [32]. Therefore, we will also discuss the system entropy of nonlinear system as a special case in this paper.

However, because direct measurement of the system entropy of SPDSs in the spatio-temporal domain using current methods is difficult, in this study, an indirect method for system entropy measurement was developed through the minimization of its upper bound. That is, we first determined the upper bound of the system entropy and then decreased it to the minimum possible value to achieve the system entropy. For simplicity, we first measure the system entropy of linear stochastic partial differential systems (LSPDSs) and then the system entropy of nonlinear stochastic partial differential systems (NSPDSs) by solving a nonlinear Hamilton–Jacobi integral inequality (HJII)-constrained optimization problem. We found that the intrinsic random fluctuation of SPDSs will increase the system entropy.

To overcome the difficulty in solving the system entropy measurement problem due to the complexity of the nonlinear HJII, a global linearization technique was employed to interpolate several local LSPDSs to approximate a NSPDS; a finite difference scheme was employed to approximate a partial differential operator with a finite difference operator at all grid points. Hence, the LSPDSs at all grid points can be represented by a spatial stochastic state space system and the system entropy of the LSPDSs can be measured by solving a linear matrix inequalities (LMIs)-constrained optimization problem using the MATLAB LMI toolbox [12]. Next, the NSPDSs at all grid points can be represented by an interpolation of several local linear spatial state space systems; therefore, the system entropy of NSPDSs can be measured by solving the LMIs-constrained optimization problem.

Finally, based on the proposed systematic analysis and measurement of the system entropy of SPDSs, two system entropy measurement simulation examples of heat transfer system and biochemical system are given to illustrate the proposed system entropy measurement procedure of SPDSs.

2. General System Entropy of LSPDSs

For simplicity, we will first calculate the entropy of linear partial differential systems (LPDSs). Then, the result will be extended to the measure of NSPDSs. Consider the following LPDS [15,16]:

$$\begin{array}{l}\frac{\partial y(x,t)}{\partial t}=\kappa {\nabla }^{2}y(x,t)+Ay(x,t)+Bv(x,t)\\ z(x,t)=Cy(x,t),\end{array}$$

(1)

where $x={[x_1{x}_2]}^T\in U$ is the space variable, $y(x,t)={[y_1(x,t),\dots ,y_n(x,t)]}^T\in {\mathrm{ℝ}}^n$ is the state variable, $v(x,t)={[v_1(x,t),\dots ,v_l(x,t)]}^T\in {\mathrm{ℝ}}^l$ is the random input signal, and $z\left(x,t\right)=[z_1\left(x,t\right)$,… $z_m\left(x,t\right){]}^T\in {\mathrm{ℝ}}^m$ is the output signal. $x$ and $t$ are the space and time variable, respectively. The space domain $U$ is a two-dimensional bounded domain. The system coefficients are $\kappa \in {\mathrm{ℝ}}^{n\times n}$, $A\in {\mathrm{ℝ}}^{n\times n}$, $B\in {\mathrm{ℝ}}^{n\times l}$, and $C\in {\mathrm{ℝ}}^{m\times n}$. The Laplace (diffusion) operator ${\nabla }^2$ is defined as follows [15,16]:

$$\begin{array}{l}{\nabla }^{2}y(x,t):={\displaystyle \sum _{k=1}^2\frac{{\partial }^{2}y(x,t)}{\partial {x_k}^2}}\\ \frac{{\partial }^{2}y(x,t)}{\partial {x_k}^2}:={\left[\frac{{\partial }^2}{\partial {x_k}^2}{y}_1(x,t),\mathrm{...},\frac{{\partial }^2}{\partial {x_k}^2}{y}_n(x,t)\right]}^T\in {\mathrm{ℝ}}^n.\end{array}$$

(2)

Suppose that the initial value $y(x,0):=y_0(x)$. For simplicity, the boundary condition is usually given by the Dirichlet boundary condition, i.e., $y(x,t)=$ a constant on $\partial U$, or by the Neumann boundary condition $\nabla y(x,t)\cdot \stackrel{\rightharpoonup }{n}=0$ on $\partial U$, where $\stackrel{\rightharpoonup }{n}$ is a normal vector to the boundary $\partial U$ [15,16]. The randomness of the random input signal is measured by the average energy in the domain $U$ and the entropy of the random input signal is measured by the logarithm of the input signal randomness as follows [1,2,24]:

$$-\log \frac{1}{U\cdot t_f}E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\},$$

where E denotes the expectation operator and t_f denotes the period of the random input signal, i.e., $v(x,t)\in U\times [0,t_f]$. Similarly, the entropy of the random output signal $z(x,t)$ is obtained as:

$$-\log \frac{1}{U\cdot t_f}E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}.$$

In this situation, the system entropy S of the LPDS given in Equation (1) can be obtained from the differential entropy between the output signal and input signal, i.e., input signal entropy minus output signal entropy, or the net signal entropy of the LPDS [33]:

$$\begin{array}{lll}S& =& \log \frac{1}{U\cdot t_f}E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}\\ & & -\log \frac{1}{U\cdot t_f}E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}\\ & =& \text{log}\frac{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}}{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}}=-\log \frac{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}}{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}}.\end{array}$$

(3)

Let us denote the system randomness as the following normalized randomness:

$$S_0=\frac{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}}{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}},\text{ if }{y}_0(x)=0.$$

(4)

Then, the system entropy $S=\log S_0.$ That is, if system randomness can be obtained, the system entropy can be determined from the logarithm of the system randomness. Therefore, our major work of measuring the entropy of the LPDS given in Equation (1) first involves the calculation of the system randomness $S_0$ given in Equation (4). However, it is not easy to directly calculate the normalized randomness $S_0$ in Equation (4) in the spatio-temporal domain. Suppose there exists an upper bound of $S_0$ as follows:

$$S_0=\frac{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}}{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}}\le {\overline{S}}_0,$$

(5)

and we will determine the condition with that $S_0$ has an upper bound ${\overline{S}}_0$. Then, we will decrease the value of the upper bound ${\overline{S}}_0$ as small as possible to approach $S_0$, and then obtain the system entropy using $S=\log S_0$.

Remark 1. (i) From the system entropy of LPDS Equation (1), if the randomness of the input signal $v(x,t)$ is larger than the randomness of the output signal $z(x,t)$, i.e.,:

$$E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}>E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\},$$

(6)

then $S_0<1$ and $S<0$. A negative system entropy implies that the system can absorb external energy to increase the structure order of the system. All the biological systems are of this type, and according to Schrödinger’s viewpoint, biological systems consume negative entropy, leading to construction and maintenance of their system structures, i.e., life can access negative entropy to produce high structural order. (ii) If the randomness of the output signal $z(x,t)$ is larger than the randomness of the input signal $v(x,t)$, i.e.,:

$$E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}<E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\},$$

(7)

then $S_0>1$ and $S>0$. A positive system entropy indicates that the system structure disorder increases and the system can disperse entropy to the environment. (iii) If the randomness of the input signal $v(x,t)$ is equal to the randomness of the system signal $z(x,t)$, i.e.,:

$$E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}=E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\},$$

(8)

then $S_0=1$ and $S=0$. In this case, the system structure order is maintained constantly with zero system entropy. (iv) If the initial value $y_0(x)\ne 0,$ then the system randomness $S_0$ in Equation (4) should be modified as:

$$S_0=\frac{E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}}{{{\displaystyle \int }}_U\frac{V\left(y_0(x)\right)}{S}dx+E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{v}^T(x,t)v(x,t)}dt}dx\right\}}\le {\overline{S}}_0$$

(9)

for a positive Lyapunov function $V(y(x,t))>0$, and the randomness due to the initial condition $y_0(x)\ne 0$ should be considered a type of input randomness.

Based on the upper bound ${\overline{S}}_0$ of the system randomness as given in Equation (5), we get the following result:

Proposition 1. For the LPDS in Equation (1), if the following HJII holds for a Lyapunov function $V(y(x,t))>0$ and with $V(0)=0$:

$$\begin{array}{l}{{\displaystyle \int }}_U[y^T(x,t)C^{T}Cy(x,t)+{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^T\left(\kappa {\nabla }^{2}y(x,t)+Ay(x,t)\right)\\ +\frac{1}{4{\overline{S}}_0}{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^{T}B{B}^T\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)]dx<0,\end{array}$$

(10)

then the system randomness $S_0$ has an upper bound ${{\displaystyle \overline{S}}}_0$ as given in Equation (5).

Proof. See Appendix A.                   ☐

Since ${\overline{S}}_0$ is the upper bound of $S_0$, it can be calculated by solving the following HJII-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{V\left(y\left(x,t\right)\right)>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the HJII in Equation (10)}.\end{array}$$

(11)

Consequently, we can calculate the system entropy using $S=\log S_0$.

Remark 2. If the system in Equation (1) is free of a partial differential term ${\nabla }^{2}y(x,t)$, i.e., in the case of the following conventional linear dynamic system:

$$\begin{array}{l}\frac{dy(t)}{dt}=Ay(t)+Bv(t)\\ z(t)=Cy(t),\end{array}$$

(12)

then the system entropy of linear dynamic system in Equation (12) is written as [24]:

$$S=\log \frac{E\left\{{{\displaystyle \int }}_0^{t_f}{z}^T(t)z(t)dt\right\}}{E\left\{{{\displaystyle \int }}_0^{t_f}{v}^T(t)v(t)dt\right\}}.$$

(13)

Therefore, the result of Proposition 1 is modified as the following corollary.

Corollary 1. For the linear dynamic system in Equation (12), if the following Riccati-like inequality holds for a positive definite symmetric $P>0$:

$$PA+A^{T}P+C^{T}C+\frac{1}{{\overline{S}}_0}PB{B}^{T}P<0,$$

(14)

or equivalently (by the Schur complement [12]):

$$\left[\begin{array}{cc}PA+A^{T}P+C^{T}C& PB\\ B^{T}P& -{\overline{S}}_{0}I\end{array}\right]<0,$$

(15)

then the system randomness $S_0$ of the linear dynamic system in Equation (12) has an upper bound ${\overline{S}}_0$.

Proof. See Appendix B.                       ☐

Thus, the randomness $S_0$ of the linear dynamic system in Equation (12) is obtained by solving the following LMI-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{P>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the LMI in Equation (15)}.\end{array}$$

(16)

Hence, the system entropy of the linear dynamic system in Equation (12) can be calculated using $S=\log S_0$. The LMI-constrained optimization problem given in Equation (16) is easily solved by decreasing ${\overline{S}}_0$ until no positive definite solution $P$ exists for the LMI given in Equation (15), which can be solved using the MATLAB LMI toolbox [12]. Substituting $S_0$ into ${\overline{S}}_0$ in Equation (14), we get:

$$C^{T}C+\frac{1}{S_0}PB{B}^{T}P<-(PA+A^{T}P).$$

(17)

The right hand side of Equation (17) can be considered as an indication of the system stability. If the eigenvalues of $A$ are more negative (more stable), i.e., the right hand side is more large, then $S_0$ and the system entropy $S$, are smaller. Obviously, the system entropy is inversely related to the stability of the dynamic system. If $A$ is fixed, then the increase in input signal coupling $B$ may increase $S_0$ and $S$.

Remark 3. If the LPDS in Equation (1) suffers from the following intrinsic random fluctuation:

$$\begin{array}{l}\frac{\partial y(x,t)}{\partial t}=\kappa {\nabla }^{2}y(x,t)+Ay(x,t)+Bv(x,t)+Hy(x,t)w(x,t)\\ z(x,t)=Cy(x,t),\end{array}$$

(18)

where the constant matrix $H\in {\mathrm{ℝ}}^{n\times n}$ denotes the deterministic part of the parametric variation of system matrix $A$ and $w(x,t)\in \mathrm{ℝ}$ is a stationary spatio-temporal white noise to denote the random source of intrinsic parametric variation [34,35], then the LSPDS in Equation (18) can be rewritten in the following $\text{It}\widehat{\text{o}}$ differential form:

$$\begin{array}{l}\partial y(x,t)=\left(\kappa {\nabla }^{2}y(x,t)+Ay(x,t)+Bv(x,t)\right)\partial t+Hy(x,t)\partial W(x,t)\\ z(x,t)=Cy(x,t),\end{array}$$

(19)

where $\partial W(x,t)=w(x,t)\partial t$ with $W(x,t)$ being the Wiener process or Brownian motion in a zero mean Gaussian random field with unit variance at each location x [15].

For the LSPDS in Equation (19), we get the following result.

Proposition 2. For the LSPDS in Equation (19), if the following HJII holds for a Lyapunov function $V(y(x,t))>0$ with $V(0)=0$:

$$\begin{array}{l}E\{{\displaystyle {\int }_U[y^T(x,t)C^{T}Cy(x,t)+{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^T\left(\kappa {\nabla }^{2}y(x,t)+Ay(x,t)\right)}\\ +\frac{1}{4{\overline{S}}_0}{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^{T}B{B}^T\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)\\ +\frac{1}{2}{y}^T(x,t)H^T{\left(\frac{{\partial }^{2}V\left(y(x,t)\right)}{\partial y^2}\right)}^{T}Hy(x,t)]dx\}<0,\end{array}$$

(20)

then the system randomness $S_0$ has an upper bound ${\overline{S}}_0$ as given in (5).

Proof. See Appendix C.          ☐

Since ${\overline{S}}_0$ is the upper bound of $S_0$, it could be calculated by solving the following HJII-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{\mathrm{V}\left(y(x,t)\right)>0}{\min }{\overline{S}}_0\\ \text{subject to the HJII in Equation (20)}.\end{array}$$

(21)

Hence, the system entropy of LSPDS in Equations (18) or (19) could be obtained using $S=\log S_0$, where $S_0$ is the system randomness solved from Equations (21).

Remark 4. Comparing the HJII in Equation (20) with the HJII in Equation (10) and replacing ${\overline{S}}_0$ with $S_0$, we find that Equation (20) has an extra positive term $(1/2)y^T(x,t)H^T{({\partial }^{2}V(y(x,t))/\partial y^2)}^{T}Hy(x,t)$ due to the intrinsic random parametric fluctuation given in Equation (18). To maintain the left-hand side of Equation (20) as negative, the system randomness $S_0$ in Equation (20) must be larger than the randomness $S_0$ in Equation (10), i.e., the system entropy of the LPDS in Equations (18) or (19) is larger than that of the LPDS in Equation (1) because the intrinsic random parametric variation $H(y(x,t))w(x,t)$ in Equation (18) can increase the system randomness and the system entropy.

Remark 5. If the LSPDS in Equation (18) is free of the partial differential term ${\nabla }^{2}y(x,t)$, i.e., in the case of the conventional linear dynamic system:

$$\begin{array}{l}\frac{dy(t)}{dt}=Ay(t)+Bv(t)+Hy(t)w(t)\\ z(t)=Cy(t),\end{array}$$

(22)

or the following $\text{It}\widehat{\text{o}}$ form:

$$\begin{array}{l}dy(t)=\left(Ay(t)+Bv(t)\right)dt+Hy(t)dW(t)\\ z(t)=Cy(t),\end{array}$$

(23)

then we modify Proposition 2 as the following corollary.

Corollary 2. For the linear dynamic system in Equations (22) or (23), if the following Riccati-like inequality holds for a positive definite symmetric $P=P^T>0$:

$$PA+A^{T}P+C^{T}C+H^{T}PH+\frac{1}{{\overline{S}}_0}PB{B}^{T}P<0$$

(24)

or equivalently:

$$\left[\begin{array}{cc}PA+A^{T}P+C^{T}C+H^{T}PH& PB\\ B^{T}P& -{\overline{S}}_{0}I\end{array}\right]<0,$$

(25)

then the system randomness $S_0$ of the linear dynamic system in Equations (22) or (23) has a upper bound ${\overline{S}}_0$.

Proof. See Appendix D.          ☐

Therefore, the system randomness $S_0$ of the linear stochastic system in Equations (22) or (23) can be obtained by solving the following LMI-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{P>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the LMI in Equation (25)},\end{array}$$

(26)

Hence, the system entropy Equation (13) of the linear stochastic system in Equations (22) or (23) can be calculated using $S=\log S_0$, where the system randomness $S_0$ is the optimal solution of Equation (26).

By substituting $S_0$ calculated by Equation (26) into Equation (24), we can get:

$$C^{T}C+H^{T}PH+\frac{1}{ S_0}PB{B}^{T}P<-\left(PA+A^{T}P\right).$$

(27)

Remark 6. Comparing Equation (27) with Equation (17), it can be seen that the term $H^{T}PH$ due to the intrinsic random parametric fluctuation $Hy(t)w(t)$ in Equation (22) can increase the system randomness $S_0$ which consequently increases the system entropy $S$.

3. The System Entropy Measurement of LSPDSs via a Semi-Discretization Finite Difference Scheme

Even though the entropy of the linear systems in Equations (12) and (22) can be easily measured by solving the optimization problem in Equations (16) and (26), respectively, using the LMI toolbox in MATLAB, it is still not easy to solve the HJII-constraint optimization problem in Equations (11) and (21) for the system entropy of the LPDS in Equation (1) and the LSPDS in Equation (18), respectively. To simplify this system entropy problem, the main method is obtaining a more suitable spatial state space model to represent the LPDSs. For this purpose, the finite difference method and the Kronecker product are used together in this study. The finite difference method is employed to approximate the partial differential term ${\nabla }^{2}y(x,t)$ in Equation (1) in order to simplify the measurement procedure of entropy [14,16].

Consider a typical mesh grid as shown in Figure 1. The state variable $y(x,t)$ is represented by $y_{k,l}(t)\in {\mathrm{ℝ}}^n$ at the grid node $x_{k,l}(x_1=k{\Delta }_x,x_2=l{\Delta }_x)$, where $k=1,\dots N_1$ and $l=1,\dots N_2$, i.e., $y(x,t){|}_{x=x_{k,l}}=y_{k,l}(t)$ at the grid point $x_{k,l}$, and the finite difference approximation scheme for the partial differential operator can be written as follows [14,16]:

$$\kappa {\nabla }^{2}y(x,t)\simeq \kappa \frac{y_{k+1,l}(t)+y_{k-1,l}(t)-2y_{k,l}(t)}{{\Delta }_x^2}+\kappa \frac{y_{k,l+1}(t)+y_{k,l-1}(t)-2y_{k,l}(t)}{{\Delta }_x^2}.$$

(28)

.

Based on the finite difference approximation in Equation (28), the LPDS in Equation (1) can be represented by the following finite difference system:

$$\begin{array}{ll}\frac{d}{dt}{y}_{k,l}(t)& \simeq \kappa \frac{1}{{\Delta }_x^2}[y_{k+1,l}(t)+y_{k-1,l}(t)+y_{k,l+1}(t)+y_{k,l-1}(t)-4y_{k,l}(t)]\\ & +A{y}_{k,l}(t)+B{v}_{k,l}(t),k=1,\dots N_1, l=1,\dots N_2,\end{array}$$

(29)

$\text{where }{y}_{k,l}(t)=y(x,t){|}_{x=x_{k,l}},v_{k,l}(t)=v(x,t){|}_{x=x_{k,l}}$.

Let us denote:

$$T_{k,l}{y}_{k,l}(t)=\frac{1}{{\Delta }_x^2}[y_{k+1,l}(t)+y_{k-1,l}(t)+y_{k,l+1}(t)+y_{k,l-1}(t)-4y_{k,l}(t)],$$

(30)

then we get:

$$\begin{array}{l}\frac{d}{dt}{y}_{k,l}(t)\simeq \kappa T_{k,l}{y}_{k,l}(t)+A{y}_{k,l}(t)+B{v}_{k,l}(t)\\ z_{k,l}(t)=C{y}_{k,l}(t).\end{array}$$

(31)

For the simplification of entropy measurement for the LPDS in Equation (1), we will define a spatial state vector $y_{k,l}(t)\in {\mathrm{ℝ}}^n$ at all grid node in Figure 1. For the Dirichlet boundary conditions [16], the values of $y_{k,l}(t)$ at the boundary are fixed. For example, $y(x,t)=0$, where $x\in \partial U$. We have $y_{k,l}(t)=0$ at $k=0, N_1+1$ or $l=0, N_2+1$. Therefore, the spatial state vector $y(t)\in {\mathrm{ℝ}}^{nN}$ for state variables at all grid nodes is defined as follows:

$$y(t)=[y_{1,1}^T(t),\mathrm{...},y_{k,1}^T(t),\mathrm{...},y_{N_1,1}^T(t),\mathrm{...},y_{k,l}^T(t),\mathrm{...},y_{1,N_2}^T(t),\mathrm{...},y_{k,N_2}^T(t),\mathrm{...},y_{N_1,N_2}^T(t)],$$

(32)

where $N:=N_1\times N_2$. Note that $N$ is the dimension of the vector $y_{k,l}(t)$ for each grid node and $N_1\times N_2$ is the number of grid nodes. For example, let $N_1=2$ and $N_2=2$, then we have $y(t)=[y_{1,1}^T(t),y_{1,2}^T(t),$$y_{2,1}^T(t),y_{2,2}^T(t){]}^T\in {\mathrm{ℝ}}^{4n}$. To simplify the index of the node $y_{k,l}(t)\in {\mathrm{ℝ}}^n$ in the spatial state vector $y(t)\in {\mathrm{ℝ}}^{nN}$, we will denote the symbol $y_j(t)\in {\mathrm{ℝ}}^n$ to replace $y_{k,l}(t)$. Note that the index $j$ is from 1 to N, i.e.:

$$y_1(t):=y_{1,1}(t),y_2(t):=y_{2,1}(t),\mathrm{...},y_j(t):=y_{k,l}(t),\mathrm{...},y_N(t):=y_{N_1,N_2}(t),$$

where $j=(l-1)N_1+k$ in Equation (32). Thus, the linear difference model of two indices in Equation (31) could be represented with only one index as follows:

$$\begin{array}{l}\frac{d}{dt}{y}_j(t)=\kappa T_j{y}_j(t)+A{y}_j(t)+B{v}_j(t),j=1,2,\mathrm{...},N\\ z_j(t)=C{y}_j(t),\end{array}$$

(33)

where $v_j(t)=v_{k,l}(t)$ with $j=(l-1)N_1+k$ and $T_j$ is defined as follows:

$$\begin{array}{l}{T}_{j}y\left(t\right)=\frac{1}{{\Delta }_x^2}\left[O_n\mathrm{...}{O}_n{I}_n{O}_n\mathrm{...}{O}_n{I}_n-4I_n{I}_n{O}_n\mathrm{...}{O}_n{I}_n{O}_n\mathrm{...}{O}_n\right]\\ \text{Position\hspace{1em}\hspace{1em}1\hspace{1em}\hspace{1em}\hspace{1em}}j-N_{1}j-1jj+1j+N_1{N}_1{N}_2,\end{array}$$

(34)

where $O_n$ and $I_N$ denote the $n\times n$ zero matrix and $N\times N$ identity matrix, respectively.

We will collect all states $y_j(t)$ of the grid nodes given in Equation (33) to the spatial state vector given in Equation (32). The Kronecker product can be used to simplify the representation. Using the Kronecker product, the systems at all grid nodes given in Equation (33) can be represented by the following spatial state space system (i.e., the linear dynamic systems of Equation (33) at all grid points within domain $U$ in Figure 1 are represented by a spatial state space system [14]):

$$\begin{array}{l}\frac{dy(t)}{dt}=\{[I_N\otimes \kappa ]T+[I_N\otimes A]\}y(t)+[I_N\otimes B]v(t)\\ z(t)=[I_N\otimes C]y(t),\end{array}$$

(35)

where $T=[T_1^T\dots T_N^T]\in {\mathrm{ℝ}}^{nN\times nN}$, $v(t)={(v_1(t)\dots v_N(t))}^T\in {\mathrm{ℝ}}^{lN}$, and $I_N\otimes \kappa$ denotes the Kronecker product between $I_N$ and $\kappa$.

Definition 1. [17,36]: Let $M\in {\mathrm{ℝ}}^{a\times b}$, $N\in {\mathrm{ℝ}}^{c\times d}$. Then the Kronecker product of $M$ and $N$ is defined as the following matrix:

$$M\otimes N=\left[\begin{array}{ccc}{m}_{11}N& \cdots & m_{1b}N\\ ⋮& \ddots & ⋮\\ m_{a1}N& \cdots & m_{ab}N\end{array}\right]\in {\mathrm{ℝ}}^{ac\times bd}.$$

Remark 7. Since the spatial state vector $y(t)$ in Equation (32) is used to represent $y(x,t)$ at all grid points, in this situation, $E\left\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}{z}^T(x,t)z(x,t)}dt}dx\right\}$, $E\left\{{\displaystyle {\int }_{U}V(y_0(x))dx}\right\}$, and $E\{{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}v}}}^T(x,t)v(x,t)$ $dtdx\}$ in the measurement of system randomness in Equations (5) or (9) could be modified by the temporal forms $E\left\{{\displaystyle {\int }_0^{t_f}{z}^T(t)z(t){\Delta }_x^2}dt\right\}$, $E\{V(y(0)){\Delta }_x^2\}$, and $E\left\{{\displaystyle {\int }_0^{t_f}{v}^T(t)v(t)}{\Delta }_x^{2}dt\right\}$, respectively, for the spatial state space system in (35), where the Lyapunov function $V(y(t))$ is related to the Lyapunov function $V(y(x,t))$ as $V(y(t))=$${\displaystyle {\sum }_{j=1}^{N}V(y_j(t))}$. Therefore, for the spatial state space system in Equation (35), the system randomness in Equations (5) or (9) is modified as follows:

$$S_0=\frac{E\left\{{{\displaystyle \int }}_0^{t_f}{z}^T(t)z(t)dt\right\}}{E\left\{{{\displaystyle \int }}_0^{t_f}{v}^T(t)v(t)dt\right\}}\le {\overline{S}}_0$$

(36)

or:

$$S_0\le \frac{E\left\{{{\displaystyle \int }}_0^{t_f}{z}^T(t)z(t)dt\right\}}{E\left\{\frac{V\left(y(0)\right)}{S}+{{\displaystyle \int }}_0^{t_f}{v}^T(t)v(t)dt\right\}}\le {\overline{S}}_0,\text{ for }y(0)\ne 0.$$

(37)

Hence, our entropy measurement problem of the LPDS in Equation (1) becomes the measurement of the entropy of the spatial state system Equation (35), as given below.

Proposition 3. For the linear spatial state space system in Equation (35), if the following Riccati-like inequality holds for a positive definite matrix $\overline{P}>0$:

$$\overline{P}\overline{A}+{\overline{A}}^T\overline{P}+{\overline{C}}^T\overline{C}+\overline{I}+\frac{1}{{\overline{S}}_0}\overline{P}\overline{B}{\overline{B}}^T\overline{P}<0$$

(38)

or equivalently:

$$\left[\begin{array}{cc}\overline{P}\overline{A}+{\overline{A}}^T\overline{P}+{\overline{C}}^T\overline{C}& \overline{P}\overline{B}\\ {\overline{B}}^T\overline{P}& -{\overline{S}}_{0}I\end{array}\right]<0,$$

(39)

where $\overline{A}=[I_N\otimes \kappa ]T+[I_N\otimes A], \overline{B}=[I_N\otimes B],\overline{C}=[I_N\otimes C]$ , then the system randomness $S_0$ in Equations (36) or (37) of linear spatial state space system in Equation (35) has the upper bound ${\overline{S}}_0$.

Proof. The proof is similar to the proof of Corollary 1 in Appendix B and can be obtained by replacing $A,B,C$, and $P$ with $\overline{A},\overline{B},\overline{C}$, and $\overline{P}$, respectively.      ☐

Therefore, the randomness $S_0$ of the linear spatial state space system in Equation (35) can be obtained by solving the following LMI-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{\overline{P}>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the LMI in Equation (39)}.\end{array}$$

(40)

Hence, the system entropy $S$ of the linear spatial state space system in Equation (33) can be calculated using $S=\log S_0$.

Remark 8. (i) The Riccati-like inequality in Equation (38) or the LMI in Equation (39) is an approximation of the HJII in Equation (10) with the finite difference scheme given in Equation (28). If the finite difference, shown in Equation (28), ${\Delta }_x\to 0$, then $S_0$ in Equation (40) will approach $S_0$ in Equation (11). (ii) Substituting $S_0$ into Equation (38), we get:

$${\overline{C}}^T\overline{C}+\frac{1}{S_0}\overline{P}\overline{B}{\overline{B}}^T\overline{P}<-(\overline{P}\overline{A}+{\overline{A}}^T\overline{P}).$$

(41)

If the eigenvalues of $\overline{A}$ are more negative (more stable), the randomness$S_0$ as well as the entropy $S$ is smaller. Similarly, the LSPDS in Equation (18) can be approximated by the following stochastic spatial state space system via finite difference scheme [14]:

$$\begin{array}{l}dy(t)=\{[I_N\otimes \kappa ]Ty(t)dt+[I_N\otimes A]y(t)dt\}+[I_N\otimes B]v(t)+[I_N\otimes H]y(t)\circ dW\\ z(t)=[I_N\otimes C]y(t),\end{array}$$

(42)

where $dW=[d{W}_1(t)\dots d{W}_N(t)]\in {\mathrm{ℝ}}^{nN}$, and the Hadamard product of matrices (or vectors) $X={[X_{ij}]}_{m\times n}$ and $Y={[Y_{ij}]}_{m\times n}$ of the same size is the entry-wise product denoted as $X\circ Y={[X_{ij}{Y}_{ij}]}_{m\times n}$.

Then we can get the following result.

Corollary 3. For the linear stochastic spatial state space system Equation (42), if the following Riccati-like inequality holds for a positive definite symmetric $\overline{P}>0$:

$$\overline{P}\overline{A}+{\overline{A}}^T\overline{P}+{\overline{C}}^T\overline{C}+{\overline{H}}^T\overline{P}\overline{H}+\frac{1}{{\overline{S}}_0}\overline{P}\overline{B}{\overline{B}}^T\overline{P}<0,$$

(43)

or equivalently, the following LMI has a positive definite symmetric solution $\overline{P}>0$:

$$\left[\begin{array}{cc}\overline{P}\overline{A}+{\overline{A}}^T\overline{P}+{\overline{C}}^T\overline{C}+{\overline{H}}^T\overline{P}\overline{H}& \overline{P}\overline{B}\\ {\overline{B}}^T\overline{P}& -{\overline{S}}_{0}I\end{array}\right]<0,$$

(44)

then the system randomness $S_0$ of the stochastic state space system in Equation (42) has an upper bound ${\overline{S}}_0$, where $\overline{H}=[I_N\otimes H]$.

Proof. The proof is similar to the proof of Corollary 2 in Appendix D.               ☐

Therefore, the system randomness $S_0$ of the linear stochastic state space system Equation (42) can be obtained by solving the following LMI-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{\overline{P}>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the LMI in (44)},\end{array}$$

(45)

and hence the system entropy $S$ of the stochastic spatial state space system in Equation (42) can be obtained using $S=\log S_0$. Substituting $S_0$ into (43), we get:

$${\overline{C}}^T\overline{C}+{\overline{H}}^T\overline{P}\overline{H}+\frac{1}{S_0}\overline{P}\overline{B}{\overline{B}}^T\overline{P}<-(\overline{P}\overline{A}+{\overline{A}}^T\overline{P}).$$

(46)

Remark 9. Comparing Equation (41) with Equation (46), because of the term ${\overline{H}}^T\overline{P}\overline{H}$ from the intrinsic random fluctuation, it can be seen that the LSPDS with random fluctuations will lead to a larger $S_0$ and a larger system entropy S.         ☐

4. System Entropy Measurement of NSPDSs

Most partial dynamic systems are nonlinear; hence, the measurement of the system entropy of nonlinear partial differential systems (NPDSs) will be discussed in this section. Consider the following NPDSs in the domain $U$:

$$\begin{array}{l}\frac{\partial y(x,t)}{\partial t}=\kappa \left(y(x,t)\right){\nabla }^{2}y(x,t)+f\left(y(x,t)\right)+g\left(y(x,t)\right)v(x,t)\\ z(x,t)=C\left(y(x,t)\right),\end{array}$$

(47)

where $f(y(x,t))\in {\mathrm{ℝ}}^n$, $C(y(x,t))\in {\mathrm{ℝ}}^{m\times n}$ and $g(y(x,t))\in {\mathrm{ℝ}}^{n\times l}$ are the nonlinear functions with $f(0)=0$, $C(0)=0$, and $g(0)=0$, respectively. The nonlinear diffusion functions $\kappa (y(x,t))\in {\mathrm{ℝ}}^{n\times n}$ satisfy $\kappa (y(x,t))\ge 0$, and $\kappa (0)=0$. If the equilibrium point of interest is not at the origin, for the convenience of analysis, the origin of the NPDS must be shifted to the equilibrium point (shifted to zero). The initial and boundary conditions are the same as the LPDS in Equation (1); then, we get the following result.

Proposition 4. For the NPDS in Equation (47), if the following HJII holds for a Lyapunov function $V(y(x,t))>0$ with $V(0)=0$:

$$\begin{array}{l}E\{{\displaystyle {\int }_U{\displaystyle {\int }_0^{t_f}[z^T(x,t)z(x,t)+{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^T}}\left(\kappa \left(y(x,t)\right){\nabla }^{2}y(x,t)+f\left(y(x,t)\right)\right)\\ +\frac{1}{4{\overline{S}}_0}{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^{T}g\left(y(x,t)\right)g^T\left(y(x,t)\right)\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)]dtdx\}<0,\end{array}$$

(48)

then the system randomness $S_0$ of the NPDS in Equation (47) has an upper bound ${\overline{S}}_0$ as given in Equation (5).

Proof. See Appendix E.                    ☐

Based on the condition of upper bound ${\overline{S}}_0$ given in Equation (48), the system randomness $S_0$ could be obtained by solving the following HJII-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{\mathrm{V}\left(y(x,t)\right)>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the HJII in Equation (48)}.\end{array}$$

(49)

Hence, the system entropy of NPDS in Equation (47) can be obtained using $S=\log S_0$. If the NPDS in Equation (47) is free of the diffusion operator ${\nabla }^{2}y(x,t)$ as with the following conventional nonlinear dynamic system:

$$\begin{array}{l}\frac{dy(t)}{dt}=f\left(y(t)\right)+g\left(y(t)\right)v(t)\\ z(t)=C\left(y(t)\right)y(t),\end{array}$$

(50)

then the result of Proposition 4 is reduced to the following corollary.

Corollary 4. For the nonlinear dynamic system Equation (50), if the following HJII holds for a positive Lyapunov function $V(y(t))>0$ with $V(0)=0$:

$$\begin{array}{l}E\{{\displaystyle {\int }_0^{t_f}[z^T(t)z(t)+{\left(\frac{\partial V\left(y(t)\right)}{\partial y}\right)}^T\left(f\left(y(t)\right)\right)}\\ +\frac{1}{4{\overline{S}}_0}{\left(\frac{\partial V\left(y(t)\right)}{\partial y}\right)}^{T}g\left(y(t)\right)g^T\left(y(t)\right)\left(\frac{\partial V\left(y(t)\right)}{\partial y}\right)]dt\}<0,\end{array}$$

(51)

then the system randomness $S_0$ of the nonlinear dynamic system in Equation (50) has an upper bound ${\overline{S}}_0$

Proof. The proof is similar to that of Proposition 4 without consideration of the diffusion operator ${\nabla }^{2}y(x,t)$ and spatial integration on the domain $U$.

Hence, the system randomness of the nonlinear dynamic system in Equation (50) can be obtained by solving the following HJII-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{\mathrm{V}\left(y(t)\right)>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the HJII in Equation (51)},\end{array}$$

(52)

and the system entropy is obtained using $S=\log S_0$. If the NPDS in Equation (47) suffers from random intrinsic fluctuations as with the NSPDSs:

$$\begin{array}{cc}\hfill \frac{\partial y(x,t)}{\partial t}& =\kappa \left(y(x,t)\right){\nabla }^{2}y(x,t)+f\left(y(x,t)\right)+g\left(y(x,t)\right)v(x,t)\hfill \\ & +H\left(y(x,t)\right)y(x,t)w(x,t)\hfill \\ \hfill z(x,t)=& C\left(y(x,t)\right)y(x,t),\hfill \end{array}$$

(53)

where $H(y(x,t))y(x,t)w(x,t)$ denotes the random intrinsic fluctuation, then the NSPDS in Equation (53) can be written in the following $\text{It}\widehat{\text{o}}$ form:

$$\begin{array}{ll}\partial y(x,t)& =\left(\kappa \left(y(x,t)\right){\nabla }^{2}y(x,t)+f\left(y(x,t)\right)+g\left(y(x,t)\right)v(x,t)\right)\partial t\\ & +H\left(y(x,t)\right)y(x,t)\partial W(x,t)\\ z(x,t)& =C\left(y(x,t)\right)y(x,t).\end{array}$$

(54)

Therefore, we can get the following result:

Proposition 5. For the NSPDS in Equations (53) or (54), if the following HJII holds for a Lyapunov function $V(y(x,t))>0$ with $V(0)=0$:

$$\begin{array}{l}E\{{\displaystyle {\int }_0^{t_f}[z^T(x,t)z(x,t)+{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^T\left(\kappa \left(y(x,t)\right){\nabla }^{2}y(x,t)+f\left(y(x,t)\right)\right)}\\ +\frac{1}{2}{y}^T(x,t)H^T\left(y(x,t)\right){\left(\frac{{\partial }^{2}V\left(y(x,t)\right)}{{\partial }^{2}y}\right)}^{T}H\left(y(x,t)\right)y(x,t)\\ +\frac{1}{4{\overline{S}}_0}{\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)}^{T}g\left(y(x,t)\right)g^T\left(y(x,t)\right)\left(\frac{\partial V\left(y(x,t)\right)}{\partial y}\right)]dt\}<0,\end{array}$$

(55)

then the system randomness $S_0$ of the NSPD S in Equations (53) or (54) can be obtained by solving the following HJII-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{V\left(y(x,t)\right)>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the HJII in Equation (55)}.\end{array}$$

(56)

Proof. See Appendix F.                           ☐

Remark 10. By comparing the HJII in Equation (48) with the HJII in Equation (55), due to the extra term $(1/2)y^T(x,t)H^T(y(x,t)){({\partial }^{2}V(y(x,t))/\partial y^2)}^{T}H(y(x,t))y(x,t)$ from the random intrinsic fluctuation $H(y(x,t))y(x,t)w(x,t)$ in Equation (53), it can be seen that the system randomness of the NSPDS in Equation (53) must be larger than the system randomness of the NPDS in Equation (47). Hence, the system entropy of the NSPDS in Equation (53) is larger than that of the NPDS in Equation (47).

5. System Entropy Measurement of NSPDS via Global Linearization and Semi-Discretization Finite Difference Scheme

In general, it is very difficult to solve the HJII in Equations (48) or (55) for the system entropy measurement of the NPDS in Equation (47) or the NSPDS in Equation (53), respectively. In this study, the global linearization technique and a finite difference scheme were employed to simplify the entropy measurement of the NPDS in Equation (47) and NSPDS in Equation (53). Consider the following global linearization of the NPDS in Equation (47), which is bounded by a polytope consisting of L vertices [12,37]:

$$\left(\begin{array}{l}\frac{\partial \kappa \left(y\left(x,t\right)\right)}{\partial y}\\ \frac{\partial f\left(y\left(x,t\right)\right)}{\partial y}\\ \frac{\partial g\left(y\left(x,t\right)\right)}{\partial y}\\ \frac{\partial C\left(y\left(x,t\right)\right)}{\partial y}\end{array}\right)\in C_0\left(\begin{array}{ccccc}\left[\begin{array}{c}{\kappa }_1\\ A_1\\ B_1\\ C_1\end{array}\right]& \cdots & \left[\begin{array}{c}{\kappa }_i\\ A_i\\ B_i\\ C_i\end{array}\right]& \cdots & \left[\begin{array}{c}{\kappa }_L\\ A_L\\ B_L\\ C_L\end{array}\right]\end{array}\right),\forall y(x,t),$$

(57)

where $C_0$ denotes the convex hull of a polytope with $L$ vertices defined in Equation (57). Then, the trajectories of $y(x,t)$ for the NPDS in Equation (47) will belong to the convex combination of the state trajectories of the following $L$ linearized PDSs derived from the vertices of the polytope in Equation (57):

$$\begin{array}{l}\frac{\partial y(x,t)}{\partial t}={\kappa }_i{\nabla }^{2}y(x,t)+A_{i}y(x,t)+B_{i}v(x,t), i=1,2,\mathrm{...},L\\ z(x,t)=C_{i}y(x,t).\end{array}$$

(58)

From the global linearization theory [16,37], if Equation (57) holds, then every trajectory of the NPDS in Equation (47) is a trajectory of a convex combination of $L$ linearized PDSs in Equation (58), and they can be represented by the convex combination of $L$ linearized PDSs in Equation (58) as follows:

$$\begin{array}{l}\frac{\partial y(x,t)}{\partial t}={\displaystyle {\displaystyle \sum }_{i=1}^L}{\alpha }_i(y)[{\kappa }_i{\nabla }^{2}y(x,t)+A_{i}y(x,t)+B_{i}v(x,t)]\\ z(x,t)={\displaystyle {\displaystyle \sum }_{i=1}^L}{\alpha }_i(y)C_{i}y(x,t),\end{array}$$

(59)

where the interpolation functions are selected as ${\alpha }_i(y)=(1/||y_i-y|{|}_2^2)/({\displaystyle {\sum }_{i=1}^L||y_i-y|{|}_2^2})$ and they satisfy $0\le {\alpha }_i(y)\le 1$ and ${\displaystyle {\sum }_{i=1}^L{\alpha }_i}(y)=1$. That is, the trajectory of the NPDS in Equation (47) can be approximated by the trajectory of the interpolated local LPDS given in Equation (59).

Following the semi-discretization finite difference scheme in Equations (28)–(34), the spatial state space system of the interpolated PDS in Equation (59) can be represented as follows:

$$\begin{array}{l}\frac{dy(t)}{dt}={\displaystyle {\displaystyle \sum }_{i=1}^L}{\alpha }_i(y)\{[I_N\otimes {\kappa }_i]T+[I_N\otimes A_i]\}y(t)+[I_N\otimes B_i]v(t)\\ z(t)={\displaystyle {\displaystyle \sum }_{i=1}^L}{\alpha }_i(y)[I_N\otimes C_i]y(t),\end{array}$$

(60)

where $y(t)$ and $v(t)$ are defined in (35). That is, the NPDS in Equation (47) is interpolated through local linearized PDSs in Equation (59) to approximate the NPDS in Equation (47) using global linearization and semi-discretization finite difference scheme.

Remark 11. In fact, there are many interpolation schemes for approximating a nonlinear dynamic system with several local linear dynamic systems such as Equation (60); for example, fuzzy interpolation and cubic spline interpolation methods [13]. Then, we get the following result.                   ☐

Proposition 6. For the linear dynamic systems in Equation (60), if the following Riccati-like inequalities hold for a positive definite symmetric $\overline{P}>0$:

$$\overline{P}{\overline{A}}_i+{{\overline{A}}_i}^T\overline{P}+{\overline{C}}_i^T{\overline{C}}_i+\frac{1}{{\overline{S}}_0}\overline{P}{\overline{B}}_i{{\overline{B}}_i}^T\overline{P}<0, i,j=1,\dots L$$

(61)

or equivalently:

$$\left[\begin{array}{cc}\overline{P}{\overline{A}}_i+{{\overline{A}}_i}^T\overline{P}+{\overline{C}}_i^T{\overline{C}}_i& \overline{P}{\overline{B}}_i\\ {{\overline{B}}_i}^T\overline{P}& -{\overline{S}}_0\end{array}\right]<0, i,j=1,\mathrm{...},L,$$

(62)

where ${\overline{A}}_i$, ${\overline{B}}_i$, and ${\overline{C}}_i$ are defined as ${\overline{A}}_i=[I_N\otimes {\kappa }_i]T+[I_N\otimes A_i],{\overline{B}}_i=[I_N\otimes B_i]$, and ${\overline{C}}_i=[I_N\otimes C_i]$, respectively, then the system randomness $S_0$ of the NPDSs in Equation (47) or the interpolated dynamic systems in Equation (60) have an upper bound ${\overline{S}}_0$.

Proof. See Appendix G.          ☐

Therefore, the system randomness $S_0$ of the NPDSs in Equation (47) or the interpolated dynamic systems in Equation (60) can be obtained by solving the following LMIs-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{\overline{\text{P}}>0}{\mathrm{min }}{\overline{S}}_0\\ \text{subject to the LMI in Equation (62)}.\end{array}$$

(63)

Hence, the system entropy $S$ of the NPDSs in Equation (47) or the interpolated dynamic systems in Equation (60) can be obtained using $S=\log S_0$. By substituting $S_0$ into the Riccati-like inequalities in Equation (61), we can obtain:

$${\overline{C}}_i^T{\overline{C}}_i+\frac{1}{S_0}\overline{P}{\overline{B}}_i{{\overline{B}}_i}^T\overline{P}<-(\overline{P}{\overline{A}}_i+{{\overline{A}}_i}^T\overline{P}).$$

(64)

Obviously, if the eigenvalues of local system matrices ${\overline{A}}_i$ are more negative (more stable), the randomness $S_0$ is smaller and the corresponding system entropy $S$ is also smaller, and vice versa.

The NSPDs given in Equation (54) can be approximated using the following global linearization technique [12,37]:

$$\left(\begin{array}{l}\frac{\partial \kappa \left(y\left(x,t\right)\right)}{\partial y}\\ \frac{\partial f\left(y\left(x,t\right)\right)}{\partial y}\\ \frac{\partial g\left(y\left(x,t\right)\right)}{\partial y}\\ \frac{\partial H\left(y\left(x,t\right)\right)}{\partial y}\\ \frac{\partial C\left(y\left(x,t\right)\right)}{\partial y}\end{array}\right)\in C_0\left(\begin{array}{ccccc}\left[\begin{array}{c}{\kappa }_1\\ A_1\\ \begin{array}{l}{B}_1\\ H_1\end{array}\\ C_1\end{array}\right]& \cdots & \left[\begin{array}{c}{\kappa }_i\\ A_i\\ \begin{array}{l}{B}_i\\ H_i\end{array}\\ C_i\end{array}\right]& \cdots & \left[\begin{array}{c}{\kappa }_L\\ A_L\\ \begin{array}{l}{B}_L\\ H_L\end{array}\\ C_L\end{array}\right]\end{array}\right),\forall y(x,t).$$

(65)

Then, the NSPDs with the random intrinsic fluctuation given in Equation (53) can be approximated by the following interpolated spatial state space system [14]:

$$\begin{array}{ll}\frac{dy(t)}{dt}=& {\displaystyle {\displaystyle \sum }_{i=1}^L}{\alpha }_i(y)\left\{[I_N\otimes {\kappa }_i]T+[I_N\otimes A_i]\right\}y(t)+[I_N\otimes B_i]v(t)\\ & +[I_N\otimes H_i]y(t)\circ dW(t)\\ z(t)=& {\displaystyle {\displaystyle \sum }_{i=1}^L}{\alpha }_i(y)[I_N\otimes C_i]y(t),\end{array}$$

(66)

i.e., we could interpolate $L$ local interpolated stochastic spatial state space systems to approximate the NSPDs in Equation (53). Then, we get the following result.

Proposition 7. For the NSPDs in Equation (54) or the linear interpolated stochastic spatial state space systems in (66), if the following Riccati-like inequalities hold for a positive definite symmetric $\overline{P}>0$:

$$\overline{P}{\overline{A}}_i+{{\overline{A}}_i}^T\overline{P}+{\overline{C}}_i^T{\overline{C}}_i+{{\overline{H}}_i}^T\overline{P}{\overline{H}}_i+\frac{1}{{\overline{S}}_0}\overline{P}{\overline{B}}_i{{\overline{B}}_i}^T\overline{P}<0 i,j=1,\mathrm{...},L$$

(67)

or equivalently:

$$\left[\begin{array}{cc}\overline{P}{\overline{A}}_i+{{\overline{A}}_i}^T\overline{P}+{\overline{C}}_i^T{\overline{C}}_i+{{\overline{H}}_i}^T\overline{P}{\overline{H}}_i& \overline{P}{\overline{B}}_i\\ {{\overline{B}}_i}^T\overline{P}& -{\overline{S}}_0\overline{I}\end{array}\right]<0,$$

(68)

where ${\overline{H}}_i=[I_N\otimes H_i]$ , then the system randomness $S_0$ of the NSPDs in Equation (53) or the interpolated stochastic systems in Equation (66) can be obtained by solving the following LMIs-constrained optimization problem:

$$\begin{array}{c}{S}_0=\underset{\overline{P}>0}{\mathrm{min }}{\overline{S}}_0\\ subjecttotheLMIsin\text{ Equation }(68).\end{array}$$

(69)

Then, the system entropy $S$ of NSPD in Equation (53) or the interpolated stochastic systems in Equation (66) could be obtained as $S=\log S_0$.

Proof. See Appendix H.                   ☐

Substituting $S_0$ into in Equation (67), we get:

$${\overline{C}}_i^T{\overline{C}}_i+H_i^T\overline{P}{\overline{H}}_i+\frac{1}{S_0}\overline{P}{\overline{B}}_i{{\overline{B}}_i}^T\overline{P}<-(\overline{P}{\overline{A}}_i+{{\overline{A}}_i}^T\overline{P}).$$

(70)

Comparing (64) with Equation (70), $S_0$ of the NSPDS in Equation (53) is larger than $S_0$ of the NPDS in Equation (47), i.e., the random intrinsic fluctuation $H(y(x,t))y(x,t)w(x,t)$ will increase the system entropy of the NSPDS. Based on the above analysis, the proposed system entropy measurement procedure of NSPDSs is given as follows:

Step 1: 

Given the initial value of state variable, the number of finite difference grids, the vertices of the global linearization, and the boundary condition.

Step 2: 

Construct the spatial state space system in Equation (60) by finite difference scheme.

Step 3: 

Construct the interpolated state space system Equation (66) by global linearization method.

Step 4: 

If the error between the original model Equation (54) and the approximated model Equation (66) is too large, we could adjust the density of grid nodes of finite difference scheme and the number of vertices of global linearization technique and return to Step 1.

Step 5: 

Solve the eigenvalue problem in Equation (69) to obtain $\overline{P}$ and ${\overline{S}}_0$, and then system entropy $S=\log {\overline{S}}_0$.

6. Computational Example

Based on the aforementioned analyses for the system entropy of the considered PDSs, two computational examples are given below for measuring the system entropy.

Example 1. Consider a heat transfer system in a 1m × 0.5m thin plate with a surrounding temperature of 0 °C as follows [38]:

$$\begin{array}{l}\frac{\partial y(x,t)}{\partial t}=\kappa {\nabla }^{2}y(x,t)+Ay(x,t)+Bv(x,t)\\ z(x,t)=Cy(x,t),\end{array}$$

(71)

$y(x,0)=20\times e^{(-10\times |0.5-x_1|-0.6738)}\times e^{(-30\times |0.5-2x_2|)}$ and y(x,t) = 0 °C, ∀ t, ∀ x on the boundary of U =[0,1]×[0,0.5]. Here, y(x,t) is the temperature function, location x is in meters, time t is in s, $\kappa ={10}^{-4}{\mathrm{m}}^2/\text{s}$ is the thermal diffusivity [4,5,6,7,9], and the term $Ay(x,t)$ with $A=-0.1s^{-1}$ denotes the thermal dissipation when the temperature of the plate is greater than the surrounding temperature, i.e., $y(x,t)>0$ °C, or the thermal absorption when the temperature on the plate is less than the surrounding temperature, i.e., $y(x,t)<0$ °C. The output coupling $C=1$. $Bv(x,t)$ is the environmental thermal fluctuation input with $B=0.1$. We can estimate the system entropy of the heat transfer system in Equation (71). Based on Proposition 3 and the LMI-constrained optimization problem Equation (40), we can calculate the system entropy of the heat transfer system in Equation (71) as $S=\log S_0=\log (0.0046)=-2.3372$. In this calculation of the system entropy, the grid spacing ${\Delta }_x$ of the finite difference scheme was chosen as 0.125m such that there are $N=7\times 3=21$ interior grid points and 24 boundary points in $U$. The temperature distributions $y(x,t)$ of the heat transfer system in Equation (71) at $t=1$, 10, 30 and 50 s are shown in Figure 2 with $v(x,t)$$=30\sin (t)$. Due to the diffusion term $\kappa {\nabla }^{2}y(x,t)$, the heat temperature of transfer system Equation (71) will be uniformly distributed gradually. Even if the thin plate has initial value (heat source) or some other influences like input signal and intrinsic random fluctuation, the temperature of the thin plate will gradually achieve a uniform distribution to increase the system entropy. This phenomenon can be seen in Figure 2, Figure 3, Figure 4 and Figure 5.

Suppose that the heat transfer system in Equation (71) suffers from the following random intrinsic fluctuation:

$$\begin{array}{l}\frac{\partial y(x,t)}{\partial t}=\kappa {\nabla }^{2}y(x,t)+Ay(x,t)+Bv(x,t)+Hy(x,t)w(x,t)\\ z(x,t)=Cy(x,t),\end{array}$$

(72)

where the term $Hy(x,t)w(x,t)$ with $H=0.02$ is due to the random parameter variation of the term $Ay\left(x,t\right)$. Then, the temperature distributions $y(x,t)$ of the heat transfer system in Equation (72) at $t=$ 1, 10, 30 and 50 s are shown in Figure 3. Based on the Corollary 3 and the LMI-constrained optimization problem in Equation (45), we can calculate the system entropy of the stochastic heat transfer system in Equation (72) as $S=\log S_0=\log (0.0339)=-1.4698$. Obviously, it can be seen that the system entropy of the stochastic heat transfer system in Equation (72) is larger than the heat transfer system in Equation (71) without intrinsic random fluctuation.

Example 2. A biochemical enzyme system is used to describe the concentration distribution of the substrate in a biomembrane. For the enzyme system, the thickness $\ell$ of the artificial biomembrane is 1 μm. The concentration of the substrate is uniformly distributed inside the artificial biomembrane. Since the biomembrane is immersed in the substrate solution, the reference axis is chosen to be perpendicular to the biomembrane. The biochemical system can be formulated as follows [13]:

$$\begin{array}{cc}\hfill \frac{\partial y(x,t)}{\partial t}=& \kappa \left(y(x,t)\right){\nabla }^{2}y(x,t)-V_M\frac{y(x,t)}{K_M+y(x,t)+y^2(x,t)/K_S}\hfill \\ & +g\left(y(x,t)\right)v(x,t)\hfill \\ \hfill z(x,t)=& Cy(x,t),\hfill \end{array}$$

(73)

where $y(x,t)$ is the concentration of the substrate in the biomembrane, $\kappa$ is the substrate diffusion coefficient, $V_M$ is the maximum activity in one unit of the biomembrane, $K_M$ is the Michaelis constant, and $K_S$ is the substrate inhibition constant. The parameters of the biochemical enzyme system are given by $\kappa (y(x,t))=e^{y\left(x,t\right)}$, $V_M=0.5$, $K_M=1$, $K_S=1$ and the output coupling $C=1$. Note that the equilibrium point in Example 2 is at zero. The concentration of the initial value of the substrate is given by $y_0(x)=0.3\sin (\pi x)$. The boundary conditions used to restrict the concentration are zero at $x=0$ and $x=1$, i.e., $y(0,t)=0$, $y(1,t)=0$. A more detailed discussion about the enzyme can be found in [13]. Suppose that the biochemical enzyme system is under the effect of an external signal $v(x,t)$. For the convenience of computation, the external signal $v(t)$ is assumed as a zero mean Gaussian noise with a unit variance. The influence function of external signal is defined as $g(y(x,t))=0.5y(x,t)$ at $x=4/9, 5/9,$ and $6/9$ (μm). Based on the global linearization in Equation (57), we get ${\overline{A}}_1-{\overline{A}}_3$ and ${\overline{B}}_1-{\overline{B}}_3$, as shown in detail in Appendix I. The concentration distributions $y(x,t)$ of the real system and approximated system are given in Figure 4 with ${\Delta }_x=0.125$, i.e., $y_i(t)=[y_1(t),y_2(t),\mathrm{...},y_9(t)]=[y_1(0,t),y_2(0.125,t),y_3(0.375,t),y_4(0.5,t),y_5(0.625,t),y_6(0.75,t),y_7(0.875,t),y_8(0.125,t),y_9(1,t)]$. Clearly, the approximated system based on the global linearization technique and finite difference scheme efficiently approach the nonlinear function. Based on Proposition 6 and the LMIs-constrained optimization given in Equation (63), we can obtain $\overline{P}$ as shown in detail in Appendix J and calculate the system entropy of the enzyme system in Equation (73) as $S=\log S_0=\log (7.6990\times {10}^{-7})=-6.1136$.

Therefore, it is clear that the approximated system in Equation (60) can efficiently approximate biochemical enzyme system in Equation (73). In this simulation, ${\Delta }_x=0.125$. Suppose that the biochemical system in Equation (73) suffers from the following random intrinsic fluctuation:

$$\begin{array}{ll}\frac{\partial y(x,t)}{\partial t}=& \kappa \left(y(x,t)\right){\nabla }^{2}y(x,t)-V_M\frac{y(x,t)}{K_M+y(x,t)+y^2(x,t)/K_S}\\ & +g\left(y(x,t)\right)v(x,t)+H\left(y(x,t)\right)y(x,t)w(x,t)\\ z(x,t)=& Cy(x,t),\end{array}$$

(74)

where the term $H(y(x,t))y(x,t)w(x,t)$ with $H(y(x,t))=y(x,t)$ is the random parameter variation from the term $V_{M}y(x,t)/(K_M+y(x,t)+y^2(x,t)/K_S)$. Based on the global linearization in Equation (65), we can get ${\overline{H}}_1-{\overline{H}}_3$ as shown in detail in Appendix K. Based on the Proposition 7 and the LMIs-constrained optimization given in Equation (69), we can solve $\overline{P}$ as shown in detail in Appendix L and calculate the system entropy of the enzyme system in Equation (74) as $S=\log S_0=\log (1.3177\times {10}^{-6})=-5.8802$.

Clearly, because of the intrinsic random parameter fluctuation, the system entropy of the stochastic enzyme system given in Equation (74) is larger than that of the enzyme system given in Equation (73).

The computation complexities of the proposed LMI-based indirect entropy measurement method is about $O(rn(n+1)/2)$ in solving LMIs, where $n$ is the dimension of $\overline{P}$, $r$ is the number of global interpolation points. We also calculate the elapsed time of the simulations examples by using MATLAB. The computation times including the drawing of the corresponding figures to solve the LMI constrained optimization problem are given as follows: in Example 1, the case of heat transfer system in Equation (71) is 183.9 s; the case of heat transfer system with random fluctuation in Equation (72) is 184.6 s. In Example 2, the case of biochemical system in Equation (73) is 17.7 s, the case of biochemical system with random fluctuation in Equation (74) is 18.6 s. The RAM of the computer is 4.00 GB, the CPU we used is AMD A4-5000 CPU with Radeon(TM) HD Graphics, 1.50 GHz. The results are reasonable. Because the dimension of grid nodes in Example 1 is $45\times 45$ and the dimension of grid nodes in Example 2 is $9\times 9$, obviously, the computation time in Example 1 is much larger than in Example 2. Further, the time spent of the system without the random fluctuation is slightly faster than the system with the random fluctuation. The conventional algorithms of calculating entropy have been applied in image processing, digital signal processing, and particle filters, like in [39,40,41]. The conventional algorithms for calculating entropy just can be used in linear discrete systems, but in fact many systems are nonlinear and continuous. The indirect entropy measurement method we proposed can deal with the nonlinear stochastic continuous systems. Though the study in [24] is about the continuous nonlinear stochastic system, many physical systems are always modeled using stochastic partial differential dynamic equation in the spatio-temporal domain. The indirect entropy measurement method we proposed can be employed to solve the system entropy measurement in nonlinear stochastic partial differential system problem.

7. Conclusions

In this study, the system entropy of stochastic partial differential systems (SPDSs) was introduced as the difference between input signal entropy and output signal entropy and was found to be the logarithm of the output signal randomness-to-input signal randomness ratio. We found that the system stability was inversely related to the system entropy and that intrinsic random fluctuation could increase the system entropy. If the eigenvalues of the system matrices are further in the left-hand side of the s-complex domain, then the SPDS has lower system entropy, and vice versa. If the output and the input signal randomness values are equal and the system is independent of the initial value, then the system entropy is zero. To estimate the system entropy of nonlinear stochastic partial differential systems (NSPDSs), the global linearization technique and finite difference scheme were employed to represent the NSPDS using the spatial state space system given in Equation (66). Therefore, the system entropy measurement problem of NPDSs became the problem of solving the HJII-constrained optimization problem given in Equation (55), which can be replaced by a simple LMIs-constrained optimization problem given in Equation (69). Hence, using the LMI-toolbox of MATLAB, we could easily calculated the system entropy of NSPDS. Finally, two examples were provided to illustrate the measurement procedure of the system entropy and to confirm that the PDSs with intrinsic random fluctuation possess greater system entropy.