Generalizations of Talagrand Inequality for Sinkhorn Distance Using Entropy Power Inequality ^†

Abstract

The distance that compares the difference between two probability distributions plays a fundamental role in statistics and machine learning. Optimal transport (OT) theory provides a theoretical framework to study such distances. Recent advances in OT theory include a generalization of classical OT with an extra entropic constraint or regularization, called entropic OT. Despite its convenience in computation, entropic OT still lacks sufficient theoretical support. In this paper, we show that the quadratic cost in entropic OT can be upper-bounded using entropy power inequality (EPI)-type bounds. First, we prove an HWI-type inequality by making use of the infinitesimal displacement convexity of the OT map. Second, we derive two Talagrand-type inequalities using the saturation of EPI that corresponds to a numerical term in our expressions. These two new inequalities are shown to generalize two previous results obtained by Bolley et al. and Bai et al. Using the new Talagrand-type inequalities, we also show that the geometry observed by Sinkhorn distance is smoothed in the sense of measure concentration. Finally, we corroborate our results with various simulation studies.

1. Introduction

OT theory studies how to transport one measure to another in the path with minimal cost. The Wasserstein distance is the cost given by the optimal path and closely connected with information measures; see, e.g., [1,2,3,4,5].

During the last decade, OT has been studied and applied extensively, especially in the machine learning community; see, e.g., [6,7,8,9]. Entropic OT, a technique to approximate the solution of the original OT, was given for computational efficiency in [10]. A key concept in the entropic OT is the Sinkhorn distance, which is a generalization of the Wasserstein distance with an extra entropic constraint. Due to the extra entropic constraint in the domain of the optimization problem, randomness is added to the original deterministic system, and the total cost increases from the original Wasserstein distance to a larger value. Therefore, a natural question is how to quantify the extra cost caused by the entropic constraint.

In this paper, we derive upper bounds for the quadratic cost of entropic OT, which are shown to include a term of entropy power responsible for quantifying the amount of uncertainty caused by the entropic constraint. This work is an extended version of [11].

1.1. Literature Review

The dynamical formulation of OT, also known as the Benamou–Brenier formula [12], generalizes the original Monge–Kantorovich formulation into a time-dependent problem. It changes the original distance problem (i.e., find the distance between two prescribed measures) into a geodesic problem (i.e., find the optimal path between two prescribed measures). Using the displacement convexity of relative entropy along the geodesic, functional inequalities such as HWI inequality and Talagrand inequality can be obtained (see, e.g., ([13] Chapter 20)).

Talagrand inequality, first given in [1], upper bounds the Wasserstein distance by relative entropy. Recent results in [2,4] obtain several refined Talagrand inequalities with dimensional improvements on the multidimensional Euclidean space. These inequalities bound Wasserstein distance with entropy power, which is sharper compared to the original one with relative entropy.

An analogue of the dynamical OT problem is the SP [14]. The SP aims to find the most likely evolution of a system of particles with respect to a reference process. The most likely evolution is called a Schrödinger bridge. SP and OT intersect on many occasions; see, e.g., [15,16,17]. The problem we study in this paper is in this intersection and mostly related to [15]. In particular, Léonard in [15] showed that the entropic OT with quadratic cost is equivalent to the SP with a Brownian motion as the reference process. He further derived that the Schrödinger bridge also admits a Benamou–Brenier formula with an additional diffusion term. Conforti in [18,19] claimed that the process can also be formulated as a continuity equation and proved that the acceleration of particles is the gradient of the Fisher information. The result therein leads to a generalized Talagrand inequality for relative entropy. Later, Bai et al. in [20] upper-bounded the extra cost from the Brownian motion by separating one Gaussian marginal into two independent random vectors. Using this approach, they showed that the dimensional improvement can be generalized to entropic OT and gave a Gaussian Talagrand inequality for the Sinkhorn distance. Additional results in [20] include a strong data processing inequality derived from their new Talagrand inequality and a bound on the capacity of the relay channel.

Entropic OT has other interesting properties. For example, Rigollet and Weed studied the case with one side of empirical measure in [21]. Their result shows that entropic OT performs maximum-likelihood estimation for Gaussian deconvolution of the empirical measure. This result can be further applied in uncoupled isotonic regression (see [9]). The dimensionality is also observed in the applications of entropic OT. For example, sample complexity bounds in [22,23] appear to be dimensional-dependent. In the GAN model, Reshetova et al. in [24] showed that the entropic regularization of OT promotes sparsity in the generated distribution.

Another element in our paper is EPI (for details on EPI, see, e.g., [25,26,27]). This inequality provides a clear expression to bound the differential entropy of two distributions’ convolution. We refer the interested reader to [28,29,30,31,32] for the connections between EPI and functional inequalities, and [33] for the connections between EPI and SP.

1.2. Contributions

In this paper, we upper-bound the quadratic cost of entropic OT by deconvolution of one side measure and EPIs. Using this approach, we avoid any discussion related to the dynamics of SP and instead we capture the uncertainty caused by the Brownian motion quantitatively. Our contributions can be articulated as follows:

(1)

We derive an HWI-type inequality for Sinkhorn distance using a modification of Bolley’s proof in [4] (see Theorem 2).

(2)

We prove two new Talagrand-type inequalities (see Theorems 3 and 4). These inequalities are obtained via a numerical term C related to the saturation, or the tightness, of EPI. We claim that this term can be computed with arbitrary deconvolution of one side marginal, while the optimal deconvolution is shown to be unknown beyond the Gaussian case. Nevertheless, we simulate suboptimally this term for a variety of distributions in Figure 1.

(3)

We show that the geometry observed by Sinkhorn distance is smoothed in the sense of measure concentration. In other words, Sinkhorn distance implies a dimensional measure concentration inequality following Marton’s method (see Corollary 2). This inequality has a simple form of normal concentration that is related to the term C and is weaker than the one implied by Wasserstein distance.

(4)

Our theoretical results are validated via numerical simulations (see Section 4). These simulations reveal several reasons for which our bounds can be either tight or loose.

Connections to Prior Art

The novelty of our work is that it comprises naturally ideas from Bolley et al. in [4] and from Bai et al. in [20] to develop new entropic OT inequalities. The dimensional improvement of Bolley et al. in [4] separates an independent term of entropy power from the original Talagrand inequality. This allows us to utilize an approach to study the entropic OT problem, which is the OT with randomness, based on the convolutional property of entropy power. On the other hand, we generalize the constructive proof of Bai et al. in [20], where they separate one Gaussian random vector into two independent Gaussian random vectors. We further claim that, for any distribution, we can always find similar independent pairs satisfying several assumptions, to upper-bound the Sinkhorn distance. As a consequence of the above, our results generalize the Talagrand inequalities of Bolley et al. in ([4] Theorem 2.1) from classical OT to entropic OT and the results of Bai et al. in ([20] Theorem 2.2) from the Gaussian case to the strongly log-concave case. In particular, we show that Theorem 3 recovers ([4] Theorem 2.1) (see Corollary 1 and the discussion in Remark 6) and that Theorem 4 recovers ([20] Theorem 2.2) (see Remark 9). It should be noted that in our analysis, we focus on the primal problem defined in [10], as opposed to the studies of its Lagrangian dual in [18,19].

1.3. Notation

$\mathbb{N}$ is the set of positive integers $\{1,2,3,...\}$. $\mathbb{R}$ is the set of real numbers. ${\mathbb{R}}^n$ is the n-dimension Euclidean space. ${\mathbb{R}}_{+}$ denotes the set $\{x\in \mathbb{R}:x\ge 0\}$.

Let $\mathcal{X},\mathcal{Y}$ be two Polish spaces, i.e., separable complete metric spaces. We write an element $x\in \mathcal{X}$ in lower-case letters and a random vector X on $\mathcal{X}$ in capital letters. We denote $\mathcal{P}\left(\mathcal{X}\right)$ as the set of all probability measures on $\mathcal{X}$. Let $\mu$ be a Borel measure on $\mathcal{X}$. For a measurable map $T:\mathcal{X}\to \mathcal{Y}$, $T_{\#}\mu$ denotes the pushing forward of $\mu$ to $\mathcal{Y}$, i.e., for all $A\subset \mathcal{Y}$, $T_{\#}\mu \left[A\right]=\mu \left[T^{-1}\left(A\right)\right]$. For $p\ge 1$, $L^p\left(\mathcal{X}\right)$ or $L^p\left(d\mu \right)$ denotes the Lebesgue space of p-th order for the reference measure $\mu$.

∇ is the gradient operator, $\mathrm{\nabla }·$ is the divergence operator, $\mathrm{\Delta }$ is the Laplacian operator, $D^2$ is the Hessian operator, $I_n$ is the n-dimension identity matrix, $Id$ is the identity map, $\parallel ·\parallel$ is the Euclidean norm, $C^k$ is the set of functions that is k-times continuously differentiable, Ric is the Ricci curvature.

$h(·)$, $I(·;·)$, $D(·\parallel ·)$, $J(·)$, $I(·|·)$ denote differential entropy, mutual information, relative entropy, Fisher information and relative Fisher information, respectively. All the logarithms are natural logarithms. $\exists !$ is unique existence. * is the convolution operator.

1.4. Organization of the Paper

The rest of the paper is organized as follows: in Section 2, we give the technical preliminaries of the theories and tools that we use; in Section 3, we state our main theoretical results; in Section 4, we give numerical simulations for our theorems, and in Section 5, we give the conclusions and future directions. Long proofs and background material are included in the Appendix.

2. Preliminaries

In this section, we give an overview of the theories and tools that we use.

2.1. Synopsis of Optimal Transport

We first give a brief introduction of OT theory. The OT problem was initialized by Gaspard Monge. The original formulation can be described as follows.

Definition 1

(Monge Problem [34]). Let $P_X$ and $P_Y$ be two probability measures supported on two Polish spaces $\mathcal{X}$, $\mathcal{Y}$. Given a lower semi-continuous (see Definition A1) cost function $c(x,y):\mathcal{X}\times \mathcal{Y}\to {\mathbb{R}}_{+}\cup \{+\infty \}$, the Monge problem wants to find a transport map $T:\mathcal{X}\to \mathcal{Y}$ minimizing the total cost:

$$\underset{T:T_{\#}{P}_X=P_Y}{inf}{\int }_{\mathcal{X}}c(x,T\left(x\right))d{P}_X\left(x\right).$$

(1)

Then, Kantorovich gave a probabilistic interpretation to the OT. This is stated next.

Definition 2

(Kantorovich Problem [35]). Let X and Y be two random vectors on two Polish spaces $\mathcal{X}$, $\mathcal{Y}$. X and Y have probability measures $P_X\in \mathcal{P}\left(\mathcal{X}\right)$, $P_Y\in \mathcal{P}\left(\mathcal{Y}\right)$. We denote $\mathrm{\Pi }(P_X,P_Y)$ as the set of all joint probability measures on $\mathcal{X}\times \mathcal{Y}$ with marginal measures $P_X$, $P_Y$. Given a lower semi-continuous cost function $c(x,y):\mathcal{X}\times \mathcal{Y}\to {\mathbb{R}}_{+}\cup \{+\infty \}$, the Kantorovich problem can be written as:

$$\underset{P\in \mathrm{\Pi }(P_X,P_Y)}{inf}{\int }_{\mathcal{X}\times \mathcal{Y}}c(x,y)dP.$$

(2)

It can be further proven that (2) gives the same optimizer as (1) (see, e.g., [36]). One can define the Wasserstein distance ([13] Definition 6.1) from (2). Let $\mathcal{X}=\mathcal{Y}$ and let d be a metric on $\mathcal{X}$. Then, the Wasserstein distance of order $p,p\ge 1$, is defined as follows:

$${\mathcal{W}}_p(P_X,P_Y):=\underset{P\in \mathrm{\Pi }(P_X,P_Y)}{inf}{\left[{\int }_{\mathcal{X}\times \mathcal{Y}}{d}^p(x,y)dP\right]}^{\frac{1}{p}}.$$

(3)

We note that the Wasserstein distance is a metric between two measures.

Cuturi in [10] gave the concept of entropic OT. In this definition, he adds an information theoretic constraint to (2), i.e.,

$$\underset{P\in \mathrm{\Pi }(P_X,P_Y;R)}{inf}{\int }_{\mathcal{X}\times \mathcal{Y}}c(x,y)dP,$$

(4)

where

$$\mathrm{\Pi }(P_X,P_Y;R):=\{P\in \mathrm{\Pi }(P_X,P_Y):I(X;Y)\le R\},$$

with $I(X;Y):=D(P\parallel P_X\times P_Y)$ denoting the mutual information [32] between X and Y, and $R\in {\mathbb{R}}_{+}$. It is well known that the constraint set is convex and compact with respect to the topology of weak convergence (for details, see, e.g., ([13] Lemma 4.4), ([37] Section 1.4)). Using the lower semi-continuity of $c(x,y)$ and ([13] Lemma 4.3), we know that the objective function $f:P\to \int cdP$ is also lower semi-continuous. Using the compactness of the constraint set and the lower semi-continuity of f, then, from Weierstrass’ extreme value theorem, the minimum in (4) is attained. Moreover, the solution is always located on its boundary, i.e., $I(X;Y)=R$, because the objective function of (4) is linear.

Entropic OT is an efficient way to approximate solutions of the Kantorovich problem. The Lagrangian dual of (4), which was introduced by Cuturi in [10], can be solved iteratively. The dual problem of (4) can be reformulated as follows:

$$\underset{ϵ\ge 0}{max}\underset{P\in \mathrm{\Pi }(P_X,P_Y)}{inf}\left\{{\int }_{\mathcal{X}\times \mathcal{Y}}c(x,y)dP+ϵ\left(I(X;Y)-R\right)\right\},$$

(5)

where $ϵ$ is a Lagrange multiplier. Using the Lagrange duality theorem ([38] Theorem 1, pp. 224–225), it can be shown that (4) and (5) give the same optimizer $P^{\ast }$.

The uncertainty of entropic OT can be understood as follows. We can write $I(X;Y)=h\left(Y\right)-h\left(Y\right|X)$, where $h\left(Y\right)$ is fixed. The conditional entropy encapsulates the randomness of the conditional distribution. The randomness decreases when $I(X;Y)$ increases. Thus, unlike (1) and (2), there is no deterministic map anymore for (4) and (5), because a one-to-one mapping leads to infinite mutual information. Note that $ϵ$ in (5) also has an explicit physical meaning. In particular, entropic OT with quadratic cost coincides with SP with a reference measure of Brownian motion (see [15]). Then, $ϵ$ is a diffusion coefficient of the Fokker–Planck equation associated with the Schrödinger bridge.

In our main results, we study (4) instead of (5) for two reasons. First, the mutual information in (4) gives a global description of the amount of uncertainty, while the coefficient $ϵ$ in (5) and its associated Fokker–Planck equation are more related to local properties, from the definitions of the Lagrangian dual and Fokker–Planck equation. Further on this point, there is no explicit expression for the correspondence between R and $ϵ$ in the duality. Second, the expectation of cost function in (2) is comparable to the Wasserstein distance. As we demonstrate in the following, it gives a smooth version of the Wasserstein distance.

Similar to the Wasserstein distance, the Sinkhorn distance of order p is defined as follows:

$${\mathcal{W}}_p(P_X,P_Y;R):=\underset{P\in \mathrm{\Pi }(P_X,P_Y;R)}{inf}{\left[{\int }_{\mathcal{X}\times \mathcal{Y}}{d}^p(x,y)dP\right]}^{\frac{1}{p}}.$$

(6)

Clearly, $\mathrm{\Pi }(P_X,P_Y;R)$ is a subset of $\mathrm{\Pi }(P_X,P_Y)$. Because of the minimization problem, it is easy to see that ${\mathcal{W}}_p(P_X,P_Y;R)>{\mathcal{W}}_p(P_X,P_Y)$. For this reason, we say that entropic OT is a smoothed version of classical OT. We note that the Sinkhorn distance is not a metric because it does not fulfill the axiom of identity of indiscernibles.

Since entropic OT is concerned with mutual information, it may be of interest to introduce a conditional Sinkhorn distance. This is defined as follows:

$${\mathcal{W}}_p(P_{X|Z},P_{Y|Z}|P_Z;R):=\underset{P\in \mathrm{\Pi }(P_{X|Z},P_{Y|Z}|P_Z);I(X;Y|Z)\le R}{inf}{\left\{{\mathbb{E}}_P\left[d^p(X,Y)\right]\right\}}^{1/p},$$

(7)

where the conditional mutual information $I(X;Y|Z):=\int I(P_{X|Z=z};P_{Y|Z=z})d{P}_Z\left(z\right)$ and $\mathrm{\Pi }(P_{X|Z},P_{Y|Z}|P_Z):=\{P_{X,Y|Z}·P_Z:P_{X,Y|Z=z}\in \mathrm{\Pi }(P_{X|Z=z},P_{Y|Z=z})\mathrm{for}z\mathrm{a}.\mathrm{e}.\}$. Conditional Sinkhorn distance is utilized in [20] and leads to a data processing inequality. Since the constraint of conditional mutual information is a linear form of $I(P_{X|Z=z};P_{Y|Z=z})$, the constraint set is still convex. The objective function is also a linear form of P. Therefore, the functional and topological properties of the conditional Sinkhorn distance are similar to the unconditional one.

Next, we state some known results of Talagrand inequality [1].

Definition 3

(Talagrand Inequality). Let $P_X$ be a reference probability measure with density $e^{-V\left(x\right)}$, where $V:\mathcal{X}\to \mathbb{R}$. We say that $P_X$ satisfies $\mathbf{T}\left(\lambda \right)>0$, i.e., Talagrand inequality with parameter $\lambda >0$, if, for any $P_Y\in \mathcal{P}\left(\mathcal{Y}\right)$,

$${\mathcal{W}}_2(P_X,P_Y)\le \sqrt{\frac{2}{\lambda }D(P_Y\parallel P_X)}.$$

(8)

Remark 1.(8) was originally introduced by Talagrand in [1] when $P_X$ is Gaussian. Blower in [39] gave a refinement and proved that

$$D^{2}V\ge \lambda I_n\Rightarrow \mathit{T}\left(\lambda \right).$$

When going beyond the Euclidean space to a manifold, Otto and Villani in [40] showed that the Bakry–Emery condition $D^{2}V+Ric$ also implies $\mathit{T}\left(\lambda \right)$.

Recently, refined inequalities with dimensional improvements were obtained in multidimensional Euclidean space. These dimensional improvements were first observed in the Gaussian case of logarithmic Sobolev inequality, Brascamp–Lieb (or Poincaré) inequality [41] and Talagrand inequality [2]. For a standard Gaussian measure $P_X$, the dimensional Talagrand inequality has the form:

$${\mathcal{W}}_2^2(P_X,P_Y)\le {\mathbb{E}[\parallel Y\parallel }^2]+n-2n{e}^{\frac{1}{2n}{\left(\mathbb{E}\right[\parallel Y\parallel }^2]-n-2D(P_Y\parallel P_X))}.$$

(9)

Bolley et al. in [4] generalized the results in [2,41] from Gaussian to strongly log-concave or log-concave. Next, we state their result. Let $d{P}_X=e^{-V}$, where $V:{\mathbb{R}}^n\to \mathbb{R}$ is $C^2$ continuous, $D^{2}V\ge \lambda I_n$. Bolley’s dimensional Talagrand inequality is given as follows:

$$\begin{array}{cc}\hfill \frac{\lambda }{2}{\mathcal{W}}_2^2(P_X,P_Y)& \le \mathbb{E}\left[V\left(Y\right)\right]-\mathbb{E}\left[V\left(X\right)\right]+n-n{e}^{\frac{1}{n}(\mathbb{E}\left[V\left(Y\right)\right]-\mathbb{E}\left[V\left(X\right)\right]-D(P_Y\parallel P_X))}.\hfill \end{array}$$

(10)

The dimensional Talagrand inequalities (9) and (10) are tighter than (8). To see this result, one may refer to our Remark 6 below.

Bai et al. in [20] gave a generalization of (9) to Sinkhorn distance. When $P_X$ is standard Gaussian,

$${\mathcal{W}}_2^2(P_X,P_Y;R)\le {\mathbb{E}[\parallel Y\parallel }^2]+n-2n\sqrt{\frac{1}{2\pi e}(1-e^{-\frac{2}{n}R})}{e}^{\frac{1}{n}h\left(Y\right)}.$$

(11)

When $R\to +\infty$, this inequality coincides with (9).

2.2. Measure Concentration

The measure concentration phenomenon describes how the probability of a random variable X changes with the deviation from a given value such as its mean or median. Marton introduced an approach of concentration directly on the level of probability measures using OT (see, e.g., ([13] Chapter 22)).

To give the notation of concentration of measure, we first introduce the probability metric space. Let $\mathcal{X}$ be a Polish space. Let d be a metric on $\mathcal{X}$. Let $\mu$ be a probability measure defined on the Borel set of $\mathcal{X}$. Then, we say that the triple $(\mathcal{X},d,\mu )$ is a probability metric space.

For an arbitrary set $A\subset \mathcal{X}$ and any $r\ge 0$, we define $A_r$ as

$$A_r:=\{x\in \mathcal{X}:d(x,A)>r\},$$

where $d(x,A):={inf}_{a\in A}d(x,a)$. Then, we say that a probability measure $\mu$ has normal (or Gaussian) concentration on $(\mathcal{X},d)$ if there exists positive K and $\kappa$ such that

$$\mu \left(A\right)\ge \frac{1}{2}\Rightarrow \mu \left(A_r\right)\le K{e}^{-\kappa r^2},\forall r>0.$$

(12)

There is another weaker statement of normal concentration, such that

$$\mu \left(A\right)\ge \frac{1}{2}\Rightarrow \mu \left(A_r\right)\le K{e}^{-\kappa {(r-r_0)}^2},\forall r>r_0.$$

(13)

It is not difficult to see that (12) can be obtained from (13), possibly with degraded constants, i.e., larger K and/or smaller $\kappa$.

The next theorem gives the connection between normal concentration and Talagrand inequality.

Theorem 1

(Theorem 3.4.7 [5]). Let $(\mathcal{X},d,\mu )$ be a probability metric space. Then, the following two statements are equivalent:

• μ satisfies $\mathit{T}\left(\lambda \right)$.
• μ has a dimension-free normal concentration with $\kappa =\frac{1}{2\lambda }$.

The intuition behind Marton’s method is that OT theory can give a metric between two probability measures by the metric structure of the supporting Polish space. The metric can be further connected with probability divergence using Talagrand inequality.

2.3. Entropy Power Inequality and Deconvolution

EPI [25] states that, for all independent continuous random vectors X and Y,

$$N(X+Y)\ge N\left(X\right)+N\left(Y\right),$$

(14)

where $N\left(X\right):=\frac{1}{2\pi e}{e}^{\frac{2}{n}h\left(X\right)}$ denotes the entropy power of X. The equality is achieved when X and Y are Gaussian random vectors with proportional covariance matrices.

Deconvolution is a problem of estimating the distribution $f\left(x\right)$ by the observations $Y_1$,...,$Y_k$ corrupted by additive noise $Z_1$,...,$Z_k$, written as

$$Y_i=X_i+Z_i,$$

where $k,i\in \mathbb{N}$ and $1\le i\le k$. $X_i$’s are i.i.d. in $f\left(x\right)$, $Z_i$’s are i.i.d in $h\left(z\right)$. $X_i$’s and $Z_i$’s are mutually independent. Let $g\left(y\right)$ be the probability density function of Y that is given by the convolution $g=f\ast h$. Then, their entropies can be bounded by EPI directly.

In our problem, we slightly abuse the concept by simply separating a random vector Y into two independent random vectors X and Z. We use this approach to introduce the uncertainty to entropic OT and consequently bound the Sinkhorn distance by EPI. Deconvolution is generally a more challenging problem than convolution. For instance, the log-concave family is convolution stable, i.e., convolution of two log-concave distributions is still log-concave, but we cannot guarantee that the deconvolution of two log-concave distributions is still log-concave. A trivial case is that wherein the deconvolution of a log-concave distribution by itself is a Dirac function. Moreover, f may not in general be positive or integrable for arbitrary given g and h, as shown in [42]. However, it should be noted that there are many numerical methods to compute deconvolution; see, e.g., [42,43,44].

3. Main Theoretical Results

In this section, we derive our main theoretical results. First, we give a new HWI-type inequality.

Theorem 2

(HWI-Type Inequality). Let $\mathcal{X}=\mathcal{Y}={\mathbb{R}}^n$. Let μ be a probability measure with density $e^{-V\left(x\right)}$ with $\lambda >0$, where $V:{\mathbb{R}}^n\to \mathbb{R}$ is $C^2$ continuous, $D^{2}V\ge \lambda I_n$. Let $P_X,P_Y$ be two probability measures on ${\mathbb{R}}^n$, $P_X,P_Y\ll \mu$. For any independent $Y_1,Y_2$ satisfying $Y_1+Y_2=Y$, $\mathbb{E}\left[Y_2\right]=0$ and $h\left(Y\right)-h\left(Y_2\right)\le R$, the following bound holds:

$$\begin{array}{cc}\hfill \frac{\lambda }{2}{\mathcal{W}}_2^2(P_X,P_Y;R)& \le \mathbb{E}\left[V\left(Y\right)\right]-\mathbb{E}\left[V\left(X\right)\right]+n-n{e}^{\frac{1}{n}(h\left(Y_1\right)-h\left(X\right))}+{\mathcal{W}}_2(P_X,P_{Y_1})\sqrt{I\left(P_X\right|\mu )},\hfill \end{array}$$

(15)

where the relative Fisher information $I\left(P_X\right|\mu ):=\int \frac{{\parallel \mathrm{\nabla }f\parallel }^2}{f}d\mu ,f=\frac{d{P}_X}{d\mu }$.

Proof. 

See Appendix A. □

Remark 2.

In Theorem 2, we construct $Y_1$ and $Y_2$, where $Y_1$ and X have a deterministic relationship. We note that the uncertainty in our construction, i.e., the independent $Y_2$, is located at one marginal, whereas the uncertainty of the true dynamics of the entropic OT is all along the path. The simplicity of our construction allows for the specific bound. We further note that there always exist such $Y_1,Y_2$ satisfying the assumptions given in Theorem 2. A trivial proof is that $Y_1=\mathbb{E}\left[Y\right]$ and $Y_2=Y-\mathbb{E}\left[Y\right]$ fulfill the above assumptions.

The next result gives a new Talagrand-type inequality.

Theorem 3

(Talagrand-Type Inequality). Let $\mathcal{X}=\mathcal{Y}={\mathbb{R}}^n$. Let $d{P}_X=e^{-V\left(x\right)}dx$, where $V:{\mathbb{R}}^n\to \mathbb{R}$ is $C^2$ continuous, $D^{2}V\ge \lambda I_n$ with $\lambda >0$, $P_Y\ll P_X$. Then, the following bound holds:

$$\begin{array}{c}\hfill \frac{\lambda }{2}{\mathcal{W}}_2^2(P_X,P_Y;R)\le \mathbb{E}\left[V\left(Y\right)\right]-\mathbb{E}\left[V\left(X\right)\right]+n-nC(P_Y,R)e^{\frac{1}{n}(h\left(Y\right)-h\left(X\right))},\end{array}$$

(16)

where $C(P_Y,R)\in [0,1]$ is a numerical term for the given $P_Y$ and $R\ge 0$.

Proof. 

Let $d{P}_X=e^{-V}$ in (15). In such a case, we have $I\left(P_X\right|\mu )=0$ from the definition of relative Fisher information. Take $C(P_Y,R)=e^{\frac{1}{n}(h\left(Y_1\right)-h\left(Y\right))}$; then, (16) is proven from (15). □

Next, we state some technical remarks on Theorem 3.

Remark 3

(On Theorem 3). In Theorem 3, we show that the Sinkhorn distance of two random vectors can be upper-bounded by a difference of a functional on two marginals, i.e., $\mathbb{E}\left[V\right(Y\left)\right]-\mathbb{E}\left[V\right(X\left)\right]$, and a term related to entropy power, i.e., $nC(P_Y,R)e^{\frac{1}{n}(h\left(Y\right)-h\left(X\right))}$. Interestingly, only the latter term is related to the constraint R. This means that the effect of information constraint is only associated with the randomness of the two random vectors instead of their positions. Recalling the physical meaning of entropic OT with quadratic cost, we can see that this expression is very natural, because the information constraint R is directly related to the randomness of the Schrödinger bridge.

Remark 4

(On the numerical term $C(·,·)$). The numerical term $C=e^{\frac{1}{n}(h\left(Y_1\right)-h\left(Y\right))}$ can be computed by arbitrary $Y_1$ satisfying the assumptions that we gave in Theorem 2, i.e., $Y_1,Y_2$ are independent, $Y_1+Y_2=Y$, $\mathbb{E}\left[Y_2\right]=0$ and $h\left(Y\right)-h\left(Y_2\right)\le R$. We observe that $e^{\frac{1}{n}h\left(Y_1\right)}$ has the form of a square root of entropy power. Using EPI and the fact that $N(·)\ge 0$, we have

$$N\left(Y\right)\ge N\left(Y_1\right)+N\left(Y_2\right)\ge N\left(Y_1\right).$$

Therefore, $C=e^{\frac{1}{n}(h\left(Y_1\right)-h\left(Y\right))}=\sqrt{N\left(Y_1\right)/N\left(Y\right)}\in [0,1]$. When $R=0$, then $Y_2=Y-\mathbb{E}\left[Y\right]$ and the density of $Y_1$ is $\delta (x-\mathbb{E}[Y\left]\right)$. This means that $e^{\frac{1}{n}h\left(Y_1\right)}=0$, hence $C=0$. When $R=+\infty$, then $Y=Y_1$, $e^{\frac{1}{n}h\left(Y_1\right)}=e^{\frac{1}{n}h\left(Y\right)}$, and consequently $C=1$. Therefore, $C(·,0)=0$, $C(·,+\infty )=1$ for all $P_Y$.

Moreover, we can show that there always exists such a sequence C non-decreasing with respect to R. We know that $C=e^{\frac{h\left(Y_1\right)}{n}-\frac{h\left(Y\right)}{n}}$ subject to $Y_1+Y_2=Y$, $\mathbb{E}\left[Y_2\right]=0$ and $h\left(Y\right)-h\left(Y_2\right)\le R$. Thus, for a larger value of R, there exists at least a $h\left(Y_2\right)$ non-increasing. This further leads to a non-decreasing $h\left(Y_1\right)$. Therefore, there exists at least a $C(·,R+\mathrm{\Delta }R)$ not smaller than $C(·,R)$, $\forall \mathrm{\Delta }R>0$, i.e., $C(·,R)$ is monotonic non-decreasing with respect to R.

We note that, for particular distributions, we may have an explicit expression of $C(·,·)$. For instance, when $P_Y$ is Gaussian, we can always take the linear combination $Y=Y_1+Y_2$, where $Y_1$ and $Y_2$ are independent Gaussian and have proportional covariance matrices. In such a case, EPI is saturated as follows:

$$\begin{array}{cc}\hfill e^{\frac{2}{n}h\left(Y_1\right)}& =e^{\frac{2}{n}h\left(Y\right)}-e^{\frac{2}{n}h\left(Y_2\right)}=(1-e^{-\frac{2}{n}R})e^{\frac{2}{n}h\left(Y\right)}.\hfill \end{array}$$

As a result, we have $C(P_Y,R)=e^{\frac{1}{n}(h\left(Y_1\right)-h\left(Y\right))}=\sqrt{1-R^{\frac{2}{n}}}$. For Cauchy distribution $Cauchy(x_0,\gamma )$, its differential entropy is $log\left(4\pi \gamma \right)$. The summation of independent Cauchy random variables ${\sum }_i^{n}Cauchy(x_i,{\gamma }_i)\sim Cauchy({\sum }_i^n{x}_i,{\sum }_i^n{\gamma }_i)$. When Y is i.i.d. Cauchy, i.e., ${\left(Y\right)}_i\sim Cauchy(x_0,\gamma )$, we take ${\left(Y_1\right)}_i\sim Cauchy(x_0,\frac{1}{4\pi }{e}^{\frac{1}{n}h\left(Y\right)}·(1-e^{-\frac{R}{n}}))$ and ${\left(Y_2\right)}_i\sim Cauchy(0,\frac{1}{4\pi }{e}^{\frac{1}{n}h\left(Y\right)}·e^{-\frac{R}{n}})$. We can see that this linear combination satisfies our assumption $h\left(Y\right)-h\left(Y_2\right)\le R$ and $C(P_Y,R)=e^{\frac{1}{n}(h\left(Y_1\right)-h\left(Y\right))}=1-e^{-\frac{R}{n}}$.

Note that the linear combination $Y=Y_1+Y_2$ is not unique, according to the assumption of Theorem 2. Consequently, this implies the non-uniqueness of $C(·,·)$. In order to obtain the tightest bound in (16), we need to solve the following optimization problem

$$C^{\ast }(P_Y,R)=sup{e}^{\frac{1}{n}(h\left(Y_1\right)-h\left(Y\right))},$$

(17)

subject to $Y_1+Y_2=Y$ and $h\left(Y\right)-h\left(Y_2\right)\le R$. To look into this optimization problem, we recall Courtade’s reverse EPI ([31] Corollary 1) as follows. If we have independent X and Y with finite second moments and choose θ to satisfy $\theta /(1-\theta )=N\left(Y\right)/N\left(X\right)$, then

$$N(X+Y)\le \left(N\right(X)+N(Y\left)\right)\left(\theta p\right(X)+(1-\theta \left)p\right(Y\left)\right),$$

(18)

where $p\left(X\right):=\frac{1}{n}N\left(X\right)J\left(X\right)\ge 1$ is the Stam defect and $J\left(X\right):=I\left(P_X\right|\mu )$, where $d\mu =dx$ is the Fisher information. We note that $p\left(X\right)$ is affine invariant, i.e., $p\left(X\right)=p\left(tX\right)$, $t>0$ because $t^{2}N\left(X\right)=N\left(tX\right)$ and $t^{2}J\left(tX\right)=J\left(X\right)$. We note that the equality $p\left(X\right)=1$ holds only if X is Gaussian. In our case, $\theta =N\left(Y_1\right)/(N\left(Y_1\right)+N\left(Y_2\right))$. When $\theta \to 1$, (18) becomes

$$N\left(Y\right)\lesssim (N\left(Y_1\right)+N\left(Y_2\right))·p\left(Y_2\right).$$

This means that the saturation of EPI is controlled by $p\left(Y_2\right)$ when the noise $Y_2$ is small, i.e., when R is large. In such a case, $C^{\ast }(P_Y,R)\approx \sqrt{1-R^{\frac{2}{n}}}$ if we let $Y_2$ be close to Gaussian, i.e., $p\left(Y_2\right)=1$. On the other hand, when $\theta \to 0$, EPI can also be saturated if we let $Y_1$ be close to Gaussian.

In Figure 1, we illustrate numerical simulations of $C(·,·)$ for the one-dimensional case. For general distributions beyond Gaussian and i.i.d. Cauchy, one can approximate $C(·,·)$ using kernel methods of deconvolution; see, e.g., [43,44]. Our strategy of deconvolution in Figure 1 is to let $Y_2=t{Y}^{\prime }$, where $Y^{\prime }$ is a copy of Y and $t\in [0,1]$. Gaussian mixture is an exception for this strategy because its spectrum would not be integrable. Instead, we let $Y_2$ be Gaussian for a Gaussian mixture. We note that this strategy is mostly not optimal and the optimal way to maximize the entropy power in (17) remains an open question.

Figure 1. Plot of the numerical term C subject to the information constraint R evaluated with respect to different distributions for the one-dimensional case.

Figure 1. Plot of the numerical term C subject to the information constraint R evaluated with respect to different distributions for the one-dimensional case.

Remark 5

(On the condition of identity of Theorem 3). To show the condition of identity of (16), we need the inequalities in (A1) and in Lemma A1 to be equalities. The equality of (A1) holds when $P_X$ is isotropic Gaussian, i.e., $P_X\sim \mathcal{N}(\mu ,{\sigma }^2{I}_n)$ for some $\mu \in {\mathbb{R}}^n$ and $\sigma >0$. The equality in Lemma A1 holds when $\mathrm{\nabla }\phi$ is affine and $D^2\phi$ has identical eigenvalues, i.e., $\mathrm{\nabla }\phi =k·Id,k\in \mathbb{R}$, see ([4] Lemma 2.6). From ([45] Theorem 1), we know that the linear combination $Y=Y_1+Y_2$ in Theorem 2 is the optimizer for entropic OT when X and Y are isotropic Gaussian. In such a case, the equality of (16) holds and $C(·,R)=\sqrt{1-R^{\frac{2}{n}}}$.

The following corollary is immediate from Theorem 3.

Corollary 1.

Wasserstein distance is bounded by

$$\frac{\lambda }{2}{\mathcal{W}}_2^2(P_X,P_Y)\le \mathbb{E}\left[V\left(Y\right)\right]-\mathbb{E}\left[V\left(X\right)\right]+n-n{e}^{\frac{1}{n}(h\left(Y\right)-h\left(X\right))}.$$

(19)

Proof. 

This is immediate from Theorem 3 when $R\to \infty$. In this case, $C(·,+\infty )=1$. □

Remark 6

(On Corollary 1). We note that (19) is equivalent to Bolley’s dimensional Talagrand inequality (10) and it is tighter than the classical Talagrand inequality (8). To make this point clear, note that, under our assumptions, $h\left(X\right)=\mathbb{E}\left[V\right(X\left)\right]$ and $D(P_Y\parallel P_X)=\mathbb{E}\left[V\left(Y\right)\right]-h\left(Y\right)$ because $d{P}_X=e^{-V\left(x\right)}dx$. Clearly, by substituting these expressions to the last term of (19), we obtain (10). Since $e^{\mu }\ge 1+\mu$, (10) is, in general, tighter than the classical Talagrand inequality (8), i.e., RHS of (10) ≤ RHS of (8). The equality holds if and only if $h\left(Y\right)=h\left(X\right)$.

We notice that C is the only difference between (10) and (16), from Remark 6. Therefore, we can immediately obtain a result related to measure concentration following ([4] Corollary 2.4). Next, we state the result on measure concentration obtained from (16).

Corollary 2.

Let $d\mu =e^{-V}$, where $V:{\mathbb{R}}^n\to \mathbb{R}$ is $C^2$ continuous, $D^{2}V\ge \lambda I_n$ with $\lambda >0$. Let $A\subset {\mathbb{R}}^n$, $A_r:=\{x\in {\mathbb{R}}^n:\forall y\in A,\parallel x-y\parallel >r\}$ for $r\ge 0$ and $c_A:=\sqrt{2{\lambda }^{-1}log(1/\mu \left(A\right))}$. Then, for $r\ge c_A$, we obtain

$$\mu \left(A_r\right)\le C^{-n}·e^{-\frac{\lambda }{2}{(r-c_A)}^2}.$$

(20)

Proof. 

See Appendix B. □

Next, we state some technical comments on Corollary 2.

Remark 7

(On Corollary 2). We note that in the derivation of Corollary 2, we follow the method of Marton in [46], which utilizes the geometrical properties of Wasserstein distance. From our discussion above, the information constraint leads to the uncertainty in entropic OT. In this result, we further show that the uncertainty smooths the geometrical properties of Wasserstein distance, i.e., Sinkhorn distance implies a looser measure concentration inequality. We begin with two extreme cases. When $C=0$, the two random vectors are independent and entropic OT has the most uncertainty. It is natural that the quadratic difference of two independent random vectors does not imply any concentration. When $C=1$, the inequality is the same as the one in Theorem 1. Between these two extremes, i.e., when $0<C<1$, Sinkhorn distance leads to a weaker normal concentration, compared to Theorem 1. Furthermore, we include in Appendix C the proof that demonstrates that the Sinkhorn distance gives a weaker measure concentration inequality in high dimensions.

The next theorem is another Talagrand-type inequality. Compared to Theorem 3, the following result is a bound obtained using a term related to the saturation of $P_X$, instead of the saturation of $P_Y$ that was used in Theorem 3.

Theorem 4.

Let $\mathcal{X}=\mathcal{Y}={\mathbb{R}}^n$. Without loss of generality, let X be a zero-mean random vector with density $e^{-V\left(x\right)}$, where $V:{\mathbb{R}}^n\to \mathbb{R}$ is $C^2$ continuous, $D^{2}V\ge \lambda I_n$ with $\lambda >0$, $P_Y\ll P_X$. Then, the following bound holds:

$$\begin{array}{c}\hfill \frac{\lambda }{2}{\mathcal{W}}_2^2(P_X,P_Y;R)\le \mathbb{E}\left[V\left(Y\right)\right]-\mathbb{E}\left[V\left(X\right)\right]+n-n{C}_x(P_X,R)e^{\frac{1}{n}(h\left(Y\right)-h\left(X\right))}+{ϵ}^{\prime },\end{array}$$

(21)

where ${ϵ}^{\prime }$ is a term related to the linearity of V.

Proof. 

See Appendix D. □

We offer the following technical comments on Theorem 4.

Remark 8

(On Theorem 4). Similar to $C(P_Y,R)$, $C_x(P_X,R)\in [0,1]$ can be computed by the equation $X=C_x{X}^{\prime }+X_2$ with arbitrary independent $X^{\prime },X_2$ under the assumptions that $X^{\prime }$ is a copy of X, $\mathbb{E}\left[X_2\right]=0$, $h\left(X\right)-h\left(X_2\right)\le R$, as shown in the proof. However, (21) is less natural than (16) because of the extra term ${ϵ}^{\prime }$. When $\mathrm{\nabla }V$ is nearly linear, ${ϵ}^{\prime }$ should be small. When $\mathrm{\nabla }V$ is far from linear, ${ϵ}^{\prime }$ is unknown.

Theorem 4 can also give a measure concentration inequality, namely

$$\mu \left(A_r\right)\le C_x^{-n}·e^{-\frac{\lambda }{2}{(r-c_A)}^2+{ϵ}^{\prime }},$$

(22)

where $A_r$ and $c_A$ are the same as those defined in Corollary 2. We omit the proof of (22) because it follows using similar steps to the ones used to prove Corollary 2.

Remark 9.

When $\mathrm{\nabla }V$ is linear, ${ϵ}^{\prime }$ is zero and $C_x(·,R)=\sqrt{1-R^{\frac{2}{n}}}$, as simply taking $t=\sqrt{1-R^{\frac{2}{n}}}$ in the proof. In such a case, (21) recovers (11) by taking X as a standard Gaussian, i.e., $V\left(x\right)={\parallel x\parallel }^2/2+k$, where k is a normalization factor. Substitute V and times 2 on both sides of (21), we have

$$\begin{array}{cc}\hfill {\mathcal{W}}_2^2(X,Y;R)\le & {\mathbb{E}[\parallel Y\parallel }^2{]-\mathbb{E}[\parallel X\parallel }^2]+2n-2n\sqrt{1-e^{-\frac{2}{n}R}}{e}^{\frac{1}{n}(h\left(Y\right)-h\left(X\right))}\hfill \\ \hfill =& {\mathbb{E}[\parallel Y\parallel }^2]+n-2n\sqrt{\frac{1}{2\pi e}(1-e^{-\frac{2}{n}R})}{e}^{\frac{1}{n}h\left(Y\right)},\hfill \end{array}$$

(23)

which is exactly the same as (11).

The next theorem gives a Talagrand-type bound for the conditional Sinkhorn distance.

Theorem 5

(Talagrand-type bound for conditional Sinkhorn distance). Let $\mathcal{X}=\mathcal{Y}={\mathbb{R}}^n$. Given a probability measure $P_Z$ and two conditional probability measures $P_{X|Z}$ and $P_{Y|Z}$, where the probability density $d{P}_{X|Z=z_0}=d{P}_X=e^{-V\left(x\right)}dx,\forall z_0\in \mathcal{Z}$, let $V:{\mathbb{R}}^n\to \mathbb{R}$ be $C^2$ continuous, $D^{2}V\ge \lambda I_n$ with $\lambda >0$, $P_{Y|Z=z_0}\ll P_X$. Then, the following bound holds:

$$\begin{array}{c}\hfill \frac{\lambda }{2}{\mathcal{W}}_2^2(P_X,P_Y|P_Z;R)\le \mathbb{E}\left[V\left(Y\right|Z)\right]-\mathbb{E}\left[V\left(X\right)\right]+n-n{C}^{\prime }(P_{Y|Z},R)e^{\frac{1}{n}(h\left(Y\right|Z)-h\left(X\right))},\end{array}$$

(24)

where $C^{\prime }(P_{Y|Z},R)$ is a numerical term. A direct observation is that $C^{\prime }(P_{Y|Z},R)$ can take ${inf}_{z_0\in \mathcal{Z}}C(P_{Y|Z=z_0},R)$.

Proof. 

See Appendix E. □

4. Numerical Simulations

In this section, we describe several numerical simulations to illustrate the validity of our theoretical findings. To check the tightness of our bounds, we use as a reference bound the numerical solution obtained via the Sinkhorn algorithm, which can be found in the POT library [47]. As an iterative method, the Sinkhorn algorithm has computational error, since the iteration stops when it converges to a certain rate. For example, in Figure 2a, we plot the result for Theorem 3 with two Gaussian marginals, which is the scenario when the identity of (16) holds. From the figure, we can see that the simulation is slightly greater than the bound. Nevertheless, we note that the precision is reasonably small.

The simulations for Theorems 3 and 4 are given in Figure 2 and Figure 3, respectively. Since the optimal value of C in Theorem 3 and the error term ${ϵ}^{\prime }$ in Theorem 4 beyond the linear case are unknown, we mainly simulate the case with one side Gaussian, i.e., with $C=\sqrt{1-R^{\frac{2}{n}}}$ and ${ϵ}^{\prime }=0$. In this way, we avoid the unknown factors and deduce several observations related to the tightness of the bounds derived in these two theorems.

The first observation is about absolute continuity. We observe that the original Talagrand inequality (8) is not tight when $P_Y$ is not absolutely continuous with respect to $P_X$, because $D(P_Y\parallel P_X)=+\infty$ in this case. In Figure 4, we illustrate one such case with almost discontinuity between two strongly log-concave distributions, i.e., the Radon–Nikodym derivative $d{P}_Y/d{P}_X=exp\left[{(x/5)}^4\right]+k^{\prime }$, where $k^{\prime }\in \mathbb{R}$ is a normalizing factor, goes to ∞ when $x\to \infty$. Consequently, the bound (16) from Theorem 3 is loose, as illustrated in Figure 5. The bound can be much looser if we increase the discontinuity, i.e., we let $d{P}_Y/d{P}_X=exp\left[{(x/5)}^8\right]+k^{\prime }$, as shown in Figure 6. By simply changing the sides of distributions $P_X$ and $P_Y$, we preserve the absolute continuity and the bound becomes tight, as we can see in Figure 5b and Figure 6b.

The second observation is related to the numerical term C. By comparing Figure 2b and Figure 3a, we observe that $C=\sqrt{1-R^{\frac{2}{n}}}$ gives a better description than $C=1-R^{\frac{1}{n}}$, i.e., the former one gives a tighter bound. This is reasonable according to our previous discussion, i.e., the independent linear combination of Cauchy random variables is not the optimal deconvolution. Actually, even if $P_Y$ is not Gaussian in (16), $C=\sqrt{1-R^{\frac{2}{n}}}$ seems to be true for all the simulated distributions.

Furthermore, we observe that the tightness of the bounds in Theorems 3 and 4 is related to the linearity of the transport map, which can be seen as a similarity between the two marginals. For example, Cauchy and Laplace distributions are similar to the Gaussian distribution. Thus, they show a tight bound in Figure 3a,d. On the other hand, Gaussian mixture and exponential distribution are relatively far from the Gaussian distribution. Hence, Figure 3c,f give looser bounds.

In Figure 7, we plot the dimensionality of Sinkhorn distance between isotropic Gaussians. Different curves correspond to a pair of Gaussian distributions in different dimensions and these pairs have the same Wasserstein distance. It can be seen that the information constraint causes more smoothing in higher dimensions, which is consistent with Corollary 2.

Figure 2. Numerical simulations and bounds via (16) for different R. (a) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim \mathcal{N}(0,\frac{1}{100})$. (b) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Cauchy(0,10)$. (c) $d{P}_X=e^{-V},V={(x/5)}^2/2+|x/10|+e^{-|x/10|}+k,k\in \mathbb{R}$ and $d{P}_Y\sim \mathcal{N}(0,\frac{1}{25})$.

Figure 2. Numerical simulations and bounds via (16) for different R. (a) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim \mathcal{N}(0,\frac{1}{100})$. (b) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Cauchy(0,10)$. (c) $d{P}_X=e^{-V},V={(x/5)}^2/2+|x/10|+e^{-|x/10|}+k,k\in \mathbb{R}$ and $d{P}_Y\sim \mathcal{N}(0,\frac{1}{25})$.

Figure 3. Numerical simulations and bounds via (21) for different R. (a) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Cauchy(0,10)$. (b) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim {\chi }^2\left(6\right)$. (c) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Exp\left(0.2\right)$. (d) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Laplace(0,5)$. (e) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y$ is Gamma distribution with $\alpha =2$ and $\beta =0.2$. (f) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y=\frac{1}{2}\mathcal{N}(-20,\frac{1}{25})+\frac{1}{2}\mathcal{N}(20,\frac{1}{25})$.

Figure 3. Numerical simulations and bounds via (21) for different R. (a) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Cauchy(0,10)$. (b) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim {\chi }^2\left(6\right)$. (c) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Exp\left(0.2\right)$. (d) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y\sim Laplace(0,5)$. (e) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y$ is Gamma distribution with $\alpha =2$ and $\beta =0.2$. (f) $d{P}_X\sim \mathcal{N}(0,\frac{1}{25})$ and $d{P}_Y=\frac{1}{2}\mathcal{N}(-20,\frac{1}{25})+\frac{1}{2}\mathcal{N}(20,\frac{1}{25})$.

Figure 4. Probability densities of $d{P}_X=e^{-V},V={(x/5)}^2/2+{(x/5)}^4+k,k\in \mathbb{R}$ and $d{P}_Y\sim \mathcal{N}(0,{\displaystyle \frac{1}{25}})$.

Figure 4. Probability densities of $d{P}_X=e^{-V},V={(x/5)}^2/2+{(x/5)}^4+k,k\in \mathbb{R}$ and $d{P}_Y\sim \mathcal{N}(0,{\displaystyle \frac{1}{25}})$.

Figure 5. Numerical simulations and bounds for different R, with $d\mu =e^{-V},V={(x/5)}^2/2+{(x/5)}^4+k,k\in \mathbb{R}$ and $d\mu \sim \mathcal{N}(0,\frac{1}{25})$. (a) Bound via (16) with $d{P}_X=d\mu$, $d{P}_Y=d\nu$. (b) Bound via (21) with $d{P}_X=d\nu$, $d{P}_Y=d\mu$.

Figure 5. Numerical simulations and bounds for different R, with $d\mu =e^{-V},V={(x/5)}^2/2+{(x/5)}^4+k,k\in \mathbb{R}$ and $d\mu \sim \mathcal{N}(0,\frac{1}{25})$. (a) Bound via (16) with $d{P}_X=d\mu$, $d{P}_Y=d\nu$. (b) Bound via (21) with $d{P}_X=d\nu$, $d{P}_Y=d\mu$.

Figure 6. Numerical simulations and bounds for different R, with $d\mu =e^{-V},V={(x/5)}^2/2+{(x/5)}^8+k,k\in \mathbb{R}$ and $d\mu \sim \mathcal{N}(0,\frac{1}{25})$. (a) Bound via (16) with $d{P}_X=d\mu$, $d{P}_Y=d\nu$. (b) Bound via (21) with $d{P}_X=d\nu$, $d{P}_Y=d\mu$.

Figure 6. Numerical simulations and bounds for different R, with $d\mu =e^{-V},V={(x/5)}^2/2+{(x/5)}^8+k,k\in \mathbb{R}$ and $d\mu \sim \mathcal{N}(0,\frac{1}{25})$. (a) Bound via (16) with $d{P}_X=d\mu$, $d{P}_Y=d\nu$. (b) Bound via (21) with $d{P}_X=d\nu$, $d{P}_Y=d\mu$.

Figure 7. Sinkhorn distances between isotropic Gaussians in different dimensions.

Figure 7. Sinkhorn distances between isotropic Gaussians in different dimensions.

5. Conclusions and Future Directions

In this paper, we considered a generalization of OT with an entropic constraint. We showed that the constraint leads to uncertainty and the uncertainty can be captured by EPI. We first derived an HWI-type inequality for the Sinkhorn distance. Then, we derived two Talagrand-type inequalities. Because of the strong geometric implication of Talagrand inequality, these two Talagrand-type inequalities can also give a weaker measure concentration inequality, respectively. From this result, we claimed that the geometry implied by the Sinkhorn distance can be smoothed out by the entropic constraint. We also showed that our results can be generalized into a conditional version of entropic OT inequality.

However, there are two factors unknown in the inequalities we derived, i.e., the optimal value of the term C in Theorem 3 and the error term ${ϵ}^{\prime }$ in Theorem 4 when one goes beyond the linear case. Although we showed that we can compute a suboptimal C using the arbitrary linear combination of two random vectors, the optimal value $C^{\ast }$ is an intriguing open question to answer. We believe that the improvement of the term C may be related to the Fisher information. Without the assumption of strong log-concavity, it requires an extra term of relative Fisher information to upper-bound the Wasserstein distance in Theorem 2. The reversing of EPI in [31] is also concerned with Fisher information. If we consider the changing of Fisher information along the Schrödinger bridge, a better estimate of term C may be feasible.