Understanding Higher-Order Interactions in Information Space

Abstract

Methods used in topological data analysis naturally capture higher-order interactions in point cloud data embedded in a metric space. This methodology was recently extended to data living in an information space, by which we mean a space measured with an information theoretical distance. One such setting is a finite collection of discrete probability distributions embedded in the probability simplex measured with the relative entropy (Kullback–Leibler divergence). More generally, one can work with a Bregman divergence parameterized by a different notion of entropy. While theoretical algorithms exist for this setup, there is a paucity of implementations for exploring and comparing geometric-topological properties of various information spaces. The interest of this work is therefore twofold. First, we propose the first robust algorithms and software for geometric and topological data analysis in information space. Perhaps surprisingly, despite working with Bregman divergences, our design reuses robust libraries for the Euclidean case. Second, using the new software, we take the first steps towards understanding the geometric-topological structure of these spaces. In particular, we compare them with the more familiar spaces equipped with the Euclidean and Fisher metrics.

1. Introduction

The motivation for the work reported in this paper is the deeper understanding of the effect of the ambient metric on the higher-order interactions arising in data, as well as its multi-scale geometric–topological properties. The first can be viewed via the Čech or Alpha complex, the second through their persistent homology. Specifically, we generalize geometric and topological data analysis methods from Euclidean geometry to Bregman geometries in which dissimilarity is measured with divergences that are generally non-symmetric and violate the triangle inequality. By necessity, these methods are sensitive to the dissimilarity defining the ambient geometry, and we exploit this sensitivity to quantify the difference between geometries.

As example geometries, we emphasize those related to information-theoretic concepts, such as the Shannon geometry and the Fisher geometry. What we call the Shannon geometry is induced by the relative entropy (Kullback–Leibler divergence) [1], which is based on Shannon’s entropy [2]. The Fisher geometry is induced by the Fisher distance (more technically the Fisher–Rao metric [3]). It was introduced by Rao [4], building on Fisher’s seminal work [5] in statistics.

These are examples of what we call information spaces  [6]. Such spaces often exhibit unfamiliar behavior, different from the Euclidean space we are used to. Understanding such spaces is however important, as—plainly speaking—this is where many types of data live [7].

Importantly, these geometries are commonly used in data science. For example, relative entropy is a standard loss function used in dimensionality reduction [8,9] and deep learning  [7,10]. (This connection may not be immediately obvious. Many models aim to explicitly minimize what is called cross-entropy. It has the same minimizer as the relative entropy, which can additionally be viewed as a distance, and therefore gives rise to a geometry). The Fisher geometry starts to be considered [11] as an alternative.

We therefore deem it important to shed light on the differences and commonalities between these geometries. Some pertinent questions are as follows: Is the simpler Fisher geometry a good approximation of the Shannon geometry? Can we see a significant difference between the Euclidean geometry and the non-Euclidean ones, as predicted by the discrepancy in their practical performances?

In the current work, we are particularly interested in the algorithms that underpin the data analysis methods, especially the topological ones. While the Fisher geometry can be handled with Euclidean tools [6], the Shannon geometry used to require customized ones [12]. We show that the Shannon geometry can also benefit from existing robust tools, although in this case the application is less direct. We also hope that this development opens new alleys for topological data analysis in information spaces.

Prior work and results. The research reported in this paper merges several lines of work. One is the study of Bregman divergences from the computational geometry point of view initiated in [13] and the extension of topological data analysis to Bregman and Fisher geometries started in [6,12]. Another is the study of higher-order interactions in high-dimensional probabilistic feature spaces [14,15], which generally lack information-theoretical interpretations.

Pivotal work at the intersection of computational geometry and Bregman geometry includes the extension of k-means clustering [16], proximity search data structures [17,18], and Voronoi tessellations [13] to the Bregman setting.

On a practical note, we mention a research direction focusing on detecting, loosely speaking, the holes in knowledge [14,15,19,20] understood as a collection of text documents. In the simplest case, each text document is represented as a discrete probability distribution (by counting characteristic keywords [21]). Interestingly, higher-order interactions among documents are necessary to capture the multi-scale topology of these data. Indeed, pairwise, triplewise, etc., intersections between balls (with respect to the appropriate metric or divergence) centered at the data points reveal the topological structure.

In this work, we will also use computational topology methods. One important topological tool is the nerve and the related Nerve Theorem. Intuitively, it allows one to capture the topology (more precisely, the homotopy type) of a union of (a finite collection of) objects using a finite simplicial complex. The theorem is usually attributed to Borsuk [22] or Leray [23], and is stated more explicitly by Weil [24]. See [25] for a comprehensive, modern treatment of this topic.

A particularly important concept in our investigations is the Bregman–Delaunay mosaic, which we formally define as the straight-line dual of the not necessarily straight-line Bregman–Voronoi tessellation obtained by measuring distance with the Bregman divergence from a data point. This mosaic was already defined in [13], and we explain how it can be computed as a weighted Euclidean Delaunay mosaic using standard geometric software. In Euclidean space, the Alpha shapes can be defined as the sublevel sets of the radius function on the Delaunay mosaic, which is a generalized discrete Morse function in the sense of Forman [26] and Freij [27]. The lower sets of the critical simplices of this function constitute the Wrap complex [28], which was introduced as a shape reconstruction tool in [29]. We extend this framework by introducing the rise function on a Bregman–Delaunay mosaic, which provides a convenient measure of the size of a Bregman ball. With these notions, we construct the shape of data in different geometries, and we use them to quantify the difference between the geometries.

We have implemented all the algorithms and used the resulting software to run experiments comparing the Euclidean, Shannon, and Fisher geometries for synthetic data. We find that the Delaunay mosaics and their Alpha and Wrap complexes in these geometries show some, occasionally subtle, differences, which we quantify.

Outline. Section 2 provides the necessary background from discrete geometry and computational topology. Section 3 gives the details needed to compute Delaunay mosaics and their Alpha and Wrap complexes in Bregman and Fisher geometries using software for weighted Delaunay mosaics in Euclidean geometry. Section 4 presents computational experiments, and Section 5 discusses the quantification of the difference between Bregman and other geometries. Section 6 concludes the paper. In Appendix A, Appendix B, Appendix C and Appendix D, we provide supplementary experiments, results, and background information.

2. Background

We need some background on Bregman divergences, Delaunay mosaics, and discrete Morse functions. Indeed, this paper combines these concepts to obtain new insights into Bregman–Delaunay mosaics and their scale-dependent subcomplexes.

Bregman divergence. Given a suitable convex function on a convex domain, the best affine approximation at a point defines a dissimilarity measure on the domain (see [30]). We follow [31] in the details of this construction, requiring a technical third condition that guarantees a conjugate function of the same kind. Let $\mathsf{\Omega }\subseteq {\mathbb{R}}^d$ be an open and convex domain. A function $F:\mathsf{\Omega }\to \mathbb{R}$ is of Legendre type [32] if

(i)

F is differentiable;

(ii)

F is strictly convex;

(iii)

$\nabla F$ diverges whenever we approach the boundary of $\mathsf{\Omega }$.

If the boundary of the domain is empty, which is the case for $\mathsf{\Omega }={\mathbb{R}}^d$, then Condition (iii) is void. In other words, $\parallel \nabla F\left(x\right)\parallel$ does not necessarily diverge when $\parallel x\parallel \to \infty$. Given points $x,y\in \mathsf{\Omega }$, the Bregman divergence from x to y associated with F is the difference between F and the best affine approximation of F at y, both evaluated at x:

$$\begin{array}{cc}\hfill D_F(x\parallel y)& =F\left(x\right)-\left[F\left(y\right)+\langle \nabla F\left(y\right),x-y\rangle \right],\hfill \end{array}$$

(1)

where $\langle .,.\rangle$ denotes the standard dot product. Note that $D_F(x\parallel y)\ge 0$, with equality iff $x=y$, resembling a metric. However, the other two axioms of a metric do not hold: the divergence is not necessarily symmetric, and it violates the triangle inequality in all non-trivial cases. In spite of these shortcomings, Bregman divergences are useful as measures of dissimilarity and are sufficient to define a geometry.

For a given $h\ge 0$, the primal ball with center x contains all points y such that the divergence from x to y is at most h, and the dual ball contains all points y such that the divergence from y to x is at most h:

$$\begin{array}{cc}\hfill B_F(x,h)& =\{y\in \mathsf{\Omega }\mid D_F(x\parallel y)\le h\},\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill B_F^{*}(x,h)& =\{y\in \mathsf{\Omega }\mid D_F(y\parallel x)\le h\}.\hfill \end{array}$$

(3)

The geometric intuition for (2) is to cast light onto the graph of F from a point vertically above $x\in {\mathbb{R}}^d$ in ${\mathbb{R}}^{d+1}$ and at distance h below the graph of F: the primal ball is the vertical projection of the lit up part of the graph onto ${\mathbb{R}}^d$. This ball is not necessarily convex. The geometric intuition for (3) is to intersect the graph of F with the tangent hyperplane at x shifted vertically upward by a distance h: the dual ball is the vertical projection of the part of the graph on or below this shifted hyperplane. This ball is necessarily convex.

The conjugate of F can be constructed with elementary geometric means. Specifically, we use the polarity transform that maps a point $A=(a,a_{d+1})\in {\mathbb{R}}^d\times \mathbb{R}$ to the affine map $A^{*}:{\mathbb{R}}^d\to \mathbb{R}$ defined by $A^{*}\left(x\right)=\langle a,x\rangle -a_{d+1}$. Similarly, it maps $A^{*}$ to $A={\left(A^{*}\right)}^{*}$. The graph of F can be described as a set of points or a set of affine maps that touch the graph. The conjugate function, $F^{*}:{\mathsf{\Omega }}^{*}\to \mathbb{R}$, is defined such that polarity maps the points of the graph of F to the tangent affine maps of the graph of $F^{*}$, and it maps the tangent affine maps of the graph of F to the points of the graph of $F^{*}$. Since A and $A^{*}$ switch position with gradient, so do F and $F^{*}$. More specifically, ${\mathsf{\Omega }}^{*}=\varphi \left(\mathsf{\Omega }\right)$ and $F^{*}:{\mathsf{\Omega }}^{*}\to \mathbb{R}$ are given by

$$\begin{array}{cc}\hfill \varphi \left(x\right)& =\nabla F\left(x\right),\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill F^{*}\left(\varphi \left(x\right)\right)& =\langle \nabla F\left(x\right),x\rangle -F\left(x\right),\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill \nabla F^{*}\left(\varphi \left(x\right)\right)& =x.\hfill \end{array}$$

(6)

The convexity of $\mathsf{\Omega }$ and Conditions (i), (ii), (iii) imply that ${\mathsf{\Omega }}^{*}$ is convex and $F^{*}$ satisfies (i), (ii), (iii). In other words, the conjugate of a Legendre type function is again a Legendre type function. Importantly, the Bregman divergences associated with F and with $F^{*}$ are symmetric: $D_F(x\parallel y)=D_{F^{*}}(\varphi \left(y\right)\parallel \varphi \left(x\right))$. Hence, $\varphi$ maps primal balls to dual balls and it maps dual balls to primal balls:

$$\begin{array}{cc}\hfill B_{F^{*}}^{*}(\varphi \left(x\right),h)& =\varphi \left(B_F(x,h)\right),\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill B_{F^{*}}(\varphi \left(x\right),h)& =\varphi \left(B_F^{*}(x,h)\right).\hfill \end{array}$$

(8)

Since all dual balls are convex, all primal balls are diffeomorphic images of convex sets. This implies that the common intersection of a collection of primal balls is either empty or contractible [12], so the Nerve Theorem applies.

Examples. An important example of a Legendre type function is $\varpi :{\mathbb{R}}^d\to \mathbb{R}$ defined by mapping x to half the square of its Euclidean norm: $\varpi \left(x\right)=\frac{1}{2}{\parallel x\parallel }^2$. It is the only Legendre type function that is its own conjugate: $\varpi ={\varpi }^{*}$. The symmetry between the divergences of a Legendre type function and its conjugate thus implies $D_{\varpi }(x\parallel y)=D_{\varpi }(y\parallel x)$ and $B_{\varpi }(x,h)=B_{\varpi }^{*}(x,h)$. Indeed, it is easy to see that the divergence is half the squared Euclidean distance, $D_{\varpi }(x\parallel y)=\frac{1}{2}{\parallel x-y\parallel }^2$, which is of course symmetric. This particular Legendre type function provides an anchor point for comparison.

The example that justifies the title of this paper is the (negative) Shannon entropy [2]. We add an extra term, resulting in $E:{\mathbb{R}}_{+}^d\to \mathbb{R}$, defined as $E\left(x\right)={\sum }_{i=1}^d[x_{i}ln{x}_i-x_i]$. Being linear, the additional term does not affect the resulting divergence, but simplifies certain computations later.

This function is of Legendre type and fundamental to information theory [2]. Its divergence,

$$\begin{array}{cc}\hfill D_E(x\parallel y)& ={\sum }_{i=1}^d[x_{i}ln{x}_i-x_{i}ln{y}_i-x_i+y_i],\hfill \end{array}$$

(9)

is generally referred to as the relative entropy [10] or the Kullback–Leibler divergence [1] from x to y. We remark that the above derivation using the Bregman machinery results in a divergence which is valid on the entire positive orthant of ${\mathbb{R}}^d$, and agrees with the standard definition [1] on the standard simplex.

The gradient of the Shannon entropy at x is the vector $\nabla E\left(x\right)$ with components $ln{x}_i$ for $1\le i\le d$. Using Equation (5), one easily computes that the conjugate of E maps this vector to ${\sum }_{i=1}^d{x}_i$. Hence $E^{*}:{\mathbb{R}}^d\to \mathbb{R}$ is defined by mapping $y\in {\mathbb{R}}^d$ to $E^{*}\left(y\right)={\sum }_{i=1}^d{e}^{y_i}$.

A case of special interest is the restriction of the Shannon entropy to the standard simplex, which is a subset of the positive orthant. Writing $x=(x_1,x_2,\dots ,x_d)$ for a point of ${\mathbb{R}}_{+}^d$, the standard $(d-1)$-simplex, denoted ${\Delta }^{d-1}$, consists of all points x that satisfy $x_1+x_2+\dots +x_d=1$. We use ${\Delta }^{d-1}$ as the domain of a Legendre type function, which is the reason why we introduce ${\Delta }^{d-1}$ as an open set. Finally, write $E_{\Delta }:{\Delta }^{d-1}\to \mathbb{R}$ for the restriction of the Shannon entropy to the standard simplex. This setting is important because each $x\in {\Delta }^{d-1}$ can be interpreted as a probability distribution on d disjoint events. Correspondingly, $-E_{\Delta }\left(x\right)=-E\left(x\right)$ is the expected efficiency to optimally encode a sample from this distribution. Finally, the relative entropy from x to y is the expected loss in coding efficiency if we use the code optimized for y to encode a sample from x. Projecting the gradient of the unrestricted Shannon entropy into the hyperplane of the simplex passing through the origin, we obtain the gradient of the restriction:

$$\begin{array}{cc}\hfill \nabla E_{\Delta }\left(x\right)& =\left[\begin{array}{c}ln{x}_1\\ ln{x}_2\\ ⋮\\ ln{x}_d\end{array}\right]-\frac{1}{d}\sum _{i=1}^{d}ln{x}_i\left[\begin{array}{c}1\\ 1\\ ⋮\\ 1\end{array}\right].\hfill \end{array}$$

(10)

Using (4) and (5), we compute the conjugate of $E_{\Delta }$, which we state in terms of the barycentric coordinates parametrizing ${\mathbb{R}}^{d-1}$. Specifically, we obtain ${\varphi }_{\Delta }\left(x\right)=\nabla E_{\Delta }\left(x\right)$ and

$$\begin{array}{cc}\hfill E_{\Delta }^{*}\left({\varphi }_{\Delta }\left(x\right)\right)& =\langle \nabla E_{\Delta }\left(x\right),x\rangle -E_{\Delta }\left(x\right)\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill & =1-\frac{1}{d}{\sum }_{i=1}^{d}ln{x}_i\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill & =1+ln{\sum }_{i=1}^d{e}^{y_i},\hfill \end{array}$$

(13)

in which the $y_i=ln{x}_i-\frac{1}{d}{\sum }_{i=1}^{d}ln{x}_i$ are the coordinates in conjugate space. Indeed, it is not difficult to verify (13) using $ln{\sum }_{i=1}^d{x}_i=0$ for points in the standard simplex.

Antonelli isometry. A Bregman divergence gives rise to a path metric in which length is measured by integrating the square root of the divergence. As explained in [6], any divergence that decomposes into a term per coordinate implies an isometry between this path metric and the Euclidean metric. By (9), the relative entropy is an example of such a divergence, and the corresponding path metric is known as the Fisher metric, which plays an important role in statistics and information geometry [3]. Instead of formalizing the recipe for constructing the Fisher metric from the relative entropy, we note that the mapping

$$\begin{array}{cc}\hfill \mathsf{ȷ}\left(x\right)& =(\sqrt{2x_1},\sqrt{2x_2},\dots ,\sqrt{2x_d})\hfill \end{array}$$

(14)

is an isometry with Euclidean space, as first observed by Antonelli [33]. By virtue of being an isometry, the distance between points $x,y\in {\mathbb{R}}_{+}^d$ under the Fisher metric satisfies ${\parallel x-y\parallel }_{\mathrm{Fsh}}=\parallel \mathsf{ȷ}\left(x\right)-\mathsf{ȷ}\left(y\right)\parallel$. The path of this length from x to y is the preimage of the line segment from $\mathsf{ȷ}\left(x\right)$ to $\mathsf{ȷ}\left(y\right)$, which is generally not straight.

Of special interest is the Fisher metric restricted to the standard $(d-1)$-simplex, denoted ${\Delta }^{d-1}$. The mentioned isometry maps ${\Delta }^{d-1}$ to $\mathsf{ȷ}\left({\Delta }^{d-1}\right)$, which is the positive orthant of the sphere with radius $\sqrt{2}$ and center at the origin in ${\mathbb{R}}^d$. The shortest path between $x,y\in {\Delta }^{d-1}$ is thus the preimage of the great-circle arc that connects $\mathsf{ȷ}\left(x\right)$ and $\mathsf{ȷ}\left(y\right)$ on the sphere. Since this arc is generally longer than the straight line segment connecting $\mathsf{ȷ}\left(x\right)$ and $\mathsf{ȷ}\left(y\right)$ in ${\mathbb{R}}_{+}^d$, the distance between x and y under the Fisher metric restricted to ${\Delta }^{d-1}$ is generally larger than in the unrestricted case.

Alpha shapes and Wrap complexes. Two popular shape reconstruction methods based on Delaunay mosaics are the Alpha shapes introduced in [34] and the Wrap complexes first published in [29]. Both extend to generalized discrete Morse functions and therefore to Bregman–Delaunay mosaics and Bregman–Wrap complexes. While working with Bregman divergences, we can construct them using weighted Euclidean Delaunay mosaics, for which there is readily available fast software. For brevity, standard definitions and properties are available in Appendix B. Letting D be a simplicial complex and $f:D\to \mathbb{R}$ a generalized discrete Morse function, the Alpha complex for h is the sublevel set,

$$\begin{array}{cc}\hfill {\mathrm{Alpha}}_h\left(f\right)& =f^{-1}(-\infty ,h],\hfill \end{array}$$

(15)

and the Alpha shape is the underlying space of the Alpha complex. In contrast to the Alpha shape, the assumption that f be a generalized discrete Morse function is essential in the definition of the Wrap complex. Recall that every step of a generalized discrete Morse function is an interval of simplices in the Hasse diagram. We form the step graph, $\mathcal{G}={\mathcal{G}}_f$, whose nodes are the steps and whose arcs connect step $\phi$ to step $\psi$ if there are simplices $P\in \phi$ and $Q\in \psi$ with an arc from P to Q in the Hasse diagram. By construction, f is strictly increasing along directed paths in the step graph, which implies that the graph is acyclic.

The lower set of a node $\nu$ in $\mathcal{G}$, denoted $\downarrow \nu$, is the set of nodes $\phi$ for which there are directed paths from $\phi$ to $\nu$. Similarly, we write $\downarrow N={\bigcup }_{\nu \in N}\downarrow \nu$ for the lower set of a collection of nodes, and $\bigcup \downarrow N$ for the corresponding collection of simplices. We are particularly interested in the set of singular intervals, and we recall that each such interval contains a critical simplex of f. We write ${\mathrm{Sg}}_f$ for the set of singular intervals, and ${\mathrm{Sg}}_f\left(h\right)\subseteq {\mathrm{Sg}}_f$ for the subset whose simplices satisfy $f\left(Q\right)\le h$. The Wrap complex for h is the union of steps in the lower sets of the singular intervals with value at most h:

$$\begin{array}{cc}\hfill {\mathrm{Wrap}}_h\left(f\right)& =\bigcup \downarrow {\mathrm{Sg}}_f\left(h\right).\hfill \end{array}$$

(16)

There are alternative constructions of the Wrap complex. Starting with the Alpha complex for h, we obtain the Wrap complex for the same value by collapsing all non-singular intervals that can be collapsed. The order of the collapses is not important as all orders produce the same result, namely ${\mathrm{Wrap}}_h\left(f\right)$. Symmetrically, we may start with the critical simplices of value at most h and add the minimal collection of non-singular intervals needed to obtain a simplicial complex. The minimal collection is unique and so is the result, ${\mathrm{Wrap}}_h\left(f\right)$. A proof of the equivalence of these three definitions of the Wrap complex is given in Appendix C.

3. Mosaics and Algorithms

In this section, we review Bregman–Delaunay and Fisher–Delaunay mosaics as well as their scale-dependent subcomplexes. All mosaics are constructed using software for weighted Delaunay mosaics in Euclidean geometry; all subcomplexes are computed by convex optimization, adapting the method from [12]. We begin with the mosaics in Bregman geometry.

Bregman–Delaunay mosaics. Let $\mathsf{\Omega }\subseteq {\mathbb{R}}^d$ be open and convex, consider a Legendre type function $F:\mathsf{\Omega }\to \mathbb{R}$, and let $U\subseteq \mathsf{\Omega }$ be locally finite. Following [12,13], we define the Bregman–Voronoi domain of $u\in U$, denoted ${\mathrm{dom}}_F\left(u,\mathsf{\Omega }\right)$, as the points $a\in \mathsf{\Omega }$ that satisfies $D_F(u\parallel a)\le D_F(v\parallel a)$ for all $v\in U$. The Bregman–Voronoi tessellation is the collection of such domains, and the Bregman–Delaunay mosaic records all non-empty common intersections:

$$\begin{array}{cc}\hfill {\mathrm{Vor}}_F\left(U,\mathsf{\Omega }\right)& =\{{\mathrm{dom}}_F\left(u,\mathsf{\Omega }\right)\mid u\in U\},\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill {\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)& =\{Q\subseteq U\mid {\bigcap }_{u\in Q}{\mathrm{dom}}_F\left(u,\mathsf{\Omega }\right)\ne \varnothing \},\hfill \end{array}$$

(18)

and we note that the mosaic is isomorphic to the nerve [35,36] of the tessellation. To develop geometric intuition, we observe that ${\mathrm{Vor}}_F\left(U,\mathsf{\Omega }\right)$ can be obtained by growing primal Bregman balls with centers at the points $u\in U$. When two such balls meet, they freeze where they touch but keep growing everywhere else. Eventually, each ball covers exactly the corresponding domain. Since the primal balls are not necessarily convex, it is not surprising that the faces shared by the domains are not necessarily straight. Nevertheless, the Delaunay mosaic has a natural straight-line embedding as all its cells are vertical projections of the lower faces of the convex hull of the points $(u,F\left(u\right))\in {\mathbb{R}}^{d+1}$. To see this, we note that each cell of the mosaic corresponds to a dual Bregman ball whose boundary passes through the vertices of the cell, and this ball is the vertical projection of the part of the graph of F on or below the graph of an affine function.

Construction. To construct the mosaic, we assume that $U\subseteq \mathsf{\Omega }$ is in general position, by which we mean that Conditions I and II from Appendix D are satisfied after transforming $U\subseteq \mathsf{\Omega }$ to $X\subseteq {\mathbb{R}}^d\times \mathbb{R}$ such that ${\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$ is a subcomplex of the weighted Delaunay mosaic of X. Lifting the points from ${\mathbb{R}}^d$ to ${\mathbb{R}}^{d+1}$ and projecting the lower boundary of the convex hull back to ${\mathbb{R}}^d$, we obtain the mosaic. We remind the reader that relevant background information can be found in Appendix B, and define $\varpi \left(a\right)=\frac{1}{2}{\parallel a\parallel }^2$. We formalize this method while stating all steps in terms of weighted points in d dimensions:

Step 1: 

Let $X\subseteq {\mathbb{R}}^d\times \mathbb{R}$ be the set of weighted points $x\left(u\right)=(u,2\varpi (u)-2F(u\left)\right)$, with $u\in U$.

Step 2: 

Compute the weighted Delaunay mosaic of X in Euclidean geometry, denoted $\mathrm{Del}\left(X\right)$.

Step 3: 

Select ${\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$ as the collection of simplices in $\mathrm{Del}\left(X\right)$ whose corresponding weighted Voronoi cells have a non-empty intersection with ${\mathsf{\Omega }}^{*}$.

Indeed, the weighted Delaunay mosaic computed in Step 2 may contain simplices that do not belong to the Delaunay–Bregman mosaic of F. To implement Step 3, we note that ${\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$ is dual to ${\mathrm{Vor}}_F\left(U,\mathsf{\Omega }\right)$, which is isomorphic to ${\mathrm{Vor}}_{F^{*}}\left(\varphi \left(U\right),{\mathsf{\Omega }}^{*}\right)$, and this Bregman–Voronoi tessellation is the weighted Voronoi tessellation of X restricted to ${\mathsf{\Omega }}^{*}$. This tessellation has convex polyhedral cells and is readily available as the dual of $\mathrm{Del}\left(X\right)$. Writing $Y\left(Q\right)\subseteq X$ for the points $x\left(u\right)$ with $u\in Q\subseteq U$ and $\mathrm{dom}\left(Y\right)$ for the weighted Voronoi cell that corresponds to $Y\in \mathrm{Del}\left(X\right)$, we have

$$\begin{array}{cc}\hfill {\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)& =\{Q\subseteq U\mid \mathrm{dom}\left(Y\left(Q\right)\right)\cap {\mathsf{\Omega }}^{*}\ne \varnothing \}.\hfill \end{array}$$

(19)

Instead of computing all these intersections, we can collapse $\mathrm{Del}\left(X\right)$ to the desired subcomplex and thus save time by looking only at a subset of the mosaic. We explain how the simplices can be organized to facilitate such a collapse. Recalling that ${\mathsf{\Omega }}^{*}\subseteq {\mathbb{R}}^d$ is open and convex, we introduce the signed distance function, $\theta :{\mathbb{R}}^d\to \mathbb{R}$, which maps every $a\in {\mathbb{R}}^d$ to plus or minus $r=r\left(a\right)$ such that the sphere with center a and radius r touches $\partial {\mathsf{\Omega }}^{*}$ but does not cross the boundary. Finally, $\theta \left(a\right)=r\left(a\right)$ if $a\notin {\mathsf{\Omega }}^{*}$ and $\theta \left(a\right)=-r\left(a\right)$ if $a\in {\mathsf{\Omega }}^{*}$. Note that ${\mathsf{\Omega }}^{*}={\theta }^{-1}[-\infty ,0)$ and that ${\mathsf{\Omega }}_t^{*}={\theta }^{-1}[-\infty ,t)$ is open and convex for every t. Now construct $\vartheta :\mathrm{Del}\left(X\right)\to \mathbb{R}$ by mapping $Y\in \mathrm{Del}\left(X\right)$ to the maximum $t\in \mathbb{R}$ for which $\mathrm{dom}\left(Y\right)\cap {\mathsf{\Omega }}_t^{*}=\varnothing$. By (19), we obtain ${\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$ by removing all simplices Y with $\vartheta \left(Y\right)\ge 0$. The crucial observation is that for X in general position, $\vartheta$ is a generalized discrete Morse function with a single critical vertex. To see this, we observe that $\mathrm{Vor}\left(X\right)$ decomposes ${\mathsf{\Omega }}_t^{*}$ into convex domains for every value t, which by the Nerve Theorem [22] implies that ${\vartheta }^{-1}(-\infty ,t]$ is contractible. Removing the simplices in a sequence of decreasing values of $\vartheta$ thus translates into a sequence of collapses that preserve the homotopy type of the mosaic.

Rise functions. To introduce scale into the construction of Bregman–Delaunay mosaics, we generalize the radius function from Euclidean geometry to Bregman geometries, changing the name because size is more conveniently measured by height difference in the $(d+1)$-st coordinate direction as opposed to the radius in ${\mathbb{R}}^d$. Let $\dot{u}=(u,F\left(u\right))$ and $\overline{u}:{\mathbb{R}}^d\to \mathbb{R}$ be the point and affine map that correspond to $u\in \mathsf{\Omega }$, and let $\upsilon :{\mathbb{R}}^d\to \mathbb{R}$ be the upper envelope of the $\overline{u}$, $u\in U$. We introduce the rise function, ${\varrho }_F:{\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)\to \mathbb{R}$, which maps each simplex, Q, to the minimum difference between $F^{*}$ and $\upsilon$ at points in the conjugate Voronoi cell:

$$\begin{array}{cc}\hfill {\varrho }_F\left(Q\right)& =\underset{a\in \varphi \left(\mathrm{dom}\left(Q,\mathsf{\Omega }\right)\right)}{inf}[F^{*}\left(a\right)-\upsilon \left(a\right)].\hfill \end{array}$$

(20)

It is the infimum amount we have to lower the graph of $F^{*}$ until it intersects the graph of $\upsilon$ at a point vertically above the Voronoi cell in conjugate space. Without going to the conjugate, we can interpret ${\varrho }_F\left(Q\right)$ in terms of (primal) Voronoi domains and cones of light cast from the $\dot{u}$ onto the graph, which we raise until the cones clipped to within their Voronoi domains have a point in common. This interpretation motivates the name of the function. Comparing (20) with (A8), we see that the two agree when $F=\varpi$ and $\mathsf{\Omega }={\mathbb{R}}^d$. Indeed, we obtain $F^{*}=\varpi$ and $\upsilon =\xi$. Furthermore, $\varphi \left(\mathrm{dom}\left(Q,\mathsf{\Omega }\right)\right)=\mathrm{dom}\left(Q,\mathsf{\Omega }\right)$, and taking the infimum is the same as taking the minimum.

For every $h\in \mathbb{R}$, we have a sublevel set, ${\mathrm{Del}}_{F,h}\left(U,\mathsf{\Omega }\right)={{\varrho }_F}^{-1}(-\infty ,h]$, which we refer to as the Bregman–Alpha complex of U and F for size h. For $h<0$, this complex is empty, for $h=0$, it is a set of vertices namely the points in U, and for sufficiently large positive h, this complex is ${\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$.

Information-theoretic interpretations. We offer a brief information-theoretical interpretation for many of the geometric objects mentioned above in the case of relative entropy.

Let us start with a primal relative entropy ball of radius r around a finite probability distribution c. It contains all distributions that can be used to approximate c, incurring the efficiency loss of at most r bits. With this, it is easy to see that the Voronoi and Delaunay constructions have intuitive information-theoretical interpretations. For simplicity, let us consider the Čech rise (radius) function, which arises from the first point of a non-empty intersection between a number of primal balls [35,36]. This radius is therefore the smallest number of bits that need to be incurred when approximating the centers of these balls with a single distribution. This value can be interpreted as an information-theoretical quantification of the high-order interactions among these distributions. Of course, the intersection point is exactly the unique best choice for such a distribution.

The remaining objects can be interpreted in a similar way, which we leave for the reader. We stress that the birth and death values in the resulting persistence diagrams also express the loss of coding efficiency in bits—and stress that the diagrams are identical for the Delaunay and Čech case.

Computation. We compute the rise function following the intuition based on primal Voronoi domains explained below (20). Equivalently, ${\varrho }_F\left(Q\right)$ is the minimum amount we have to raise the graph of F so it has a supporting hyperplane that passes through all points $\dot{u}$, with $u\in Q$, while all other point $\dot{u}$, with $u\in U$, lie on or above the hyperplane.

To turn this intuition into an algorithm, we consider the affine hull of Q and write $\overline{v}:\mathrm{aff}Q\to \mathbb{R}$ for the affine function that satisfies $\overline{v}\left(u\right)=F\left(u\right)$ for all $u\in Q$. Let $H:\mathrm{aff}Q\cap \mathsf{\Omega }\to \mathbb{R}$ measure the difference: $H\left(a\right)=F\left(a\right)-\overline{v}\left(a\right)$. Since F is of Legendre type, so is H. We are interested in the infimum of H, which either occurs at a point in $\mathrm{aff}Q\cap \mathsf{\Omega }$ or at the limit of a divergent sequence. We therefore introduce a numerical routine that returns both, the infimum and the point where it occurs:

1

InfSize $(\mathrm{function}F,\mathrm{simplex}Q)$:

2

      $(a_Q,h_Q)=(\mathrm{arginf}H,infH)$;

3

      return $(a_Q,h_Q)$.

Note that the dual Bregman ball centered at $a_Q\in \mathrm{aff}Q\cap \mathsf{\Omega }$ of size $h_Q$ contains Q in its boundary, and it may or may not contain points of $U\backslash Q$ in its interior. If it does not, then ${\varrho }_F\left(Q\right)=h_Q$, otherwise, ${\varrho }_F\left(Q\right)$ is the minimum function value of the proper cofaces of Q. To express this more formally, we write $\mathrm{coFacets}\left(Q\right)$ for the collection of simplices $R\in \mathrm{Del}\left(X\right)$ with $Q\subseteq R$ and $\#R=\#Q+1$. Since Q gets its value either directly or from a coface, it is opportune to compute the rise function in the order of decreasing dimension:

1

for $p=d$ downto 0 do

2

      forall p-simplices $Q\in {\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$ do

3

            $(a_Q,h_Q)$ = InfSize(F, Q);

4

            if $B_F^{*}(a_Q,h_Q)\cap [U\backslash Q]=\varnothing$

5

                  then ${\varrho }_F\left(Q\right)=h_Q$

6

                  else ${\varrho }_F\left(Q\right)=\underset{R\in \mathrm{coFacets}\left(Q\right)}{min}{\varrho }_F\left(R\right)$.

Note that this algorithm assigns a value to every simplex in ${\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$. Indeed, the simplices in $\mathrm{Del}\left(X\right)$ that are not in ${\mathrm{Del}}_F\left(U,\mathsf{\Omega }\right)$ have been culled in Step 3, as explained above.

Fisher metric. In addition to the Bregman divergences, we consider Delaunay mosaics under the Fisher metric. To construct them, we recall that the mapping $\mathsf{ȷ}:{\mathbb{R}}_{+}^d\to {\mathbb{R}}_{+}^d$ defined by $\mathsf{ȷ}\left(x\right)=(\sqrt{2x_1},\sqrt{2x_2},\dots ,\sqrt{2x_d})$ is an isometry between the Fisher metric and the Euclidean metric. This suggests the following algorithm:

Step 1: 

Compute the Delaunay mosaic of $\mathsf{ȷ}\left(U\right)$ in Euclidean space.

Step 2: 

Remove the simplices from $\mathrm{Del}\left(\mathsf{ȷ}\left(U\right)\right)$ whose dual Voronoi cells have an empty intersection with ${\mathbb{R}}_{+}^d$.

Step 3: 

Draw the resulting complex by mapping each point $\mathsf{ȷ}\left(u\right)$ to the original point $u\in U\subseteq {\mathbb{R}}_{+}^d$.

The rise function in Euclidean geometry maps every simplex $\mathsf{ȷ}\left(Q\right)\in \mathrm{Del}\left(\mathsf{ȷ}\left(U\right)\right)$ to the squared radius of the smallest empty circumsphere of $\mathsf{ȷ}\left(Q\right)$. By isometry, the preimage of this Euclidean sphere is the smallest empty circumsphere of Q under the Fisher metric, and the squared radius is the same. We thus obtain the rise function on the Fisher–Delaunay mosaic by copying the values of the rise function on the Delaunay mosaic in Euclidean geometry.

The construction of the mosaic for the Fisher metric restricted to the standard simplex, ${\Delta }^{d-1}$, is only slightly more complicated. As mentioned in Section 2, the isometry maps ${\Delta }^{d-1}$ to $\sqrt{2}{\mathbb{S}}_{+}^{d-1}$, which is our notation for the positive orthant of the sphere with radius $\sqrt{2}$ centered at the origin in ${\mathbb{R}}^d$. The distance between points $u,v\in {\Delta }^{d-1}$ under the Fisher metric thus equals the Euclidean length of the great-circle arc connecting $\mathsf{ȷ}\left(u\right),\mathsf{ȷ}\left(v\right)\in \sqrt{2}{\mathbb{S}}_{+}^{d-1}$. The Delaunay mosaic of $\mathsf{ȷ}\left(U\right)$ under the geodesic distance can be obtained by constructing the convex hull of $\mathsf{ȷ}\left(U\right)\cup \left\{0\right\}$ in ${\mathbb{R}}^d$ and centrally projecting all faces not incident to 0 onto the sphere. As before, we cull simplices whose dual Voronoi cells have an empty intersection with the positive orthant of the sphere, and we draw the mosaic in ${\Delta }^{d-1}$ by mapping the vertices back to the original points. Furthermore, the rise functions of the mosaics in $\sqrt{2}{\mathbb{S}}_{+}^{d-1}$ and in ${\Delta }^{d-1}$ are the same. Note, however, that the geodesic radius is the arc-sine of and therefore slightly larger than the straight Euclidean radius in ${\mathbb{R}}^d$.

4. Computational Experiments

We illustrate the Bregman–Alpha and Bregman–Wrap complexes while comparing them to the conjugate, the Fisher, and the Euclidean constructions.

Example in positive quadrant. Let X be a set of 1000 points uniformly distributed according to the Fisher metric in ${(0,2]}^2\subseteq {\mathbb{R}}_{+}^2$. To sample X, we use the isometry, $\mathsf{ȷ}:{\mathbb{R}}_{+}^2\to {\mathbb{R}}_{+}^2$, between the Fisher and the Euclidean metric mentioned in Section 2. Specifically, we sample 1000 points uniformly at random according to the Euclidean metric in ${(0,2]}^2$, and we map each point with coordinates $x_1$, $x_2$ to ${\mathsf{ȷ}}^{-1}(x_1,x_2)=\frac{1}{2}(x_1^2,x_2^2)$, which is again a point in ${(0,2]}^2$. To compute the Delaunay mosaic in Fisher geometry, we construct the (Euclidean) Delaunay mosaic of $\mathsf{ȷ}\left(X\right)$ and draw this mosaic with the vertices at the points in X. Recall however that the domain is $\mathsf{\Omega }={\mathbb{R}}_{+}^d$ and not ${\mathbb{R}}^d$. A simplex whose corresponding Voronoi cell has an empty intersection with the positive orthant thus does not belong to the mosaic, which is restricted to $\mathsf{\Omega }$. We identify these simplices and remove them from the Delaunay mosaic as described in Section 3.

Example in standard triangle. Motivated by our interest in information-theoretic applications, we repeat the above experiment within the standard triangle, ${\Delta }^2$, which consists of all points $(x_1,x_2,x_3)\in {\mathbb{R}}_{+}^3$ that satisfy $x_1+x_2+x_3=1$. Every point in ${\Delta }^2$ can be interpreted as a probability distribution on three disjoint events, which is indeed the most relevant scenario for the application of the relative entropy. To sample a set Y of 1000 points uniformly at random according to the Fisher metric in ${\Delta }^2$, we use again j, now restricted to ${\Delta }^2$, whose image is the positive orthant of the sphere with radius $\sqrt{2}$ centered at the origin of ${\mathbb{R}}^3$. Sampling 1000 points uniformly at random according to the geodesic distance on the sphere, we take the convex hull of $\mathsf{ȷ}\left(Y\right)\cup \left\{0\right\}$ and obtain the mosaic by mapping the vertices to the points in $Y={\mathsf{ȷ}}^{-1}\left(\mathsf{ȷ}\left(Y\right)\right)$. Before drawing the faces in ${\Delta }^2$, we remove 0 and all incident faces, as well as all faces whose corresponding Voronoi cells have an empty intersection with ${\mathbb{R}}_{+}^2$.

Recall that the squared Fisher metric matches the relative entropy in the infinitesimal regime, which explains the random appearance of the reconstruction in Figure 1, for which we set the threshold to $0.0025$. The reconstruction in Shannon geometry is similar to those in conjugate Shannon geometry in Figure 1a and in Fisher geometry in Figure 1b. To interpret the reconstruction in Figure 1d, we observe that the difference between the Shannon entropy and the squared Euclidean norm has a minimum at the center and no other critical points in the interior of the triangular domain. Accordingly, the reconstruction removes simplices near the corners and the three sides first. More drastically, the Bregman–Wrap complex for the same data removes all simplices except for a single critical edge near the center (see Figure 2d).

5. Quantification of Difference

We take a data-centric approach to quantifying the differences between the geometries. Given a common domain, $\mathsf{\Omega }$, and a finite set of points, $U\subseteq \mathsf{\Omega }$, we compare the corresponding mosaics and rise functions.

Mosaics. The Delaunay mosaics of U depend on the local shape of the balls defined by the metric or the divergence. Letting D and E be two Delaunay mosaics with vertex sets U, we compare them by counting the common cells:

$$\begin{array}{cc}\hfill J(D,E)& =1-{\displaystyle \frac{\#(D\cap E)}{\#D+\#E-\#(D\cap E)}},\hfill \end{array}$$

(21)

which is sometimes referred to as the Jaccard distance between the two sets. It is normalized so that $J=0$ iff $D=E$ and $J=1$ iff D and E share no cells at all. In our application, the two mosaics share all vertices, so J is necessarily strictly smaller than 1. To apply this measure to the constructions in Section 4, we write $D_0, D_1, D_2, D_3, D_4$ for the mosaics in Figure A1, and we write $E_0, E_1, E_2, E_3, E_4$ for the mosaics in Figure 3. All mosaics are different, except for $D_0=D_4$ and $E_0=E_4$. The Jaccard distances are given in

Table 1. The Jaccard distances between the Delaunay mosaics in Shannon, conjugate Shannon, Fisher, Euclidean, and weighted Euclidean geometries for points in the positive quadrant on the top and in the standard triangle on the bottom.

J	$D_0$	$D_1$	$D_2$	$D_3$	$D_4$
$D_0$	0.00	0.06	0.04	0.48	0.00
$D_1$		0.00	0.02	0.47	0.06
$D_2$			0.00	0.47	0.04
$D_3$				0.00	0.48
$D_4$					0.00
	$E_0$	$E_1$	$E_2$	$E_3$	$E_4$
$E_0$	0.00	0.10	0.06	0.52	0.00
$E_1$		0.00	0.04	0.51	0.10
$E_2$			0.00	0.51	0.06
$E_3$				0.00	0.52
$E_4$					0.00

.

We see that the mosaics in conjugate Shannon geometry and in Fisher geometry are most similar to each other and less similar to the mosaic in Shannon geometry. The mosaic in Euclidean geometry is most dissimilar to the others (see Figure 3 and related Figure A1 in Appendix A for visual confirmation).

Rise functions. Different rise functions on the same mosaic can be compared by counting the inversions, which are the pairs of cells whose orderings are different under the two functions. Recall that $D_0=D_4$ and $E_0=E_4$, let $d_0:D_0\to \mathbb{R}$ and $e_0: E_0\to \mathbb{R}$ be the rise functions in Shannon geometry, and let $d_4: D_4\to \mathbb{R}$ and $e_4: E_4\to \mathbb{R}$ be the rise functions in weighted Euclidean geometry. The normalized number of inversions are

$$\begin{array}{cc}\hfill I(d_0,d_4)& =0.476,\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill I(e_0,e_4)& =0.467.\hfill \end{array}$$

(23)

In words, slightly fewer than half the pairs are inversions, both for $d_0,d_4$ and for $e_0,e_4$. This is plausible because $d_4$ orders the cells along the diagonal while $d_0$ preserves the random character of the point sample (see Figure A2d). Similarly, $e_4$ orders the cells radially, from the center of the standard triangle to its periphery, while $e_0$ preserves again the random character of the sample (see Figure 4d).

We can compare the rise functions also visually, by color-coding the 2-dimensional cells, and this works even if the mosaics are different. Specifically, we shade the triangles by mapping small to large rise function values onto dark to light colors. In Figure A2a,b, this leads to randomly mixed dark and light triangles, while in Figure A2c,d, there are clear but opposing gradients parallel to the diagonal. Similarly, in Figure 4c, we see the rise function decreasing from the center to the boundary of the standard triangle, and in Figure 4d we see it increasing from the center to the boundary. In addition, we compare general rise functions by computing their persistence diagrams (see [36]). Writing $\mathrm{Dgm}\left(d\right)$ for the persistence diagram of function d, we quantify the difference with the bottleneck between the diagrams:

$$\begin{array}{cc}\hfill B(d,e)& =W_{\infty }(\mathrm{Dgm}\left(d\right),\mathrm{Dgm}\left(e\right)).\hfill \end{array}$$

(24)

As explained in [36], the bottleneck distance is 1-Lipschitz, that is: $B(d,e)\le {\parallel d-e\parallel }_{\infty }$, but $d\ne e$ does not necessarily imply $B(d,e)\ne 0$. The bottleneck distances between the $d_i: D_i\to \mathbb{R}$ and the $e_i: E_i\to \mathbb{R}$ are given in

Table 2. The bottleneck distances between the persistence diagrams of the rise functions on the Delaunay mosaics in Shannon, conjugate Shannon, Fisher, Euclidean, and weighted Euclidean geometries for points in the positive orthant on the top and points in the standard triangle on the bottom.

B	$d_0$	$d_1$	$d_2$	$d_3$	$d_4$
$d_0$	0.0000	0.0028	0.0004	0.0126	0.0048
$d_1$		0.0000	0.0028	0.0126	0.0048
$d_2$			0.0000	0.0126	0.0048
$d_3$				0.0000	0.0126
$d_4$					0.0000
	$e_0$	$e_1$	$e_2$	$e_3$	$e_4$
$e_0$	0.0000	0.0006	0.0003	0.0031	0.0035
$e_1$		0.0000	0.0003	0.0030	0.0034
$e_2$			0.0000	0.0030	0.0034
$e_3$				0.0000	0.0023
$e_4$					0.0000

.

In part, this comparison agrees with the Jaccard distances between the mosaics given in Table 1. The most obvious disagreements are for $d_0,d_4$ and for $e_0,e_4$, in which quite different functions are defined on identical mosaics.

6. Discussion

We extended two popular computational geometric-topological methods within the framework of discrete Morse functions and showed how this generalizes the methods to data in Bregman and Fisher geometries. Importantly, this allowed us to produce a robust implementation without the need to develop customized low-level software. We hope that this result will provide extra motivation for the development of geometric software for high-dimensional Euclidean space, as such software can be reused to handle data in information space. In particular, efficient computation of skeleta of weighted Delaunay mosaics would be relevant for topological analysis of high-dimensional data.

Turning the table, we use these generalized methods to compare different geometries experimentally. Our experimental approach to studying geometries is a first step towards an intuitive understanding of these often counter-intuitive geometries, as well as the high-order interactions occurring in them.

We reiterate one reason why understanding such geometries is important: they underpin many modern data science methods. It is, for example, a prerequisite for explaining the surprisingly good performance of many deep learning models (that output predictions as discrete probability distributions). More broadly, the space of discrete probability distributions is a basic object in mathematics, but is not well understood. We provide new tools for broadening the understanding of such spaces.

Perhaps the main observation coming from our experiments is the following. As we know, the pairwise distance in the Kullback–Leibler geometry and the Euclidean or Fisher one can be very different. Indeed, on the standard simplex, the former can approach infinity while it is bounded in the latter. Still, our experiments show that the geometric-topological structures—at least in low dimensions—do not typically show large discrepancies between the geometries. Is this also true in higher dimensions? How does this generalize to other Bregman divergences and the corresponding generalized Fisher distances [6]? To what extent can the Fisher space replace the Shannon space in various applications?

Several open questions about experimental understanding of geometric spaces arise:

• How do these results generalize to other notions of entropy, such as the Burg entropy [37]? The resulting Bregman Itakura–Saito divergence [38] is used to compare speech samples, but little is known about the resulting geometry and how it compares with other geometries.
• Is there a deeper reason for the similarity of the studied geometries? For example: are there Pinsker-type inequalities [39,40] between the pairwise dissimilarities (as well as other Bregman divergences)?
• Can the sensitivity of Delaunay mosaics to the dissimilarity be quantified probabilistically, as the expected Jaccard distance for random point processes?
• Persistence has been used before to compare metric spaces [41], and it would be interesting to know whether there are deeper connections to our work.
• Can the Fisher metric be extended beyond the positive part of the sphere? What is the geometry of the preimage of the Antonelli map in this case?

We finish with a technical question concerning the Delaunay mosaics in Fisher geometry: is the drawing we obtain by mapping the vertices to the corresponding points and connecting them with straight edges, flat triangles, etc., necessarily a geometric realization of the mosaic?