From Bayesian Inference to MCMC and Convex Optimisation in Hadamard Manifolds

Abstract

The present work is motivated by the problem of Bayesian inference for Gaussian distributions in symmetric Hadamard spaces (that is, Hadamard manifolds which are also symmetric spaces). To investigate this problem, it introduces new tools for Markov Chain Monte Carlo, and convex optimisation: (1) it provides easy-to-verify sufficient conditions for the geometric ergodicity of an isotropic Metropolis-Hastings Markov chain, in a symmetric Hadamard space, (2) it shows how the Riemannian gradient descent method can achieve an exponential rate of convergence, when applied to a strongly convex function, on a Hadamard manifold. Using these tools, two Bayesian estimators, maximum-a-posteriori and minimum-mean-squares, are compared. When the underlying Hadamard manifold is a space of constant negative curvature, they are found to be surprisingly close to each other. This leads to an open problem: are these two estimators, in fact, equal (assuming constant negative curvature)?

A Symmetric Hadamard Spaces

The present work focuses on symmetric Hadamard spaces. These are Hadamard manifolds which are also symmetric spaces. This class of spaces includes: spaces of constant negative curvature (precisely, hyperbolic spaces), cones of covariance matrices (real, complex, or quaternion), as well as Siegel domaines (recently used in describing spaces of block-Toeplitz covariance matrices). These spaces play a prominent role in several applications [2, 4, 8].

1.1 A.1 Hadamard Manifolds

Definition

A Hadamard manifold is a Riemannian manifold which is complete, simply connected, and has negative sectional curvatures [5]. Completeness means that geodesics extend indefinitely in both directions, and is equivalent to the property that closed and bounded sets are compact (by the Hopf-Rinow theorem [5]). The simple connectedness and negative curvature properties combine to ensure that the Riemannian exponential map is a diffeomorphism. In particular, any two points x, y in a Hadamard manifold M are connected by a unique length-minimising geodesic $c:[0,1]\rightarrow M$, with $c(0) = x$, and $dc/dt|_{t=0} = \mathrm {Exp}^{-1}_x(y)$.

Convexity

Now, this geodesic $c:[0,1]\rightarrow M$ is called the segment between x and y. For $t \in [0,1]$, the point $x\, \#_{t}\, y = c(t)$ is said to be a geodesic convex combination of x and y, with respective weights $(1-t)$ and t. In a Hadamard manifold [5], any open geodesic ball $B(x^*,R)$ (or any closed geodesic ball $\bar{B}(x^*,R)$) is convex: if x and y belong to this ball, so does $x\, \#_{t}\, y$ for all $t \in (0,1)$.

A function $f:M \rightarrow \mathbb {R}$ is called $(\alpha /2)$-strongly convex, if $(f\circ c)(t)$ is an $(\alpha /2)$-strongly convex function of t, for any geodesic $c:[0,1]\rightarrow M$. That is, if there exists $\alpha > 0$, such that for any x and y in M, and $t \in [0,1]$,

$$\begin{aligned} f(x\, \#_{t}\, y) \le (1-t)f(x) + tf(y) - (\alpha /2)t(1-t)d^{ 2}(x,y) \end{aligned}$$

(19)

When f is $C^2$-smooth, this is equivalent to (14).

Squared Distance Function

For $y \in M$, consider the function $f_y:M\rightarrow \mathbb {R}$, where $f_y(x) = d^{ 2}(x,y)/2$. If M is a Hadamard manifold, then this function has several remarkable properties.

Specifically, it is both smooth and (1/2)-strongly convex (irrespective of y). It’s gradient is given by $\mathrm {grad}\,f_y(x) = -\mathrm {Exp}^{-1}_x(y)$, in terms of the Riemannian exponential map. On the other hand, on any geodesic ball B(y, R), the operator norm of its Hessian is bounded by $cR\coth (cR)$, where $c > 0$ is such that the sectional curvatures of M lie in the interval $[-c^{2},0]$ (see [5], Theorem 27).

A result of the strong convexity of these functions $f_y$ is the uniqueness of Riemannian barycentres. If $\pi$ is a probability distribution on M, its Riemannian barycentre $\hat{x}_{\pi }$ is the unique global minimiser of the so-called variance function $\mathcal {E}_{\pi }(x) \,=\, \int _M\,f_y(x)\pi (dy)$. If M is a Hadamard manifold, each function $f_y$ is (1/2)-strongly convex, and therefore $\mathcal {E}_{\pi }$ is (1/2)-strongly convex. Thus, $\mathcal {E}_{\pi }$ has a unique global minimiser.

1.2 A.2 Symmetric Spaces

Definition

A Riemannian symmetric space is a Riemannian manifold M, such that for each $x \in M$ there exists an isometry $s_{x}:M \rightarrow M$, which fixes x and reverses geodesics passing through x: if c is a geodesic curve with $c(0) = x$, then $s_x(c(t)) = c(-t)$. As a consequence of this definition, a Riemannian symmetric space is always complete and homogeneous. Completeness was discussed in Appendix A.1. On the other hand, homogeneity means that for each x and y in M, there exists an isometry $g:M\rightarrow M$, with $g(x) = y$. A symmetric space M may always be expressed as a quotient manifold $M = G/K$, where G is a certain Lie group of isometries of M, and K is the subgroup of elements of G which fix some point $x \in M$.

A Familiar Example

A familiar example of a symmetric Hadamard manifold is the cone $\mathcal {P}_d$ of $d \times d$ symmetric positive-definite matrices, which was discussed in some length in [8, 9]. A geodesic curve through $x \in \mathcal {P}_d$ is of the form $c(t) = x\exp (tx^{-1}u)$ where $\exp$ is the matrix exponential, and u a symmetric matrix, $dc/dt|_{t=0} = u$. The isometry $s_x$ is given by $s_x(y) = xy^{-1}x$ for $y \in \mathcal {P}_d$, and one easily checks $s_x(c(t)) = c(-t)$.

Let $G = \mathrm {GL}(d,\mathbb {R})$ denote the group of invertible $d \times d$ real matrices. Then, $g \in G$ defines an isometry $\tau _g:\mathcal {P}_d \rightarrow \mathcal {P}_d$, given by $\tau _g(x) = gxg^\dagger$ ($\dagger$ the transpose). Typically, one writes $\tau _g(x) = g(x)$. Note that $x = \mathrm {I}_d$ (the identity matrix) belongs to $\mathcal {P}_d$, and then $g(x) = x$ if and only if $g \in K$, where $K = O(d)$ is the orthogonal group. In fact, one has the quotient manifold structure $\mathcal {P}_d = \mathrm {GL}(d,\mathbb {R})/O(d)$.