Bregman Divergences from Comparative Convexity

Abstract

Comparative convexity is a generalization of ordinary convexity based on abstract means instead of arithmetic means. We define and study the Bregman divergences with respect to comparative convexity. As an example, we consider the convexity induced by quasi-arithmetic means, report explicit formulas, and show that those Bregman divergences are equivalent to conformal ordinary Bregman divergences on monotone embeddings.

A Quasi-arithmetic to Ordinary Convexity Criterion

Lemma 1

( $(\rho ,\tau )$ -convexity $\leftrightarrow$ ordinary convexity [1]). Let $\rho :I\rightarrow \mathbb {R}$ and $\tau :J\rightarrow \mathbb {R}$ be two continuous and strictly monotone real-valued functions with $\tau$ increasing, then function $F:I\rightarrow J$ is $(\rho ,\tau )$-convex iff function $G=F_{\rho ,\tau } = \tau \circ F\circ \rho ^{-1}$ is (ordinary) convex on $\rho (I)$.

Proof

Let us rewrite the $(\rho ,\tau )$-convexity midpoint inequality as follows:

$$\begin{aligned} F(M_\rho (x,y))\le & {} M_\tau (F(x),F(y)),\\ F\left( \rho ^{-1}\left( \frac{\rho (x)+\rho (y)}{2}\right) \right)\le & {} \tau ^{-1}\left( \frac{\tau (F(x))+\tau (F(y))}{2}\right) , \end{aligned}$$

Since $\tau$ is strictly increasing, we have:

$$\begin{aligned} (\tau \circ F\circ \rho ^{-1}) \left( \frac{\rho (x)+\rho (y)}{2}\right) \le \frac{(\tau \circ F)(x)+(\tau \circ F)(y)}{2}. \end{aligned}$$

(17)

Let $u=\rho (x)$ and $v=\rho (y)$ so that $x=\rho ^{-1}(u)$ and $y=\rho ^{-1}(v)$ (with $u,v\in \rho (I)$). Then it comes that:

$$\begin{aligned} (\tau \circ F\circ \rho ^{-1})\left( \frac{u+v}{2}\right) \le \frac{(\tau \circ F\circ \rho ^{-1})(u)+(\tau \circ F\circ \rho ^{-1})(v)}{2}. \end{aligned}$$

(18)

This last inequality is precisely the ordinary midpoint convexity inequality for function $G=F_{\rho ,\tau }=\tau \circ F\circ \rho ^{-1}$. Thus a function F is $(\rho ,\tau )$-convex iff $G=\tau \circ F\circ \rho ^{-1}$ is ordinary convex, and vice-versa.