Relative entropy optimization and its applications

Abstract

In this expository article, we study optimization problems specified via linear and relative entropy inequalities. Such relative entropy programs (REPs) are convex optimization problems as the relative entropy function is jointly convex with respect to both its arguments. Prominent families of convex programs such as geometric programs (GPs), second-order cone programs, and entropy maximization problems are special cases of REPs, although REPs are more general than these classes of problems. We provide solutions based on REPs to a range of problems such as permanent maximization, robust optimization formulations of GPs, and hitting-time estimation in dynamical systems. We survey previous approaches to some of these problems and the limitations of those methods, and we highlight the more powerful generalizations afforded by REPs. We conclude with a discussion of quantum analogs of the relative entropy function, including a review of the similarities and distinctions with respect to the classical case. We also describe a stylized application of quantum relative entropy optimization that exploits the joint convexity of the quantum relative entropy function.

Appendix

The second-order cone $L_2 \subset \mathbb {R}^3$ from (5) can be written as:

$$\begin{aligned} L_2 = \left\{ (\mathbf {x},y) \in \mathbb {R}^2 \times \mathbb {R}~|~ \begin{pmatrix} y-\mathbf {x}_1 &{} \mathbf {x}_2 \\ \mathbf {x}_2 &{} y+\mathbf {x}_1 \end{pmatrix} \succeq \mathbf {0}\right\} . \end{aligned}$$

Combining this reformulation with the next result gives us the description (6).

Proposition 6

We have that $\begin{pmatrix} a &{} c \\ c &{} b\end{pmatrix} \in \mathbb {S}^2_+$ if and only if there exists $\nu \in \mathbb {R}_+$ such that:

$$\begin{aligned} \begin{aligned} \nu \log \left( \frac{\nu }{a}\right) + \nu \log \left( \frac{\nu }{b}\right) - 2 \nu&\le 2 c \\ \nu \log \left( \frac{\nu }{a}\right) + \nu \log \left( \frac{\nu }{b}\right) - 2 \nu&\le - 2 c \\ a,b \in \mathbb {R}_+. \end{aligned} \end{aligned}$$

Proof

We have that $\begin{pmatrix} a &{} c \\ c &{} b\end{pmatrix} \in \mathbb {S}^2_+$ if and only if $a \bar{\mathbf {z}}_1^2 + b \bar{\mathbf {z}}_2^2 + 2c \bar{\mathbf {z}}_1 \bar{\mathbf {z}}_2 \ge 0, ~ \forall \bar{\mathbf {z}} \in \mathbb {R}^2$. This latter condition can in turn be rewritten to obtain the following reformulation:

$$\begin{aligned} \begin{pmatrix} a &{} c \\ c &{} b\end{pmatrix} \in \mathbb {S}^2_+ \Leftrightarrow a \mathbf {z}_1^2 + b \mathbf {z}_2^2 + 2c \mathbf {z}_1 \mathbf {z}_2 \ge 0 \,\,{\mathrm {and}}\,\, a \mathbf {z}_1^2 + b \mathbf {z}_2^2 - 2c \mathbf {z}_1 \mathbf {z}_2 \ge 0 ~ \forall \mathbf {z}\in \mathbb {R}^2_+. \end{aligned}$$

(44)

Each of these inequalities with universal quantifiers can be reformulated by appealing to GP duality. Specifically, based on the change of variables $\mathbf {w}_i \leftarrow \log (\mathbf {z}_i), i=1,2$, which is commonly employed in the GP literature [11, 20], we have from (44) that $\begin{pmatrix} a &{} c \\ c &{} b\end{pmatrix} \in \mathbb {S}^2_+$ if and only if:

$$\begin{aligned} \inf _{\mathbf {w}\in \mathbb {R}^2} ~ a \exp \{\mathbf {w}_1 - \mathbf {w}_2\} + b \exp \{\mathbf {w}_2 - \mathbf {w}_1\} ~~ \ge ~~ \max \{2c,-2c\}. \end{aligned}$$

(45)

As the optimization problem on the left-hand-side is a GP for $a,b \in \mathbb {R}_+$, we can appeal to convex duality to conclude that

$$\begin{aligned} \inf _{\mathbf {w}\in \mathbb {R}^2} ~ a \exp \{\mathbf {w}_1 - \mathbf {w}_2\} + b \exp \{\mathbf {w}_2 - \mathbf {w}_1\} = \sup _{\nu \in \mathbb {R}_+} ~ -\nu \log \left( \frac{\nu }{a}\right) - \nu \log \left( \frac{\nu }{b}\right) + 2\nu . \end{aligned}$$

Combining this result with (45), we have that $\begin{pmatrix} a &{} c \\ c &{} b\end{pmatrix} \in \mathbb {S}^2_+$ if and only if:

$$\begin{aligned} a,b \in \mathbb {R}_+ ~{\mathrm {and}}~ \exists \nu \in \mathbb {R}_+ ~{\mathrm {s.t.}}~ -\nu \log \left( \frac{\nu }{a}\right) - \nu \log \left( \frac{\nu }{b}\right) + 2\nu \ge \max \{2c,-2c\}. \end{aligned}$$

(46)

This concludes the proof. $\square$