Sequential optimality conditions for nonlinear optimization on Riemannian manifolds and a globally convergent augmented Lagrangian method

Abstract

Recently, the approximate Karush–Kuhn–Tucker (AKKT) conditions, also called the sequential optimality conditions, have been proposed for nonlinear optimization in Euclidean spaces, and several methods to find points satisfying such conditions have been developed by researchers. These conditions are known as genuine necessary optimality conditions because all local optima satisfy them with no constraint qualification (CQ). In this paper, we extend the AKKT conditions to nonlinear optimization on Riemannian manifolds and propose an augmented Lagrangian (AL) method that globally converges to points satisfying such conditions. In addition, we prove that the AKKT and KKT conditions are indeed equivalent under a certain CQ. Finally, we examine the effectiveness of the proposed AL method via several numerical experiments.

Appendix A

We present the proof of Lemma 2.

Proof of Lemma 2

We prove this lemma by contradiction. Suppose that $\mathop{{\rm grad}}\theta (w) \not = 0$. Since the optimal solution $w \in {{\mathcal {M}}}$ satisfies $w \in \mathrm{int}B(x^{*}, \delta )$, there exists $\varepsilon > 0$ such that $B(w, \varepsilon ) \subset B(x^{*}, \delta )$. Let $t > 0$ and $u := - \mathop{{\rm grad}}\theta (w)$. Note that $u \not = 0$. We consider the Taylor expansion $\theta (\mathrm{Exp}_{w}(tu)) = \theta (w) + t \langle \mathop{{\rm grad}}\theta (w), u \rangle _{w} + t r(t)$, where $r : \mathbb {R}_{+} \rightarrow \mathbb {R}$ is a certain function satisfying $r(t) \rightarrow 0$ as $t \rightarrow 0$. This result implies that

$$\begin{aligned} \textstyle \theta (\mathrm{Exp}_{w}(tu)) = \theta (w) + t \{ r(t) - \Vert u \Vert _{w}^{2} \}. \end{aligned}$$

(A.1)

Since $\mathrm{Exp}_{w}(tu) \rightarrow w$ and $r(t) \rightarrow 0$ as $t \rightarrow 0$, there exists $t_{0} > 0$ such that $\mathrm{Exp}_{w}(t_{0}u) \in B(w, \varepsilon )$ and $r(t_{0}) \le \frac{1}{2} \Vert u \Vert _{w}^{2}$. Note that $v := \mathrm{Exp}_{w}(t_{0} u)$ is feasible because $v \in B(w, \varepsilon ) \subset B(x^{*}, \delta )$. Substituting $t = t_{0}$ into (A.1) yields $\theta (v) \le \theta (w) - \frac{t_{0}}{2} \Vert u \Vert _{w}^{2} < \theta (w)$. However, this contradicts the assumption that w is an optimal solution. $\square$

Lemma 3 is shown in the following.

Proof of Lemma 3

Let $S_{+} := \{ j \in \mathbb {N}; \, [\xi ]_{j} \ge 0 \}$, $S_{-} := \{ j \in \mathbb {N}; \, [\xi ]_{j} < 0 \}$, $T_{+} := \{ j \in \mathbb {N}; \, [\xi ]_{j} \ge [\zeta ]_{j} \}$, and $T_{-} := \{ j \in \mathbb {N}; \, [\xi ]_{j} < [\zeta ]_{j} \}$. The left-hand side of the inequality is rewritten as

$$\begin{aligned} \Vert [-\xi ]_{+} \Vert = \left[ \sum _{j=1}^{n} \max \{ -[\xi ]_{j}, 0 \}^{2} \right] ^{\frac{1}{2}} = \left[ \sum _{j \in S_{-}} [\xi ]_{j}^{2} \right] ^{\frac{1}{2}}. \end{aligned}$$

(A.2)

Meanwhile, we obtain

$$\begin{aligned} \Vert \min \{ \xi , \zeta \} \Vert = \left[ \sum _{j=1}^{n} \min \{ [\xi ]_{j}, [\zeta ]_{j} \}^{2} \right] ^{\frac{1}{2}} = \left[ \sum _{j \in T_{+}} [\zeta ]_{j}^{2} + \sum _{j \in T_{-}} [\xi ]_{j}^{2} \right] ^{\frac{1}{2}}. \end{aligned}$$

(A.3)

Note that $S_{-} \subset T_{-}$ is a sufficient condition under which the desired inequality holds. Indeed, if $S_{-} \subset T_{-}$, then from (A.2) and (A.3), we have

$$\begin{aligned} \Vert [-\xi ]_{+} \Vert = \left[ \sum _{j \in S_{-}} [\xi ]_{j}^{2} \right] ^{\frac{1}{2}} \le \left[ \sum _{j \in T_{-}} [\xi ]_{j}^{2} \right] ^{\frac{1}{2}} \le \left[ \sum _{j \in T_{+}} [\zeta ]_{j}^{2} + \sum _{j \in T_{-}} [\xi ]_{j}^{2} \right] ^{\frac{1}{2}} = \Vert \min \{ \xi , \zeta \} \Vert . \end{aligned}$$

We take an arbitrary element $j \in S_{-}$. The definition of $S_{-}$ and $\zeta \in \mathbb {R}_{+}^{n}$ imply that $[\xi ]_{j} < 0 \le [\zeta ]_{j}$. Note that the definition of $T_{-}$ yields $j \in T_{-}$, i.e., $S_{-} \subset T_{-}$. Therefore, the proof is completed. $\square$