An inexact proximal point algorithm for maximal monotone vector fields on Hadamard manifolds

Abstract

In this paper, an inexact proximal point algorithm concerned with the singularity of maximal monotone vector fields is introduced and studied on Hadamard manifolds, in which a relative error tolerance with squared summable error factors is considered. It is proved that the sequence generated by the proposed method is convergent to a solution of the problem. Moreover, an application to the optimization problem on Hadamard manifolds is given. The main results presented in this paper generalize and improve some corresponding known results given in the literature.

Introduction

The theory of maximal monotone operators provides a powerful general framework for the study of convex programming problems and variational inequalities. A basic problem in the theory of maximal monotone operators is to find $x\in {\mathbb{R}}^n$ such that $0\in T\left(x\right)$, where $T$ is a multivalued maximal monotone operator from ${\mathbb{R}}^n$ to itself. There is an extensive literature concerning this classical problem (see, for example,  [10], [23], [28], [27], [26], [33]).

In this paper, we will extend an inexact proximal point algorithm with a relative error tolerance from Euclidean spaces to Hadamard manifolds. Thus, we start with an introduction of related inexact proximal point algorithms on Euclidean spaces. The proximal point algorithm is one of the most important methods for solving the original problem, which, starting with any vector $x^0\in {\mathbb{R}}^n$, iteratively updates $x^{k+1}$ conforming to the following recursion

$$0\in c_{k}T\left(x^{k+1}\right)+x^{k+1}-x^k\text{,}$$

where $\left\{c_k\right\}\subset [c,+\infty ),c>0$, is a sequence of scalars. However, as pointed out in  [23], [10], the ideal form of the method is often impractical, since in many cases, solving problem (1.1) exactly is either impossible or as difficult as solving the original problem $0\in T\left(x\right)$. In  [23], Rockafellar gave an inexact variant of the method

$$e^k\in c_{k}T\left({\tilde{x}}^k\right)+{\tilde{x}}^k-x^k\text{,}$$

where $\left\{e^k\right\}$ is regarded as an error sequence. This method is called an inexact proximal point algorithm. Rockafellar  [23] provided the following two classes of error criteria:

$$\Vert e^k\Vert \le {\eta }_k\text{with }\sum _{k=0}^{+\infty }{\eta }_k<+\infty \text{,}$$

and

$$\Vert e^k\Vert \le {\eta }_k\Vert {\tilde{x}}^k-x^k\Vert \text{with }\sum _{k=0}^{+\infty }{\eta }_k<+\infty \text{.}$$

Under suitable assumptions and letting $x^{k+1}={\tilde{x}}^k$, Rockafellar  [23] proved the global convergence result and locally linear rate of convergence, respectively. Since then, the summability of errors or error factors has been the standard assumption for ensuring the convergence of inexact proximal and proximal-like methods. The error criterions (1.3), (1.4) can be seen as a absolute error and a relative error, respectively.

From the point of view of numerical analysis, relative errors are easier to estimate and analyze. Therefore, some researchers concentrated their attention on the inexact proximal point algorithms with relative errors. Solodov and Svaiter proposed the following two relative error tolerances:

$$\Vert e^k\Vert \le \eta \Vert {\tilde{x}}^k-x^k\Vert \text{with}\eta \in [0,1)$$

in  [26], and

$$\Vert e^k\Vert \le \eta max\{\underset{k}{\overset{-1}{c}}\Vert v^k\Vert ,\Vert {\tilde{x}}^k-x^k\Vert \}\text{with }\eta \in [0,1)\text{and }{v}^k\in T\left(x^k\right)$$

in  [27], respectively. Note that the error factor $\eta$ in both relative error tolerances above can be a constant in $[0,1)$. As a consequence, from the point of view of computation, (1.5), (1.6) are more attractive than (1.4). However, from two examples supplied by Solodov and Svaiter (see,  [26], [27]), we know that the traditional inexact proximal point algorithm (i.e., letting $x^{k+1}≔{\tilde{x}}^k$) may not converge under the relative error tolerances (1.5), (1.6). To ensure the convergence of the proximal point algorithms under (1.5), (1.6), either extragradient step  [26] or projection step  [27] was required. Therefore, the two methods proposed by Solodov and Svaiter  [26], [27] were named as hybrid extragradient-proximal algorithm and hybrid projection-proximal algorithm, respectively.

Later, without adding an additional extragradient or projection step to the algorithm, Han and He  [10] introduced the following error criterion:

$$\Vert e^{k+1}\Vert \le {\eta }_k\Vert x^{k+1}-x^k\Vert \text{with }\sum _{k=0}^{\infty }{\eta }_k^2<+\infty \text{.}$$

It is clear that the error criterion (1.7) is weaker than the one (1.4).

The extension to Riemannian manifolds of the concepts and techniques that fit in Euclidean spaces is natural and nontrivial. Actually, in recent years a large number of researchers have been making great efforts to this topic (see, for example,  [1], [2], [3], [4], [5], [6], [7], [8], [9], [13], [14], [15], [12], [16], [18], [17], [19], [21], [22], [20], [32], [11], [31], [29], [30]).

In particular, Li et al.  [13] studied the proximal point algorithm for singularities (solutions of the inclusion $0\in A\left(x\right)$) of a maximal monotone vector field $A$ on a Hadamard manifold $\mathbb{M}$, which extends the earlier results of Rockafellar  [23] from Euclidean spaces to Hadamard manifolds. Its iterative scheme is as follows: given $x^0\in \mathbb{M}$ and $\left\{c_k\right\}\subset [c,+\infty ),c>0$, define $x^{k+1}$ such that

$$0\in c_{k}A\left(x^{k+1}\right)-{exp}_{x^{k+1}}^{-1}{x}^k\text{.}$$

Up to now, most of the proximal point algorithms on Riemannian manifolds are exact versions (see, for example,  [13], [8], [4], [22], [20], [30]). However, since the proximal point algorithms are implicit methods in essence, different from projection-type methods, the cost of solving subproblems exactly is quite expensive at each iteration step. As mentioned by Quiroz and Oliveira  [20], for a computational implementation, it is important to analyze the convergence of the algorithm with an inexact iteration. Recently, Wang and López  [31] proposed a modified proximal point algorithm on Hadamard manifolds, which extended the corresponding results of Xu  [33] to Hadamard manifolds. The method consists of proximal step and Halpern’s iteration, where the proximal step is an inexact one with the standard error criterion (1.3) (see Algorithm MP and its variant of  [31]). As discussed above, when applying inexact proximal point algorithms for solving related problems, we prefer the relative error tolerance to the absolute one. However, to the best of our knowledge, we cannot find any inexact proximal point algorithm with a relative error tolerance on Riemannian manifolds.

Inspired and motivated by the research works above, in this paper, we extend an inexact proximal point algorithm with the relative error tolerance proposed by Han and He  [10] from Euclidean spaces to Hadamard manifolds. Under suitable assumptions, we prove the sequence generated by the proposed method converges to the singularity of maximal monotone vector fields on Hadamard manifolds. Since the sequence generated by the proposed method is not Fejér (or quasi Fejér) convergent to the solution set of the problem, our techniques in this paper are mostly different with the previous ones for dealing with the Fejér (or quasi Fejér) convergent sequence on Hadamard manifolds. Moreover, we give an application to the optimization problem on Hadamard manifolds, which also generalizes and improves the corresponding results of Ferreira and Oliveira  [8] and Li et al.  [13].