Accelerated forward–backward algorithms for structured monotone inclusions

Abstract

In this paper, we develop rapidly convergent forward–backward algorithms for computing zeroes of the sum of two maximally monotone operators. A modification of the classical forward–backward method is considered, by incorporating an inertial term (closed to the acceleration techniques introduced by Nesterov), a constant relaxation factor and a correction term, along with a preconditioning process. In a Hilbert space setting, we prove the weak convergence to equilibria of the iterates $(x_n)$, with worst-case rates of $o(n^{-1})$ in terms of both the discrete velocity and the fixed point residual, instead of the rates of $\mathcal {O}(n^{-1/2})$ classically established for related algorithms. Our procedure can be also adapted to more general monotone inclusions. In particular, we propose a fast primal-dual algorithmic solution to some class of convex-concave saddle point problems. In addition, we provide a well-adapted framework for solving this class of problems by means of standard proximal-like algorithms dedicated to structured monotone inclusions. Numerical experiments are also performed so as to enlighten the efficiency of the proposed strategy.

Appendix

1.1 Appendix A.1: Proof of Lemma 2.2

It is divided into the following several parts.

1.1.1 Appendix A.1.1: Proof of Lemma 2.2 (part 1): an estimation from a more general model

The next proposition was established in [32, Proposition 3.2].

Proposition A.1

Let $\{ x_n, y_n, d_n\} \subset \mathcal {H}$ verify (2.16) along with $\{e_*, \kappa , \nu _n\} \subset (0,\infty )$, and suppose that condition (2.11a) holds (for some integer $n_1$). Then for $(s,q) \in ( 0, \infty ) \times \mathcal {H}$ and for $n \ge n_1$ we have

$$\begin{aligned}&\dot{G}_{n+1}(s,q) + \frac{1}{2} \rho _n^{-1} (e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \nonumber \\&\quad + s (e_*+\nu _{n+1}) \langle d_{n}, x_{n+1} -q \rangle \nonumber \\&\quad + \left( 1- s \frac{\rho _n}{e_*+\nu _{n+1}}\right) \rho _n^{-1} (e_*+\nu _{n+1})^2 \langle d_{n} ,\dot{x}_{n+1} \rangle = -{T}_{n}(s) , \end{aligned}$$

(A.1)

where $\rho _n$ and ${T} _{n}(s)$ are given by (2.9) and (2.10), respectively, while $G_n(s,q)$ is defined by

$$\begin{aligned} {G_n(s,q) = \frac{1}{2} \Vert s (q-x_{n} )+ \nu _n(y_{n} - x_{n} ) \Vert ^2 + \frac{1}{2} s(e_* -s) \Vert x_{n} -q\Vert ^2}. \end{aligned}$$

(A.2)

1.1.2 Appendix A.1.2: Proof of Lemma 2.2 (part 2)

At once we use Proposition A.1 so as to get estimates related to our proposed model. The following lemma is stated in the general setting of the parameter $\gamma _n$:

Lemma A.2

Let $\{ z_n, x_n, y_n \} \subset \mathcal {H}$ be generated by (1.8) along with $(\theta _n)$ given in (1.9) and parameters $\{\gamma _n, e_*, \kappa , t, \mu , \nu _n \} \subset (0,\infty )$ verifying (2.11a) (for some integer $n_1$). Then, for any $(s,q) \in (0, \infty ) \times \mathcal {H}$ and for $n \ge n_1$, we have

$$\begin{aligned}&\dot{\mathcal E}_{n+1}(s,q) + \frac{1}{2} \rho _n^{-1} (e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \nonumber \\&\quad + s \big ((e_*+\nu _{n}) -\gamma _n (e_*+\nu _{n+1})\big ) \langle t G(z_{n-1}),x_{n} -q \rangle \nonumber \\&\quad + \rho _n^{-1} (e_*+\nu _{n+1})^2 \big ( (\vartheta _{n}- \gamma _n) \langle t G(z_{n-1}), \dot{x}_{n+1} \rangle + \vartheta _{n}Z_n \big )= - { T}_{n}(s), \end{aligned}$$

(A.3)

where $\vartheta _{n}=1-s\rho _n (e_*+\nu _{n+1}) ^{-1}$ and $Z_n=\langle t G (z_n)- t G (z_{n-1}),\dot{x}_{n+1} \rangle$.

Proof

It can be observed (in light of Remark 2.2) that the iterates $\{z_n, x_n, y_n \}$ generated by CRIFBA enter the special case of algorithm (2.16) when $d_n= t (G z_n)- t \gamma _n (G z_{n-1})$. Hence by Proposition A.1, and denoting $W_n= \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2$ (for the sake of simplicity), we obtain

$$\begin{aligned} \dot{G}_{n+1}(s,q) + \frac{1}{2}\rho _n^{-1} \tau _n^2 W_n + (s \tau _n) \langle d_n, x_{n+1} -q \rangle + \vartheta _{n}\rho _n^{-1} \tau _n^2\langle d_n,\dot{x}_{n+1} \rangle = -{ T}_{n}(s) , \end{aligned}$$

(A.4)

where $\tau _n=e_*+ \nu _{n+1}$ and $\vartheta _{n}= 1 -s \frac{\rho _n}{\tau _n}$ ($\vartheta _{n}$ being also depending on s). Moreover, setting $U_{n} =\langle t G(z_{n-1}), x_{n} -q\rangle$ and $H_n=\langle t G (z_{n-1}), \dot{x}_{n+1} \rangle$ we have

$$\begin{aligned} \langle d_n, x_{n+1} -q \rangle&= \langle t G (z_n)- t \gamma _n G (z_{n-1}), x_{n+1} -q \rangle \\&=\langle t G (z_n), x_{n+1} -q \rangle - \gamma _n \langle t G(z_{n-1}), \dot{x}_{n+1} \rangle - \gamma _n \langle t G (z_{n-1}), x_{n} -q \rangle \\&=U_{n+1} - \gamma _n H_n - \gamma _n U_n, \end{aligned}$$

while denoting $Z_n= \langle t G (z_n) - t G (z_{n-1}),\dot{x}_{n+1} \rangle$ (hence $\langle G(z_n),\dot{x}_{n+1} \rangle = Z_n + H_n$) gives us

$$\begin{aligned} \langle d_n, \dot{x}_{n+1} \rangle&= \langle t G (z_n)- t \gamma _n G (z_{n-1}), \dot{x}_{n+1} \rangle \nonumber \\&= \langle t G (z_n),\dot{x}_{n+1} \rangle - \gamma _n \langle t G(z_{n-1}) , \dot{x}_{n+1} \rangle = Z_n + H_n - \gamma _n H_n . \end{aligned}$$

(A.5)

Then, using the previous two results and checking that $s\tau _n + \vartheta _{n}\rho _n^{-1} \tau _n^2= \rho _n^{-1} \tau _n^2$ amounts to

$$\begin{aligned}&(s \tau _n) \langle d_n, x_{n+1} -q \rangle + \vartheta _{n}\rho _n^{-1} \tau _n^2 \langle d_n,\dot{x}_{n+1} \rangle \\&\quad = (s\tau _n) \big ( U_{n+1} - \gamma _n U_n \big ) - \big (s\tau _n + \vartheta _{n}\rho _n^{-1} \tau _n^2\big ) \gamma _n H_n + \vartheta _{n}\rho _n^{-1} \tau _n^2 (Z_n + H_n) \\&\quad = (s\tau _n) \big ( U_{n+1} - \gamma _n U_n \big ) - \rho _n^{-1} \tau _n^2 \gamma _n H_n + \vartheta _{n}\rho _n^{-1} \tau _n^2 (Z_n + H_n). \end{aligned}$$

Thus, in light of (A.4) along with the previous equality, we infer that

$$\begin{aligned}&\dot{G}_{n+1}(s,q) + \frac{1}{2} \rho _n^{-1} \tau _n^2 W_n + (s \tau _n) (U_{n+1} -\gamma _{n}U_n) + { T}_{n}(s) \nonumber \\&= - \rho _n^{-1} \tau _n^2 \big ( - \gamma _n H_n + \vartheta _{n}(Z_n + H_n) \bigg ) = - \rho _n^{-1} \tau _n^2 \bigg ( (\vartheta _{n}- \gamma _n) H_n + \vartheta _{n}Z_n \big ). \end{aligned}$$

(A.6)

It is also simply seen that

$$\begin{aligned} \tau _n (U_{n+1}- \gamma _nU_{n}) = \tau _n U_{n+1}- \tau _{n-1} U_{n}+ \left( \tau _{n-1} -\tau _n \gamma _n\right) U_n, \end{aligned}$$

which by (A.6) and noticing that ${\mathcal E}_{n}(s,q)={G}_{n}(s,q) + s \tau _{n-1} U_{n}$ leads to

$$\begin{aligned}&\dot{\mathcal E}_{n+1}(s,q) + \frac{1}{2} \rho _n^{-1} \tau _n^2 W_n + s\left( \tau _{n-1} -\tau _n \gamma _n\right) U_n + { T}_{n}(s)\\&\quad = - \rho _n^{-1} \tau _n^2 \big ( (\vartheta _{n}- \gamma _n) H_n + \vartheta _{n}Z_n \big ). \end{aligned}$$

This completes the proof. $\square$

1.1.3 Appendix A.1.3: Proof of Lemma 2.2 (Part 3)

We prove the two results (2.12) and (2.14) successively.

Let us begin with proving (2.12). Suppose that (2.11a) holds (for some integer $n_1$). For $n \ge n_1$, by $\vartheta _{n}= 1-s \rho _n (e_*+\nu _{n+1})^{-1}$ and $\gamma _n= 1-s_0 \rho _n (e_*+\nu _{n+1})^{-1}$, we readily have

$$\begin{aligned}&\vartheta _{n}- \gamma _n=s_0 \rho _n (e_*+\nu _{n+1})^{-1}- s \rho _n (e_*+\nu _{n+1})^{-1} = \rho _n (e_*+\nu _{n+1})^{-1} (s_0-s), \end{aligned}$$

(A.7a)

$$\begin{aligned}&(e_*+\nu _{n}) -(e_*+\nu _{n+1}) {\gamma _n} = (e_*+\nu _{n}) - (e_*+\nu _{n+1}) + s_0 \rho _n = s_0 \rho _n-\dot{\nu }_{n+1}. \end{aligned}$$

(A.7b)

Consequently, by (A.3) and (A.7a) we are led to (2.12), namely

$$\begin{aligned}&\dot{\mathcal E}_{n+1}(s,q) + \frac{1}{2} \rho _n^{-1} (e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \nonumber \\&\quad + s \left( s_0 \rho _n -\dot{\nu }_{n+1} \right) \langle t G(z_{n-1}), x_{n} -q\rangle \nonumber \\&\quad + (s_0-s)(e_*+\nu _{n+1}) H_n + \vartheta _{n}\rho _n^{-1} (e_*+\nu _{n+1})^2 Z_n = - { T}_{n}(s), \end{aligned}$$

(A.8)

where we recall that $Z_n=\langle t G (z_n)- t G (z_{n-1}),\dot{x}_{n+1} \rangle$.

Now we prove (2.14). Suppose that (2.11b) holds (for some large enough integer $n_1$ and some value $\theta$) and that (2.13) is satisfied. From the condition $\nu _{n+1} \sim \nu _n$ (as $n \rightarrow \infty$) we clearly have $\lim _{n \rightarrow \infty } (1-\kappa ) \frac{\nu _{n+1}}{\nu _n}=1-\kappa <1$. So, for $n \ge n_1$ (with $n_1$ large enough) we simply deduce $(1-\kappa )\frac{\nu _{n+1}}{\nu _n}<1$. It follows that (2.11a) is verified (for some integer $n_1$). Hence for $n \ge n_1$, by the previous arguments we deduce that (A.8) holds. Thus, taking $s=s_0$ in (A.8) and noticing that $\vartheta _{n}$ reduces to $\gamma _n$, we get

$$\begin{aligned}&\dot{\mathcal E}_{n+1}(s_0,q) + \frac{1}{2} \rho _n^{-1} (e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \nonumber \\&\quad + s_0 \left( s_0 \rho _n -\dot{\nu }_{n+1} \right) \langle t G (z_{n-1}),x_{n} -q \rangle + \gamma _n \rho _n^{-1} (e_*+\nu _{n+1})^2 Z_n = - { T}_{n}(s_0). \end{aligned}$$

(A.9)

We proceed with our reasoning by estimating the right side of the previous equality. Clearly, by definition of $T_n(s_0)$ we have

$$\begin{aligned} -T_n(s_0)&\le \frac{1}{2} |1-\kappa | \rho _n \nu _{n} \nu _{n+1} \left\| y_{n} -x_{n} + \left( \frac{1}{\theta _n} - \frac{ e_*-s_0 }{ \nu _n \rho _n } \right) \dot{x}_{n+1} \right\| ^2 \nonumber \\&\quad - \frac{1}{2}\left( e_*-s_0 \right) \bigg ( (e_*+2 \nu _{n+1}) \rho _n^{-1} + s_0 (\rho _n^{-1}-1) \bigg ) \Vert \dot{x}_{n+1} \Vert ^2. \end{aligned}$$

(A.10)

Moreover, given $\varepsilon >0$, thanks to Young’s inequality we classically obtain

$$\begin{aligned} \left\| y_{n} -x_{n} + \left( \frac{1}{\theta _n} - \frac{ e_*-s_0 }{ \nu _n \rho _n } \right) \dot{x}_{n+1} \right\| ^2&= \left\| \frac{1}{\theta _n} \big ( \theta _n (y_{n} -x_{n} ) +\dot{x}_{n+1} \big ) - \frac{ e_*-s_0 }{ \nu _n \rho _n } \dot{x}_{n+1} \right\| ^2 \\&\le \left( 1+2 \varepsilon \right) \left\| \frac{1}{\theta _n} \big ( \theta _n (y_{n} -x_{n} ) +\dot{x}_{n+1} \big ) \right\| ^2\\&\quad + \left( 1+\frac{1}{2\varepsilon }\right) \left\| \frac{ e_*-s_0 }{ \nu _n \rho _n } \dot{x}_{n+1} \right\| ^2 \\&= \left( 1+ 2 \varepsilon \right) \frac{1}{\theta _n^2} W_n\\&\quad + \left( 1+\frac{1}{2\varepsilon }\right) \left( \frac{ e_*-s_0 }{ \nu _n \rho _n } \right) ^2 \left\| \dot{x}_{n+1} \right\| ^2, \end{aligned}$$

hence, noticing that $\theta _n = \frac{\nu _n \rho _n}{e_*+\nu _{n+1} }$, we obtain

$$\begin{aligned}&\frac{1}{2}\rho _n \nu _{n} \nu _{n+1} \left\| y_{n} -x_{n} + \left( \frac{1}{\theta _n} - \frac{ e_*-s_0 }{ \nu _n \rho _n } \right) \dot{x}_{n+1} \right\| ^2 \\&\quad \le \left( \frac{1}{2}+ \varepsilon \right) \rho _n \nu _{n} \nu _{n+1} \frac{1}{\theta _n^2} \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2\\&\qquad + \rho _n \nu _{n} \nu _{n+1} \left( \frac{1}{2} +\frac{1}{4\varepsilon }\right) \left( \frac{ e_*-s_0 }{ \nu _n \rho _n } \right) ^2 \left\| \dot{x}_{n+1} \right\| ^2 \\&\quad = \left( \frac{1}{2}+\varepsilon \right) \rho _n^{-1}\frac{\nu _{n+1}}{\nu _n} ( e_*+\nu _{n+1} )^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2\\&\qquad + \left( \frac{1}{2}+\frac{1}{4 \varepsilon }\right) (e_*-s_0)^2 \rho _n^{-1}\frac{\nu _{n+1}}{\nu _n} \Vert \dot{x}_{n+1} \Vert ^2. \end{aligned}$$

So, combining this last result with (A.10) yields

$$\begin{aligned} -T_n(s_0)&\le \left( \frac{1}{2}+\varepsilon \right) |1-\kappa | \rho _n^{-1}\frac{\nu _{n+1}}{\nu _n} (e_*+\nu _{n+1} )^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \\&\quad + \left( \frac{1}{2}+\frac{1}{4 \varepsilon }\right) |1-\kappa | (e_*-s_0)^2 \rho _n^{-1}\frac{\nu _{n+1}}{\nu _n} \Vert \dot{x}_{n+1} \Vert ^2 \\&\quad - \frac{1}{2}\left( e_*-s_0 \right) \rho _n^{-1} \big ( e_*+2 \nu _{n+1} + s_0 (1-\rho _n) \big ) \Vert \dot{x}_{n+1} \Vert ^2. \end{aligned}$$

It is also readily seen that $1-\rho _n=(1-\kappa )\nu _{n+1}\nu _n^{-1}$, so that the previous inequality can be rewritten as

$$\begin{aligned}&-T_n(s_0) - \left( \frac{1}{2}+\varepsilon \right) |1-\kappa | \rho _n^{-1}\frac{\nu _{n+1}}{\nu _n} ( e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \\&\quad \le - \frac{1}{2}(e_*-s_0)\rho _n^{-1}\\&\qquad \left( e_*+2 \nu _{n+1} + s_0 (1-\kappa )\frac{\nu _{n+1}}{\nu _n} -2(e_*-s_0)\left( \frac{1}{2}+\frac{1}{4 \varepsilon }\right) |1-\kappa | \frac{\nu _{n+1}}{\nu _n} \right) \Vert \dot{x}_{n+1} \Vert ^2 \\&\quad = - \frac{1}{2}(e_*-s_0)\rho _n^{-1}\\&\qquad \left( e_*+2 \nu _{n+1} + \left( s_0 (1-\kappa ) -(e_*-s_0)\left( 1+\frac{1}{2 \varepsilon }\right) |1-\kappa | \right) \frac{\nu _{n+1}}{\nu _n} \right) \Vert \dot{x}_{n+1} \Vert ^2, \end{aligned}$$

or equivalently

$$\begin{aligned} -T_n(s_0)&\le \frac{1}{2} C_1(\varepsilon , \kappa ) \rho _n^{-1}\frac{\nu _{n+1}}{\nu _n} ( e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \nonumber \\&\quad - \frac{1}{2}(e_*-s_0)\rho _n^{-1}\left( e_*+2 \nu _{n+1} - C_2(\varepsilon , \kappa ) \frac{\nu _{n+1}}{\nu _n} \right) \Vert \dot{x}_{n+1} \Vert ^2, \end{aligned}$$

(A.11)

where the quantities $C_1(\varepsilon , \kappa )$ and $C_2(\varepsilon , \kappa )$ are defined by

$$\begin{aligned}{} & {} {C_1(\varepsilon , \kappa )= \left( 1+2 \varepsilon \right) |1-\kappa |}, \end{aligned}$$

(A.12a)

$$\begin{aligned}{} & {} {C_2(\varepsilon , \kappa )= (e_*-s_0)\left( 1+\frac{1}{2 \varepsilon }\right) |1-\kappa | -s_0 (1-\kappa )}. \end{aligned}$$

(A.12b)

It is clear that for any given $\kappa \in (0,2)$ we have $|1-\kappa |<1$. Note that $\kappa =1$ yields $C_1(\varepsilon ,1)=C_2(\varepsilon ,1)=0$, while, for $\kappa \ne 1$, by considering positive values $\epsilon _0$ and $\theta \in (0,\frac{1}{2})$ such that $\varepsilon _0 = \frac{1}{2} \theta \frac{1-|1-\kappa |}{|1-\kappa | }$, we are led to

$$\begin{aligned}{} & {} C_1(\varepsilon _0, \kappa )= |1-\kappa |+ \theta \left( 1- |1-\kappa | \right) , \end{aligned}$$

(A.13a)

$$\begin{aligned}{} & {} C_2(\varepsilon _0, \kappa )= (e_*-s_0) |1-\kappa |\frac{\theta + (1-\theta )|1-\kappa | }{\theta (1-|1-\kappa | )}-s_0 (1-\kappa ). \end{aligned}$$

(A.13b)

Then, for $\kappa \in (0,2)$ and for $n \ge n_1$, by inequality (A.11) we deduce that

$$\begin{aligned} -T_n(s_0)&\le \frac{1}{2} \bar{C}_1 \rho _n^{-1}\frac{\nu _{n+1}}{\nu _n} ( e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \nonumber \\&\quad - \frac{1}{2}(e_*-s_0)\rho _n^{-1}\left( e_*+2 \nu _{n+1} - \bar{C}_2 \frac{\nu _{n+1}}{\nu _n}\right) \Vert \dot{x}_{n+1} \Vert ^2, \end{aligned}$$

(A.14)

where $\bar{C}_1$ and $\bar{C}_2$ are given by

$$\begin{aligned}{} & {} {\bar{C}_1=C_1(\varepsilon _0, \kappa )\;\textrm{if}\;\kappa \ne 1,\; \textrm{and}\;\bar{C}_1=0\;\textrm{otherwise}}, \end{aligned}$$

(A.15a)

$$\begin{aligned}{} & {} {\bar{C}_2= C_2(\varepsilon _0, \kappa )\;\textrm{if}\;\kappa \ne 1,\;\textrm{and}\;\bar{C}_2=0\;\textrm{otherwise}}. \end{aligned}$$

(A.15b)

Consequently, for $n\ge n_1$, by (A.9) in light of (A.14) we infer that

$$\begin{aligned}&\dot{\mathcal E}_{n+1}(s_0,q) + \frac{1}{2}\left( 1- \bar{C}_1 \frac{\nu _{n+1}}{\nu _n} \right) \rho _n^{-1} (e_*+\nu _{n+1})^2 \Vert \dot{x}_{n+1} + \theta _n (y_{n} -x_{n} )\Vert ^2 \nonumber \\&\qquad + s_0 \left( s_0 \rho _n -\dot{\nu }_{n+1} \right) \langle t G z_{n-1},x_{n} -q \rangle +\gamma _n \rho _n^{-1} (e_*+\nu _{n+1})^2 Z_n \nonumber \\&\qquad +\frac{1}{2}(e_*-s_0)\rho _n^{-1}\left( e_*+2 \nu _{n+1} - \bar{C}_2 \frac{\nu _{n+1}}{\nu _n} \right) \Vert \dot{x}_{n+1} \Vert ^2\le 0. \end{aligned}$$

(A.16)

It remains to estimate the terms $\big ( 1- \bar{C}_1 \frac{\nu _{n+1}}{\nu _n} \big )$ and $\big ( e_*+2 \nu _{n+1} - \bar{C}_2 \frac{\nu _{n+1}}{\nu _n} \big )$ arising in the previous inequality. Observe that, given $\kappa \in (0,2)$, by the condition $\nu _{n+1} \sim \nu _n$ (as $n \rightarrow \infty$) we obviously have

$$\begin{aligned} \lim _{n \rightarrow \infty } \frac{\nu _{n+1}}{\nu _n} < 1+ \frac{\theta (1- |1-\kappa |)}{ \theta + (1-\theta ) |1-\kappa |} \quad \left( \text {for any given}\;\theta \in \left( 0,\frac{1}{2}\right) \right) , \end{aligned}$$

hence, for $n \ge n_1$ (with $n_1$ large enough) we get

$$\begin{aligned} {\frac{\nu _{n+1}}{\nu _n} < 1+ \frac{\theta (1- |1-\kappa |)}{ \theta + (1-\theta ) |1-\kappa |} }. \end{aligned}$$

(A.17)

It can be also checked without any difficulties that we have the following equivalencies

$$\begin{aligned} \frac{\nu _{n+1}}{\nu _n}&< 1+ \frac{\theta (1- |1-\kappa |)}{ \theta + (1-\theta ) |1-\kappa |} \nonumber \\&\Leftrightarrow 2 \theta + (1-2 \theta )|1-\kappa |> \big ( \theta + (1-\theta ) |1-\kappa | \big ) \frac{\nu _{n+1}}{\nu _n}= C_1(\varepsilon _0, \kappa ) \frac{\nu _{n+1}}{\nu _n} \nonumber \\&\Leftrightarrow 1- C_1(\varepsilon _0, \kappa ) \frac{\nu _{n+1}}{\nu _n} > 1- 2\theta - (1-2 \theta )|1-\kappa |= (1-2 \theta )\left( 1- |1-\kappa | \right) . \end{aligned}$$

(A.18)

Then, for $n \ge n_1$, by (A.17) and (A.18) we get

$$\begin{aligned} {1- C_1(\varepsilon _0, \kappa ) \frac{\nu _{n+1}}{\nu _n} > (1-2 \theta )\left( 1- |1-\kappa | \right) }. \end{aligned}$$

(A.19)

Moreover, for $n \ge n_1$, it is not difficult to see that (2.11b) can be reformulated as

$$\begin{aligned} { \nu _{n+1} - C_2(\varepsilon _0, \kappa ) \frac{\nu _{n+1}}{\nu _n} \ge 0}. \end{aligned}$$

(A.20)

As a result, for $n \ge n_1$, and letting c be introduced in (2.15), by (A.15a) in light of (A.19) and (A.20) we conclude that

$$\begin{aligned} {1- \bar{C}_1 \frac{\nu _{n+1}}{\nu _n} \ge c} \quad \textrm{and} \quad {e_*+2 \nu _{n+1} - \bar{C}_2 \frac{\nu _{n+1}}{\nu _n} \ge e_*+ \nu _{n+1}}, \end{aligned}$$

(A.21)

which, by (A.16), amounts to (2.14). $\square$

1.2 Appendix A.2: Proof of Proposition 3.1

For any $x,y \in \mathcal {H}$, by definition of $\langle \cdot ,\cdot \rangle _M$ along with ${\mathcal B}:= M^{-1}B$ we readily have

$$\begin{aligned} \langle {\mathcal B} (x)- {\mathcal B} (y), x-y \rangle _M = \langle M \big ( {\mathcal B} (x)- {\mathcal B} (y)\big ), x-y \rangle =\langle B (x)- B(y), x-y \rangle , \end{aligned}$$

hence condition (1.2b) equivalently writes

$$\begin{aligned} \langle {\mathcal B} (x)- {\mathcal B} (y), x-y \rangle _M \ge \Vert B (x) - B(y) \Vert _{L^{-1}}^2. \end{aligned}$$

(A.22)

Thanks to the same previously used arguments we additionally have

$$\begin{aligned} \Vert {\mathcal B} (x) - {\mathcal B}(y) \Vert _M^2&= \langle M \big ( {\mathcal B} (x) - {\mathcal B}(y) \big ), {\mathcal B} (x) - {\mathcal B}(y) \rangle \\&= \langle M^{-1}\big ( B(x) - B(y)\big ), B (x) -B(y) \rangle , \end{aligned}$$

which can be rewritten as

$$\begin{aligned} \Vert {\mathcal B} (x) - {\mathcal B}(y) \Vert _M^2= \langle L^{\frac{1}{2}} M^{-1}L^{\frac{1}{2}}L^{-\frac{1}{2}}\big ( B(x) - B(y)\big ), L^{-\frac{1}{2}} \big ( B (x) -B(y) \big ) \rangle , \end{aligned}$$

(A.23)

where $L^{\frac{1}{2}}$ and $L^{-\frac{1}{2}}$ are the square roots of the positive definite self-adjoint maps L and $L^{-1}$, respectively. It is immediately deduced from (A.23) that

$$\begin{aligned} \Vert {\mathcal B} (x) - {\mathcal B}(y) \Vert _M^2 \le \Vert L^{\frac{1}{2}}\Vert \,\Vert M^{-1} \Vert \,\Vert L^{\frac{1}{2}}\Vert \,\Vert L^{-\frac{1}{2}}\big ( B(x) - B(y)\big )\Vert ^2, \end{aligned}$$

(A.24)

hence recalling that $\Vert L^{\frac{1}{2}}\Vert ^2 = \Vert L\Vert$ (see Remark 3.2) leads us to

$$\begin{aligned} \Vert {\mathcal B} (x) - {\mathcal B}(y) \Vert _M^2 \le \Vert L\Vert \,\Vert M^{-1} \Vert \,\Vert B (x) - B(y) \Vert _{L^{-1}}^2 . \end{aligned}$$

(A.25)

So, combining (A.22) and (A.25) amounts to

$$\begin{aligned} \langle {\mathcal B} (x)- {\mathcal B} (y), x-y \rangle _M \ge {\bar{\beta }}_M \Vert {\mathcal B} (x) - {\mathcal B} (y) \Vert _{M}^2, \end{aligned}$$

(A.26)

where ${\bar{\beta }}_M=\frac{1}{\Vert L\Vert \times \Vert M^{-1}\Vert }$. This proves that ${\mathcal B}$ is ${\bar{\beta }}_M$-cocoercive in $(\mathcal {H}, \langle \cdot ,\cdot \rangle _M)$. $\square$

1.3 Appendix A.3: Proof of Proposition 3.5

For simplification reasons, we write G instead of $G^M_{\mu }$ and, given any mapping $\Gamma :\mathcal {H}\rightarrow \mathcal {H}$ and any $\{x_1,x_2\} \subset \mathcal {H}$, we denote $\Delta \Gamma (x_1,x_2)=\Gamma (x_1)-\Gamma (x_2)$. Let $\bar{A}=M^{-1}A$, $\bar{B}=M^{-1}B$ and $C=I-\mu \bar{B}$. Clearly, $\bar{A}$ is monotone in $(\mathcal {H},\Vert .\Vert _M)$. It is also seen for $x \in \mathcal {H}$ that $G(x) =\mu ^{-1} \big ( x -J_{\mu \bar{A}}\big (C(x) \big )\big )$, or equivalently

$$\begin{aligned} {G(x) = \bar{B}(x)+ \bar{A}_{\mu }(C(x))}, \end{aligned}$$

(A.27)

where $\bar{A}_{\mu }:=\mu ^{-1} \left( I -J_{\mu \bar{A}}\right)$ is the Yosida regularization of $\bar{A}$. Now, given $(x_1, x_2) \in \mathcal {H}^2$, we get

$$\begin{aligned}&\langle \Delta G(x_1,x_2) , x_1-x_2 \rangle _M = \langle \bar{B}(x_1)+ \bar{A}_{\mu }(C(x_1))-\bar{B}(x_2)- \bar{A}_{\mu }(C(x_2)) , x_1-x_2 \rangle _M \\&\quad = \langle \Delta \bar{B}(x_1, x_2) , x_1-x_2 \rangle _M + \langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2) , x_1-x_2 \rangle _M \\&\quad = \langle \Delta \bar{B}(x_1, x_2) , x_1-x_2 \rangle _M + \langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2) , \Delta C(x_1, x_2)\rangle _M \\&\qquad + \langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2) , \Delta (I-C) (x_1, x_2) \rangle _M, \end{aligned}$$

hence, by $I-C=\mu \bar{B}$, we equivalently obtain

$$\begin{aligned} \langle \Delta G(x_1,x_2), x_1-x_2 \rangle _M&\ge \langle \Delta \bar{B}(x_1,x_2) , x_1-x_2 \rangle _M\nonumber \\&\quad + \langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2) , \Delta C(x_1, x_2) \rangle _M \nonumber \\&\quad + \mu \langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2) , \Delta \bar{B}(x_1,x_2) \rangle _M. \end{aligned}$$

(A.28)

Let us estimate separately the last two terms in the right side of the previous inequality. As a classical result, by the $\mu$-cocoercivity of $\bar{A}_{\mu }$ in $(\mathcal {H},\Vert \cdot \Vert _M)$, we have

$$\begin{aligned} \langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2), \Delta {C}(x_1,x_2) \rangle _M \ge \mu \Vert \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2) \Vert _M^2, \end{aligned}$$

which by $\bar{A}_{\mu }\circ C =G- \bar{B}$ (from (A.27)) can be rewritten as

$$\begin{aligned}&\langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2), \Delta {C}(x_1,x_2)\rangle _M \\&\quad \ge \mu \Vert \Delta G(x_1,x_2)- \Delta \bar{B}(x_1,x_2) \Vert _M^2 \\&\quad =\mu \Vert \Delta G(x_1,x_2) |_M^2 + \mu \Vert \Delta \bar{B}(x_1,x_2) \Vert _M^2 - 2 \mu \langle \Delta G(x_1,x_2),\Delta \bar{B}(x_1,x_2) \rangle _M. \end{aligned}$$

Moreover, by $\bar{A}_{\mu }\circ C =G- \bar{B}$ (from (A.27)), we simply get

$$\begin{aligned}&\mu \langle \Delta (\bar{A}_{\mu } \circ C)(x_1, x_2) , \Delta \bar{B}(x_1,x_2) \rangle _M \\&\quad = \mu \langle \Delta G(x_1,x_2) - ( \Delta \bar{B}(x_1,x_2) ) , \Delta \bar{B}(x_1,x_2) \rangle _M \\&\quad = \mu \langle \Delta {G}(x_1,x_2) ) , \Delta \bar{B}(x_1,x_2) \rangle _M - \mu \Vert \Delta \bar{B}(x_1,x_2) \Vert ^2 _M. \end{aligned}$$

Thus, by (A.28) and the previous arguments, we obtain

$$\begin{aligned}&\langle \Delta {G}(x_1,x_2), x_1-x_2 \rangle _M \\&\quad \ge \langle \Delta \bar{B}(x_1,x_2) , x_1-x_2 \rangle _M + \mu \Vert \Delta {G}(x_1,x_2) \Vert _M^2 + \mu \Vert \Delta \bar{B}(x_1,x_2) \Vert _M^2 \\&\qquad - 2 \mu \langle \Delta {G}(x_1,x_2), \Delta \bar{B}(x_1,x_2) \rangle _M + \mu \langle \Delta {G}(x_1,x_2), \Delta \bar{B}(x_1,x_2) \rangle _M \\&\qquad - \mu \Vert \Delta \bar{B}(x_1,x_2)\Vert ^2 _M \\&\quad = \langle \Delta \bar{B}(x_1,x_2) , x_1-x_2 \rangle _M + \mu \Vert \Delta {G}(x_1,x_2) \Vert _M^2 - \mu \langle \Delta {G}(x_1,x_2), \Delta \bar{B}(x_1,x_2) \rangle _M. \end{aligned}$$

Hence, reminding that $\bar{B}=M^{-1}B$, we equivalently obtain

$$\begin{aligned} \langle \Delta {G}(x_1,x_2), x_1-x_2 \rangle _M&\ge \langle \Delta {B}(x_1,x_2) , x_1-x_2 \rangle + \mu \Vert \Delta {G}(x_1,x_2) \Vert _M^2 \\&\quad - \mu \langle \Delta {G}(x_1,x_2), \Delta {B}(x_1,x_2) \rangle . \end{aligned}$$

Then by the cocoercivity assumption on B we infer that

$$\begin{aligned} \langle \Delta {G}(x_1,x_2), x_1-x_2 \rangle _M&\ge \Vert \Delta {B}(x_1,x_2) \Vert ^2_{L^{-1}} + \mu \Vert \Delta {G}(x_1,x_2) \Vert _M^2\nonumber \\&\quad - \mu \langle \Delta {G}(x_1,x_2) , \Delta {B}(x_1,x_2) \rangle . \end{aligned}$$

(A.29)

Now, let us prove (3.13) for $i=1$. From an easy computation, we obtain

$$\begin{aligned} \Vert \Delta {B}(x_1,x_2) \Vert ^2 \le \Vert L ^{\frac{1}{2}} \Vert ^{2} \Vert \Delta {B}(x_1,x_2) \Vert ^2_{L^{-1}}. \end{aligned}$$

hence, for $\delta >0$, using successively Peter–Paul’s inequality and the previous one gives us

$$\begin{aligned} \langle \Delta {G}(x_1,x_2) , \Delta {B}(x_1,x_2) \rangle&\le { \delta \Vert \Delta {G}(x_1,x_2) \Vert ^2}+ {\frac{1}{4 \delta } \Vert \Delta {B}(x_1,x_2) \Vert ^2} , \nonumber \\&\le { \delta \Vert \Delta {G}(x_1,x_2)\Vert ^2} + {\frac{1}{4 \delta } \Vert L ^{\frac{1}{2}}\Vert ^{2} \Vert \Delta {B}(x_1,x_2)\Vert ^2_{L^{-1}} }. \end{aligned}$$

(A.30)

Therefore, combining this last inequality with (A.29) entails

$$\begin{aligned}&\langle \Delta {G}(x_1,x_2) , x_1-x_2 \rangle _M \\&\quad \ge \left( 1 - \frac{\mu }{4 \delta } \Vert L ^{\frac{1}{2}}\Vert ^{2} \right) \Vert \Delta {B}(x_1,x_2)|^2_{L^{-1}} + \mu \left( \Vert \Delta {G}(x_1,x_2) \Vert _M^2 - \delta \Vert \Delta {G}(x_1,x_2) \Vert ^2\right) \\&\quad = \left( 1 - \frac{\mu }{4 \delta } \Vert L ^{\frac{1}{2}}\Vert ^{2} \right) \Vert \Delta {B}(x_1,x_2)\Vert ^2_{L^{-1}} + \mu \langle \left( M -\delta I \right) \Delta {G}(x_1,x_2) , \Delta {G}(x_1,x_2) \rangle , \\ \end{aligned}$$

that is (3.13) with $i=1$.

Let us prove (3.13) for $i=2$. Using again Peter–Paul’s inequality, we readily have

$$\begin{aligned} \mu \langle \Delta {G}(x_1,x_2) , \Delta {B}(x_1,x_2) \rangle&=\langle \mu \Delta {G}(x_1,x_2) , L ^{\frac{1}{2}} L ^{-\frac{1}{2}}\Delta {B}(x_1,x_2) \rangle \\&=\langle \mu L ^{\frac{1}{2}}\Delta {G}(x_1,x_2) , L ^{-\frac{1}{2}}\Delta {B}(x_1,x_2) \rangle \\&\le \mu ^2 \Vert L ^{\frac{1}{2}}\Delta {G}(x_1,x_2) \Vert ^2+ \frac{1}{4 } \Vert L ^{-\frac{1}{2}} \Delta {B}(x_1,x_2) \Vert ^2 \\&= \mu ^2\langle L \Delta {G}(x_1,x_2), \Delta {G}(x_1,x_2) \rangle \\&\quad + \frac{1}{4} \Vert \Delta {B}(x_1,x_2) \Vert ^2_{L ^{-1} }, \end{aligned}$$

which, in light (A.29), gives us

$$\begin{aligned} \langle \Delta {G}(x_1,x_2) , x_1-x_2 \rangle _M&\ge \frac{3}{4 } \Vert \Delta {B}(x_1,x_2)\Vert ^2_{L^{-1}} \\&\quad + \mu \langle \left( M -\mu \delta L \right) \Delta {G}(x_1,x_2), \Delta {G}(x_1,x_2) \rangle , \end{aligned}$$

that is (3.13) with $i=2$. $\square$

1.4 Appendix A.4: A useful inequality

Let us recall the following basic but useful result.

Proposition A.3

For $\epsilon \in (0,1)$, $\{h, a_1,a_2, \sigma _1, \sigma _1 \}\subset [0,\infty )$ such that $h \le \epsilon \sqrt{\sigma _1\sigma _2}$, we have the following inequality

$$\begin{aligned} \sigma _1 a_1^2 + \sigma _2 a_2^2 - 2 h a_1 a_2 \ge (1-\epsilon ) \big ( \sigma _1 a_1^2 + \sigma _2 a_2^2 \big ). \end{aligned}$$

(A.31)

Proof

Clearly, we have

$$\begin{aligned} \sigma _1 a_1^2 + \sigma _2 a_2^2 - 2 h a_1 a_2&= (1-\epsilon ) \big ( \sigma _1 a_1^2 + \sigma _2 a_2^2 \big ) + \epsilon \big ( \sigma _1 a_1^2 + \sigma _2 a_2^2 \big )- 2 h a_1 a_2 \\&= (1-\epsilon ) \big ( \sigma _1 a_1^2 + \sigma _2 a_2^2 \big ) + \epsilon \big ( \sigma _1 a_1^2 + \sigma _2 a_2^2 - 2 \sqrt{\sigma _1\sigma _2 } a_1 a_2\big )\\&\quad + 2 \epsilon \sqrt{\sigma _1\sigma _2 } a_1 a_2- 2 h a_1 a_2 \\&= (1-\epsilon ) \big ( \sigma _1 a_1^2 + \sigma _2 a_2^2 \big ) + \epsilon \big ( \sqrt{\sigma _1 } a_1 - \sqrt{\sigma _2 } a_2 \big )^2\\&\quad + 2 \big ( \epsilon \sqrt{\sigma _1\sigma _2 } - h \big )a_1 a_2, \end{aligned}$$

which completes the proof. $\square$