A new hybrid regularization scheme for removing salt and pepper noise

Abstract

We propose a new model for removing the Salt and Pepper Noise (SPN) by combining the high-order total variation overlapping group sparsity with the nuclear norm regularization. Since the proposed model is convex, non-smooth, and separable, the alternating direction method of multipliers (ADMM) can be employed to solve it and the convergence can be kept. Numerical comparisons with some related state-of-the-art models show that the proposed model can significantly improve the restored quality in terms of the signal to noise ratio (SNR) and the structural similarity index measure (SSIM).

Appendices

Appendix A

Proof

Based on the property of the nuclear norm $\Vert \cdot \Vert _*$ and $\phi (\cdot )$ as the extension of the Frobenius norm, we first can deduce that the objective function in the model (2) is strictly convex. For $u\in \ell ^1(\mathbf{X} )$, the objective function is also coercive whenever u goes to infinity in the fitting term $\Vert u-f\Vert _1$. Following the standard numerical optimization theory Beck (2014), we can deduce that the model (2) exists a global solution. $\square$

Appendix B

Proof

Since $(h^{*},{\mathbf {q}}^{*},r^{*},u^{*},\varrho _{1}^{*},\varvec{\varrho }_{2}^{*},\varrho _{3}^{*})$  is the saddle point of the original problem, we can deduce $h^{*}=u^{*}-f$, ${\mathbf {q}}^{*}=\nabla ^{2} u^{*}$, $r^{*}=u^{*}$ and then (10)–(12) can be replaced by

$$\begin{aligned} {\left\{ \begin{array}{ll} \varrho _1^{*}=\varrho _1^* + \beta _1(h^{*}-u^{*}+f),\\ \varvec{\varrho }_2^{*}=\varvec{\varrho }_2^* + \beta _2({\mathbf {q}}^{*}-\nabla ^{2} {u^{*}}),\\ \varrho _3^{*}=\varrho _3^* + \beta _3(r^{*}-u^{*}). \end{array}\right. } \end{aligned}$$

(18)

If setting the relative errors by  $h_{e}^{k}=h^{k}-h^{*}$, ${\mathbf {q}}_{e}^{k}={\mathbf {q}}^{k}-{\mathbf {q}}^{*}$, $r_{e}^{k}=r^{k}-r^{*}$, $u_{e}^{k}=u^{k}-u^{*}$, $\varrho _{1e}^{k}=\varrho _{1}^{k}-\varrho _{1}^{*}$, $\varvec{\varrho }_{2e}^{k}=\varvec{\varrho }_{2}^{k}-\varvec{\varrho }_{2}^{*}$, $\varrho _{3e}^{k}=\varrho _{3}^{k}-\varrho _{3}^{*}$, and subtracting (18) from (10)–(12), we can obtain that

$$\begin{aligned} \left\{ \begin{array}{llll} \varrho _{1e}^{k+1}=\varrho _{1e}^k + \beta _1(h_{e}^{k+1}-u_{e}^{k+1}),\\ \varvec{\varrho }_{2e}^{k+1}=\varvec{\varrho }_{2e}^k + \beta _2({\mathbf {q}}_{e}^{k+1}-\nabla ^{2} {u_{e}^{k+1}}),\\ \varrho _{3e}^{k+1}=\varrho _{3e}^k + \beta _3(r_{e}^{k+1}-u_{e}^{k+1}). \end{array} \right. \end{aligned}$$

(19)

Squared both sides of (19), we can obtain that

$$\begin{aligned} {\left\{ \begin{array}{ll} \Vert \varrho _{1e}^{k+1}\Vert _{2}^{2}&{}=\Vert \varrho _{1e}^k\Vert _{2}^{2}+\beta _1^{2}\Vert h_{e}^{k+1}-u_{e}^{k+1}\Vert _{2}^{2} +2\beta _{1}\langle \varrho _{1e}^k,h_{e}^{k+1}-u_{e}^{k+1}\rangle ,\\ \Vert \varvec{\varrho }_{2e}^{k+1}\Vert _{\mathrm {F}}^{2}&{}=\Vert \varvec{\varrho }_{2e}^k\Vert _{\mathrm {F}}^{2} +\beta _2^{2}\Vert {\mathbf {q}}_{e}^{k+1}-\nabla ^{2} u_{e}^{k+1}\Vert _{\mathrm {F}}^{2} +2\beta _{2}\langle \varvec{\varrho }_{2e}^k,{\mathbf {q}}_{e}^{k+1}-\nabla ^{2} u_{e}^{k+1}\rangle ,\\ \Vert \varrho _{3e}^{k+1}\Vert _{2}^{2}&{}=\Vert \varrho _{3e}^k\Vert _{2}^{2}+\beta _3^{2}\Vert r_{e}^{k+1}-u_{e}^{k+1}\Vert _{2}^{2} +2\beta _{3} \langle \varrho _{3e}^k,r_{e}^{k+1}-u_{e}^{k+1}\rangle . \end{array}\right. } \end{aligned}$$

The above facts can be equivalently expressed as follows.

$$\begin{aligned} {\left\{ \begin{array}{ll} \frac{1}{2\beta _{1}}\left( \left\| \varrho _{1e}^k\right\| _{2}^{2} -\left\| \varrho _{1e}^{k+1}\right\| _{2}^{2}\right) =\left\langle u_{e}^{k+1}-h_{e}^{k+1},\varrho _{1e}^k\right\rangle -\frac{\beta _{1}}{2}\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2},\\ \frac{1}{2\beta _{2}}\left( \left\| \varvec{\varrho }_{2e}^k\right\| _{\mathrm {F}}^{2} -\left\| \varvec{\varrho }_{2e}^{k+1}\right\| _{\mathrm {F}}^{2}\right) =\left\langle \nabla ^{2} u_{e}^{k+1}-{\mathbf {q}}_{e}^{k+1}, \varvec{\varrho }_{2e}^k\right\rangle -\frac{\beta _2}{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2} u_{e}^{k+1}\right\| _{\mathrm {F}}^{2},\\ \frac{1}{2\beta _{3}}\left( \left\| \varrho _{3e}^k\right\| _{2}^{2}-\left\| \varrho _{3e}^{k+1}\right\| _{2}^{2}\right) =\left\langle u_{e}^{k+1}-r_{e}^{k+1},\varrho _{3e}^k\right\rangle -\frac{\beta _3}{2}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}. \end{array}\right. }\nonumber \\ \end{aligned}$$

(20)

Based on Lemma 4, for the saddle point $\left( h^{*},{\mathbf {q}}^{*},r^{*},u^{*},\varrho _{1}^{*},\varvec{\varrho }_{2}^{*},\varrho _{3}^{*}\right)$, we can get

$$\begin{aligned} {\left\{ \begin{array}{ll} \Vert h\Vert _{1}-\left\| h^{*}\right\| _{1}+\left\langle \varrho _{1}^{*},h-h^{*} \right\rangle +\beta _{1}\left\langle h^{*}-u^{*}+f,h-h^{*}\right\rangle \ge 0,\\ \lambda _{1}\left( \varphi \left( {\mathbf {q}}\right) -\varphi \left( {\mathbf {q}}^{*}\right) \right) +\left\langle \varvec{\varrho }_{2}^{*},{\mathbf {q}}-{\mathbf {q}}^{*} \right\rangle +\beta _{2}\left\langle {\mathbf {q}}^{*}-\nabla ^{2}u^{*},{\mathbf {q}}-{\mathbf {q}}^{*}\right\rangle \ge 0,\\ \lambda _{2}\left( \Vert r\Vert _{*}-\left\| r^{*}\right\| _{*}\right) +\left\langle \varrho _{3}^{*},r-r^{*} \right\rangle +\beta _{3}\left\langle r^{*}-u^{*},r-r^{*}\right\rangle \ge 0,\\ \left\langle \varrho _{1}^{*}+\mathrm {div}^{2}\varvec{\varrho }_{2}^{*}+\varrho _{3}^{*}, u^{*}-u\right\rangle +\beta _{1}\left\langle h^{*}-u^{*}+f,u^{*}-u \right\rangle \\ +\beta _{2}\mathrm {div}^{2}\left\langle {\mathbf {q}}^{*}-\nabla ^{2}u^{*},u^{*}-u\right\rangle +\beta _{3}\left\langle r^{*}-u^{*},u^{*}-u\right\rangle \ge 0. \end{array}\right. }\nonumber \\ \end{aligned}$$

(21)

To the subproblem (6), using Lemma 4 again, we can get

$$\begin{aligned}&\left\langle \varrho _{1}^{k}+\mathrm {div}^{2}\varvec{\varrho }_{2}^{k} +\varrho _{3}^{k},u^{k+1}-u\rangle +\beta _{1}\langle h^{k}-u^{k+1}+f,u^{k+1}-u\right\rangle \nonumber \\&\quad +\beta _{2}\mathrm {div}^{2}\left\langle {\mathbf {q}}^{k}-\nabla ^{2}u^{k+1},u^{k+1}-u\rangle +\beta _{3}\langle r^{k}-u^{k+1},u^{k+1}-u\right\rangle \ge 0. \end{aligned}$$

(22)

By first setting u of the forth inequation in (21) to be $u^{k+1}$ and u in (22) to be $u^*$ and then adding them, we can deduce that

$$\begin{aligned}&\left\langle \varrho _{1e}^{k},u_{e}^{k+1}\right\rangle +\left\langle \varvec{\varrho }_{2e}^{k},\nabla ^{2}u_{e}^{k+1}\right\rangle +\left\langle \varrho _{3e}^{k},u_{e}^{k+1}\right\rangle +\beta _{1}\left\langle h_{e}^{k}-u_{e}^{k+1},u_{e}^{k+1}\right\rangle \nonumber \\&\quad +\beta _{2}\left\langle {\mathbf {q}}_{e}^{k}-\nabla ^{2} u_{e}^{k+1},\nabla ^{2} u_{e}^{k+1}\right\rangle +\beta _{3}\left\langle r_{e}^{k}-u_{e}^{k+1},u_{e}^{k+1}\right\rangle \ge 0. \end{aligned}$$

(23)

Similarly, we have

Summing (23) with (24)–(26) together, we have

$$\begin{aligned}&\left\langle \varrho _{1e}^{k},u_{e}^{k+1}-h_{e}^{k+1}\right\rangle +\left\langle \varvec{\varrho }_{2e}^{k},\nabla ^{2} u_{e}^{k+1}-{\mathbf {q}}_{e}^{k+1}\right\rangle +\left\langle \varrho _{3e}^{k},u_{e}^{k+1}-r_{e}^{k+1}\right\rangle \nonumber \\&\quad +\beta _{1}\left( \left\langle h_{e}^{k}-u_{e}^{k+1},u_{e}^{k+1}\right\rangle +\left\langle u_{e}^{k+1}-h_{e}^{k+1},h_{e}^{k+1}\right\rangle \right) \nonumber \\&\quad +\beta _{2}\left( \left\langle {\mathbf {q}}_{e}^{k}-\nabla ^{2}u_{e}^{k+1}, \nabla ^{2}u_{e}^{k+1}\right\rangle +\left\langle \nabla ^{2}u_{e}^{k+1}-{\mathbf {q}}_{e}^{k+1}, {\mathbf {q}}_{e}^{k+1}\right\rangle \right) \nonumber \\&\quad +\beta _{3}\left( \left\langle r_{e}^{k}-u_{e}^{k+1},u_{e}^{k+1}\right\rangle +\left\langle u_{e}^{k+1}-r_{e}^{k+1},r_{e}^{k+1}\right\rangle \right) \ge 0. \end{aligned}$$

(27)

Reorganizing the inequation (27), we can get

$$\begin{aligned}&\left\langle \varrho _{1e}^{k},u_{e}^{k+1}-h_{e}^{k+1}\right\rangle +\left\langle \varvec{\varrho }_{2e}^{k},\nabla ^{2} u_{e}^{k+1}-{\mathbf {q}}_{e}^{k+1}\right\rangle +\left\langle \varrho _{3e}^{k},u_{e}^{k+1}-r_{e}^{k+1}\right\rangle \nonumber \\&\quad \ge \beta _{1}\left\langle h_{e}^{k+1}-h_{e}^{k},u_{e}^{k+1}\right\rangle +\beta _{1}\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}+\beta _{2}\left\langle {\mathbf {q}}_{e}^{k+1}-{\mathbf {q}}_{e}^{k},\nabla ^{2}u_{e}^{k+1}\right\rangle \nonumber \\&\qquad +\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2} u_{e}^{k+1}\right\| _{\mathrm {F}}^{2}+\beta _{3}\left\langle r_{e}^{k+1}-r_{e}^{k},u_{e}^{k+1}\right\rangle +\beta _{3}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}. \end{aligned}$$

(28)

Combining (20) with (28), we have

$$\begin{aligned}&\frac{1}{2\beta _{1}}\left( \left\| \varrho _{1e}^k\right\| _{2}^{2}-\left\| \varrho _{1e}^{k+1}\right\| _{2}^{2}\right) + \frac{\beta _{1}}{2}\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}+ \frac{1}{2\beta _{2}}\left( \left\| \varvec{\varrho }_{2e}^k\right\| _{\mathrm {F}}^{2} -\left\| \varvec{\varrho }_{2e}^{k+1}\right\| _{\mathrm {F}}^{2}\right) \\&\qquad +\frac{\beta _{2}}{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2} u_{e}^{k+1}\right\| _{\mathrm {F}}^{2} +\frac{1}{2\beta _{3}}\left( \left\| \varrho _{3e}^k\right\| _{2}^{2} -\left\| \varrho _{3e}^{k+1}\right\| _{2}^{2}\right) +\frac{\beta _{3}}{2}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} \\&\quad \ge \beta _{1}\left\langle h_{e}^{k+1}-h_{e}^{k},u_{e}^{k+1}\right\rangle +\beta _{1}\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} +\beta _{2}\left\langle {\mathbf {q}}_{e}^{k+1}-{\mathbf {q}}_{e}^{k},\nabla ^{2}u_{e}^{k+1}\right\rangle \\&\qquad +\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2} u_{e}^{k+1}\right\| _{\mathrm {F}}^{2}+\beta _{3}\left\langle r_{e}^{k+1}-r_{e}^{k},u_{e}^{k+1}\right\rangle +\beta _{3}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}. \end{aligned}$$

With the simple calculation, we obtain

$$\begin{aligned}&\frac{1}{\beta _{1}}\left( \left\| \varrho _{1e}^k\right\| _{2}^{2}-\left\| \varrho _{1e}^{k+1}\right\| _{2}^{2}\right) + \frac{1}{\beta _{2}}\left( \left\| \varvec{\varrho }_{2e}^k\right\| _{\mathrm {F}}^{2} -\left\| \varvec{\varrho }_{2e}^{k+1}\right\| _{\mathrm {F}}^{2}\right) +\frac{1}{\beta _{3}}\left( \left\| \varrho _{3e}^k\right\| _{2}^{2}-\left\| \varrho _{3e}^{k+1}\right\| _{2}^{2}\right) \nonumber \\&\quad \ge 2\beta _{1}\left\langle h_{e}^{k+1}-h_{e}^{k},u_{e}^{k+1}\right\rangle +\beta _{1}\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} +2\beta _{2}\left\langle {\mathbf {q}}_{e}^{k+1}-{\mathbf {q}}_{e}^{k},\nabla ^{2}u_{e}^{k+1}\right\rangle \nonumber \\&\qquad +\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2} u_{e}^{k+1}\right\| _{\mathrm {F}}^{2}+2\beta _{3}\left\langle r_{e}^{k+1}-r_{e}^{k},u_{e}^{k+1}\right\rangle +\beta _{3}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}. \end{aligned}$$

(29)

Similar to the subproblem (7), using Lemma 4 again, we still have

Setting $h:=h^{k}$ in (21) and $h:=h^{k+1}$ in (22) and then adding them, we can deduce that

$$\begin{aligned} \left\langle \varrho _{1e}^{k-1}-\varrho _{1e}^{k},h_{e}^{k+1}-h_{e}^{k} \right\rangle -\beta _{1}\left\| h_{e}^{k+1}-h_{e}^{k}\right\| _{2}^{2} -\beta _{1}\left\langle u_{e}^{k}-u_{e}^{k+1},h_{e}^{k+1}-h_{e}^{k}\right\rangle \ge 0. \end{aligned}$$

(32)

Using the fact $\varrho _{1e}^{k}=\varrho _{1e}^{k-1}+\beta _{1}(h_{e}^{k}-u_{e}^{k})$, the inequation (32) can be rearranged as

$$\begin{aligned} \left\langle u_{e}^{k+1}-h_{e}^{k},h_{e}^{k+1}-h_{e}^{k} \right\rangle \ge \Vert h_{e}^{k+1}-h_{e}^{k}\Vert _{2}^{2}. \end{aligned}$$

(33)

Furthermore, using

$$\begin{aligned} \langle h_{e}^{k+1}-h_{e}^{k},h_{e}^{k} \rangle =\frac{1}{2}\left( \Vert h_{e}^{k+1}\Vert _{2}^{2}-\Vert h_{e}^{k}\Vert _{2}^{2} -\Vert h_{e}^{k+1}-h_{e}^{k}\Vert _{2}^{2}\right) , \end{aligned}$$

the inequation (33) can be written as

$$\begin{aligned} 2\left\langle h_{e}^{k+1}-h_{e}^{k},u_{e}^{k+1}\right\rangle \ge \left\| h_{e}^{k+1}\right\| _{2}^{2}-\left\| h_{e}^{k}\right\| _{2}^{2} +\left\| h_{e}^{k+1}-h_{e}^{k}\right\| _{2}^{2}. \end{aligned}$$

(34)

Similarly, we have

$$\begin{aligned} {\left\{ \begin{array}{ll} 2\left\langle {\mathbf {q}}_{e}^{k+1}-{\mathbf {q}}_{e}^{k},\nabla ^{2}u_{e}^{k+1} \right\rangle \ge \left\| {\mathbf {q}}_{e}^{k+1}\right\| _{\mathrm {F}}^{2}-\left\| {\mathbf {q}}_{e}^{k}\right\| _{\mathrm {F}}^{2} +\left\| {\mathbf {q}}_{e}^{k+1}-{\mathbf {q}}_{e}^{k}\right\| _{\mathrm {F}}^{2},\\ 2\left\langle r_{e}^{k+1}-r_{e}^{k},u_{e}^{k+1} \right\rangle \ge \left\| r_{e}^{k+1}\right\| _{2}^{2}-\left\| r_{e}^{k}\right\| _{2}^{2}+\left\| r_{e}^{k+1}-r_{e}^{k}\right\| _{2}^{2}. \end{array}\right. } \end{aligned}$$

(35)

By putting (34) and (35) into (29), with the simple recombination, we can get

$$\begin{aligned}&\frac{1}{\beta _{1}}\left( \left\| \varrho _{1e}^k\right\| _{2}^{2}-\left\| \varrho _{1e}^{k+1}\right\| _{2}^{2}\right) + \frac{1}{\beta _{2}}\left( \left\| \varvec{\varrho }_{2e}^k\right\| _{\mathrm {F}}^{2} -\left\| \varvec{\varrho }_{2e}^{k+1}\right\| _{\mathrm {F}}^{2}\right) +\frac{1}{\beta _{3}}\left( \left\| \varrho _{3e}^k\right\| _{2}^{2} -\left\| \varrho _{3e}^{k+1}\right\| _{2}^{2}\right) \\&\qquad +\beta _{1}\left( \left\| h_{e}^{k}\right\| _{2}^{2}-\left\| h_{e}^{k+1}\right\| _{2}^{2}\right) +\beta _{2}\left( \left\| {\mathbf {q}}_{e}^{k}\right\| _{\mathrm {F}}^{2} -\left\| {\mathbf {q}}_{e}^{k+1}\right\| _{\mathrm {F}}^{2}\right) +\beta _{3}\left( \left\| r_{e}^{k}\right\| _{2}^{2}-\left\| r_{e}^{k+1}\right\| _{2}^{2}\right) \\&\quad \ge \beta _{1}\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} +\beta _{1}\left\| h_{e}^{k+1}-h_{e}^{k}\right\| _{2}^{2} +\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2}u_{e}^{k+1}\right\| _{\mathrm {F}}^{2} \\ {}&\qquad +\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-{\mathbf {q}}_{e}^{k}\right\| _{\mathrm {F}}^{2} +\beta _{3}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} +\beta _{3}\left\| r_{e}^{k+1}-r_{e}^{k}\right\| _{2}^{2}. \end{aligned}$$

The above inequation is added from the $k=0$ to the $k=M$, we have

$$\begin{aligned}&\frac{1}{\beta _{1}}\left( \left\| \varrho _{1e}^0\right\| _{2}^{2}-\left\| \varrho _{1e}^{M+1}\right\| _{2}^{2}\right) + \frac{1}{\beta _{2}}\left( \left\| \varvec{\varrho }_{2e}^0\right\| _{\mathrm {F}}^2 -\left\| \varvec{\varrho }_{2e}^{M+1}\right\| _{\mathrm {F}}^2\right) +\frac{1}{\beta _{3}}\left( \left\| \varrho _{3e}^0\right\| _{2}^{2}-\left\| \varrho _{3e}^{M+1}\right\| _{2}^{2}\right) \\&\qquad +\beta _{1}\left( \left\| h_{e}^{0}\right\| _{2}^{2}-\left\| h_{e}^{M+1}\right\| _{2}^{2}\right) +\beta _{2}\left( \left\| {\mathbf {q}}_{e}^{0}\right\| _{\mathrm {F}}^{2}-\left\| {\mathbf {q}}_{e}^{M+1}\right\| _{\mathrm {F}}^{2}\right) +\beta _{3}\left( \left\| r_{e}^{0}\right\| _{2}^{2}-\left\| r_{e}^{M+1}\right\| _{2}^{2}\right) \\&\quad \ge \sum \limits _{k=0}^{M}\beta _{1}\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} +\sum \limits _{k=0}^{M}\beta _{1}\left\| h_{e}^{k+1}-h_{e}^{k}\right\| _{2}^{2} +\sum \limits _{k=0}^{M}\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2}u_{e}^{k+1}\right\| _{\mathrm {F}}^{2} \\&\qquad +\sum \limits _{k=0}^{M}\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-{\mathbf {q}}_{e}^{k}\right\| _{\mathrm {F}}^{2} +\sum \limits _{k=0}^{M}\beta _{3}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} +\sum \limits _{k=0}^{M}\beta _{3}\left\| r_{e}^{k+1}-r_{e}^{k}\right\| _{2}^{2}. \end{aligned}$$

Above inequation can furthermore shrunken as

$$\begin{aligned}&\frac{1}{\beta _{1}}\left\| \varrho _{1e}^0\right\| _{2}^{2}+\frac{1}{\beta _{2}} \left\| \varvec{\varrho }_{2e}^0\right\| _{\mathrm {F}}^{2} +\frac{1}{\beta _{3}}\left\| \varrho _{3e}^0\right\| _{2}^{2}+\beta _{1}\left\| h_{e}^{0}\right\| _{2}^{2} +\beta _{2}\Vert {\mathbf {q}}_{e}^{0}\Vert _{\mathrm {F}}^{2} +\beta _{3}\left\| r_{e}^{0}\right\| _{2}^{2}\nonumber \\&\quad \ge \sum \limits _{k=0}^{M}\beta _{1}\left\| h_{e}^{k+1} -u_{e}^{k+1}\right\| _{2}^{2} +\sum \limits _{k=0}^{M}\beta _{1}\left\| h_{e}^{k+1}-h_{e}^{k}\right\| _{2}^{2} +\sum \limits _{k=0}^{M}\beta _{2}\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2}u_{e}^{k+1} \right\| _{\mathrm {F}}^{2}\nonumber \\&\qquad +\sum \limits _{k=0}^{M}\beta _{2}\Vert {\mathbf {q}}_{e}^{k+1} -{\mathbf {q}}_{e}^{k}\Vert _{\mathrm {F}}^{2} +\sum \limits _{k=0}^{M}\beta _{3}\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2} +\sum \limits _{k=0}^{M}\beta _{3}\left\| r_{e}^{k+1}-r_{e}^{k}\right\| _{2}^{2}. \end{aligned}$$

(36)

Using the boundedness on the left of (36), we have

$$\begin{aligned} {\left\{ \begin{array}{ll} \lim \limits _{k\rightarrow \infty }\left\| h_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}=0.\\ \lim \limits _{k\rightarrow \infty }\left\| {\mathbf {q}}_{e}^{k+1}-\nabla ^{2}u_{e}^{k+1}\right\| _{\mathrm {F}}^{2}=0.\\ \lim \limits _{k\rightarrow \infty }\left\| r_{e}^{k+1}-u_{e}^{k+1}\right\| _{2}^{2}=0. \end{array}\right. } \end{aligned}$$

(37)

and

$$\begin{aligned} {\left\{ \begin{array}{ll} \lim \limits _{k\rightarrow \infty }\left\| h^{k+1}-h^{k}\right\| _{2}^{2}=0,\\ \lim \limits _{k\rightarrow \infty }\left\| {\mathbf {q}}^{k+1}-{\mathbf {q}}^{k}\right\| _{\mathrm {F}}^{2}=0,\\ \lim \limits _{k\rightarrow \infty }\left\| r^{k+1}-r^{k}\right\| _{2}^{2}=0. \end{array}\right. } \end{aligned}$$

(38)

Due to the saddle point $(h^{*},{\mathbf {q}}^{*},r^{*},u^{*},\varrho _{1}^{*},\varvec{\varrho }_{2}^{*},\varrho _{3}^{*})$, the limitations in (37) furthermore imply that

$$\begin{aligned} {\left\{ \begin{array}{ll} \lim \limits _{k\rightarrow \infty }\left\| h^{k+1}-u^{k+1}+f\right\| _{2}^{2}=0.\\ \lim \limits _{k\rightarrow \infty }\left\| {\mathbf {q}}^{k+1}-\nabla ^{2}u^{k+1}\right\| _{\mathrm {F}}^{2}=0.\\ \lim \limits _{k\rightarrow \infty }\left\| r^{k+1}-u^{k+1}\right\| _{2}^{2}=0. \end{array}\right. } \end{aligned}$$

(39)

Based on the convexity of the primal problem in the problem (4), there exists a convergent sequence, without loss of generality by still setting  $\left( u^{k},h^{k},{\mathbf {q}}^{k},r^{k}\right)$ such that

$$\begin{aligned} \lim \limits _{k \rightarrow \infty }\left( h^k,{\mathbf {q}}^k, r^k,u^k\right) =\left( \bar{h}^\star ,{\mathbf {q}}^\star ,r^\star ,u^\star \right) . \end{aligned}$$

Now we need to show that $\left( \bar{h}^\star ,{\mathbf {q}}^\star ,r^\star ,u^\star \right)$ is the saddle point of the problem (4). In fact, combining (10)–(12) with (38)–(39) and taking the limitation, we can deduce that

$$\begin{aligned} {\left\{ \begin{array}{ll} h^\star =u^\star -f\\ {\mathbf {q}}^\star =\nabla ^{2}u^\star \\ r^\star =u^\star \end{array}\right. }\hbox {and} {\left\{ \begin{array}{ll} \lim \limits _{k\rightarrow \infty }\varrho _{1}^{k}=\varrho _{1}^\star ,\\ \lim \limits _{k\rightarrow \infty }\varvec{\varrho }_{2}^{k}=\varvec{\varrho }_{2}^\star ,\\ \lim \limits _{k\rightarrow \infty }\varrho _{3}^{k}=\varrho _{3}^\star . \end{array}\right. } \end{aligned}$$

This implies that $\left( h^{\star },{\mathbf {q}}^{\star },r^{\star },u^{\star },\varrho _{1}^{\star }, \varvec{\varrho }_{2}^{\star },\varrho _{3}^{\star }\right)$ is a saddle point. That is to say, we can set $\left( h^{\star },{\mathbf {q}}^{\star },r^{\star },u^{\star },\varrho _{1}^{\star }, \varvec{\varrho }_{2}^{\star },\varrho _{3}^{\star }\right) :=\left( h^{\star }, {\mathbf {q}}^{\star },r^{\star },u^{\star },\varrho _{1}^{\star }, \varvec{\varrho }_{2}^{\star },\varrho _{3}^{\star }\right) .$ Following from Lemma 2, we can deduce that $\{u^k\}$ converges to the solution of the problem (4). $\square$