Algorithms of matrix recovery based on truncated Schatten p-norm

Abstract

In recent years, algorithms to recovery low-rank matrix have become one of the research hotspots, and more corresponding optimization models with nuclear norm have also been proposed. However, nuclear norm is not a good approximation to the rank function. This paper proposes a matrix completion model and a low-rank sparse decomposition model based on truncated Schatten p-norm, respectively, which combine Schatten p-norm with truncated nuclear norm, so that the models are more flexible. To solve these models, the function expansion method is first used to transform the non-convex optimization models into the convex optimization ones. Then, the two-step iterative algorithm based on alternating direction multiplier method (ADMM) is employed to solve the models. Further, the convergence of the proposed algorithm is proved mathematically. The superiority of the proposed method is further verified by comparing the existing methods in synthetic data and actual images.

Appendix

The proof of Theorem 2.1

Let $Q_{X}=M-N_{k+1}+\frac{1}{\mu _{k}}Y_{k}$, then in $(k+1)$-th iteration, its singular value decomposition is $[U_{k}, \Delta _{k}, V_{k}]=SVD(Q_{X})$. From Lemma 2.2, it follows that $X_{k+1}=U_{k}\Lambda _{k}V_{k}^{\top }$, where $\Lambda _{k}=\{(\Delta _{k}-\frac{1}{\mu _{k}}W)_{+}\}$. So from (2.14) it follows that

$$\begin{aligned} \begin{aligned} \Vert Y_{k+1}\Vert _{F}&= \left\| Y_{k}+\mu _{k}(M-X_{k+1}-N_{k+1})\right\| _{F}\\&=\mu _{k}\left\| \frac{1}{\mu _{k}}Y_{k}+M-X_{k+1}-N_{k+1}\right\| _{F}\\&=\mu _{k}\left\| U_{k}\Delta _{k}V_{k}^{\top }-U_{k}\Lambda _{k}V_{k}^{\top }\right\| _{F}\\&=\mu _{k}\Vert \Delta _{k}-\Lambda _{k}\Vert _{F}\\&\le \mu _{k}\left\| \frac{W}{\mu _{k}}\right\| _{F}=\Vert W\Vert _{F}, \end{aligned} \end{aligned}$$

(5.1)

which shows that $\{Y_{k}\}$ is bounded. Recalling the definition of augmented Lagrangian function (2.8), for the solution of $(k+1)$-th iteration $\{X_{k+1}, N_{k+1}\}$, there is

$$\begin{aligned} \Gamma (X_{k+1}, N_{k+1}, Y_{k}, \mu _{k})\le \Gamma (X_{k}, N_{k}, Y_{k}, \mu _{k}). \end{aligned}$$

Noticing $Y_{k+1}=Y_{k}+\mu _{k}(M-X_{k+1}-N_{k+1})$, we have

$$\begin{aligned} \begin{aligned}&\Gamma (X_{k}, N_{k}, Y_{k}, \mu _{k}) \\&\quad =\Gamma (X_{k}, N_{k}, Y_{k-1}, \mu _{k-1})+\frac{\mu _{k}-\mu _{k-1}}{2}\Vert M-X_{k}-N_{k}\Vert _{F}^{2}\\&\qquad +\langle Y_{k}-Y_{k-1},M-X_{k}-N_{k}\rangle \\&\quad =\Gamma (X_{k}, N_{k}, Y_{k-1}, \mu _{k-1})+\frac{\mu _{k}-\mu _{k-1}}{2}\left\| \frac{1}{\mu _{k-1}}(Y_{k}-Y_{k-1})\right\| _{F}^{2}\\&\qquad +\langle Y_{k}-Y_{k-1},\frac{1}{\mu _{k-1}}(Y_{k}-Y_{k-1})\rangle \\&\quad =\Gamma (X_{k}, N_{k}, Y_{k-1}, \mu _{k-1})+\frac{\mu _{k}+\mu _{k-1}}{2\mu _{k-1}^{2}}\Vert (Y_{k}-Y_{k-1})\Vert _{F}^{2}. \end{aligned} \end{aligned}$$

(5.2)

Therefore,

$$\begin{aligned} \begin{aligned}&\Gamma (X_{k+1}, N_{k+1}, Y_{k}, \mu _{k} ) \\&\quad \le \Gamma (X_{1}, N_{1}, Y_{0}, \mu _{0}) +\frac{\mu _{k}+\mu _{k-1}}{2\mu _{k-1}^{2}}\Vert (Y_{k}-Y_{k-1})\Vert _{F}^{2}. \end{aligned} \end{aligned}$$

(5.3)

Since $\{Y_{k}\}$ is bound, and there is $\mu _{k+1}=\min (\rho \mu _{k}, \mu _{\text {max}})$, $\Gamma (X_{k+1}, N_{k+1}, Y_{k}, \mu _{k})$ is bounded.

And because

$$\begin{aligned} \begin{aligned} \sum _{i=1}^{\min (m,n)}\omega _{i}\sigma _{i}(X_{k})&=\Vert X_{k}\Vert _{W,*}=\Gamma (X_{k}, N_{k}, Y_{k-1}, \mu _{k-1})\\&\quad +\frac{\mu _{k-1}}{2}\left( \frac{1}{\mu _{k-1}^{2}}\Vert Y_{k-1}\Vert _{F}^{2}-\Vert M-X_{k}-N_{k}+\frac{1}{\mu _{k-1}}\Vert Y_{k-1}\Vert _{F}^{2}\right) \\&=\Gamma (X_{k}, N_{k}, Y_{k-1}, \mu _{k-1})-\frac{1}{2\mu _{k-1}}(\Vert Y_{k}\Vert _{F}^{2}-\Vert Y_{k-1}\Vert _{F}^{2}), \end{aligned} \end{aligned}$$

(5.4)

$\{X_{k}\}$ is bounded. From $N_{k+1}=M-X_{k}+\frac{1}{\mu _{k}}Y_{k}$, it can be deduced that $\{N_{k}\}$ is also bounded. So $\{X_{k}, N_{k}, Y_{k}\}$ has at least one point of accumulation, and

$$\begin{aligned} \lim _{k \rightarrow +\infty }\Vert M-X_{k+1}-N_{k+1}\Vert _{F}=\lim _{k \rightarrow +\infty }\frac{1}{\mu _{k}}\Vert Y_{k+1}-Y_{k}\Vert _{F}=0. \end{aligned}$$

We prove the convergence of $X_{k}$ below. Because

$$\begin{aligned} \left\{ \begin{array}{l} X_{k}=U_{k-1}\Lambda _{k-1}V_{k-1}^{\top },\\ X_{k+1}=M-N_{k+1}-\frac{1}{\mu _{k}}(Y_{k+1}-Y_{k}),\\ N_{k}=M-X_{k}-\frac{1}{\mu _{k-1}}(Y_{k}-Y_{k-1}),\\ N_{k+1}=M-X_{k}+\frac{1}{\mu _{k}}Y_{k}.\\ \end{array} \right. \end{aligned}$$

(5.5)

we have

$$\begin{aligned} \begin{aligned}&\lim _{k \rightarrow +\infty }\Vert X_{k+1}-X_{k}\Vert _{F}\\&\quad =\lim _{k \rightarrow +\infty } \left\| M-N_{k+1}-\frac{1}{\mu _{k}}(Y_{k+1}-Y_{k})-X_{k}\right\| _{F}\\&\quad =\lim _{k \rightarrow +\infty }\left\| M-N_{k+1}-\frac{1}{\mu _{k}}(Y_{k+1}-Y_{k})-X_{k} + \left( N_{k}+\frac{1}{\mu _{k-1}}Y_{k-1}\right) - \left( N_{k}+\frac{1}{\mu _{k-1}}Y_{k-1}\right) \right\| _{F}\\&\quad \le \lim _{k \rightarrow +\infty }\left\| M+\frac{1}{\mu _{k-1}}Y_{k-1}-N_{k}-X_{k}\right\| _{F}\\&\qquad + \left\| N_{k}-N_{k+1}+\frac{1}{\mu _{k}}(Y_{k}-Y_{k+1})-\frac{1}{\mu _{k-1}}Y_{k-1})\right\| _{F}\\&\quad \le \lim _{k \rightarrow +\infty }\Vert \Delta _{k-1}-\Lambda _{k-1}\Vert _{F}+\Vert N_{k}-N_{k+1}\Vert _{F}\\&\qquad + \left\| \frac{1}{\mu _{k}}(Y_{k}-Y_{k+1})-\frac{1}{\mu _{k-1}}Y_{k-1})\right\| _{F}\\&\quad =\lim _{k \rightarrow +\infty }\Vert \Delta _{k-1}-\Lambda _{k-1}\Vert _{F}+ \left\| \frac{1}{\mu _{k-1}}(Y_{k-1}-Y_{k}) -\frac{1}{\mu _{k}}Y_{k})\right\| _{F}+ \left\| \frac{1}{\mu _{k}}(Y_{k}-Y_{k+1})-\frac{1}{\mu _{k-1}}Y_{k-1})\right\| _{F}\\&\quad =0. \end{aligned} \end{aligned}$$

(5.6)

This completes the proof of Theorem 2.1. $\square$