Abstract
The problem of recovering a low-rank matrix from partial entries, known as low-rank matrix completion, has been extensively investigated in recent years. It can be viewed as a special case of the affine constrained rank minimization problem which is NP-hard in general and is computationally hard to solve in practice. One widely studied approach is to replace the matrix rank function by its nuclear-norm, which leads to the convex nuclear-norm minimization problem solved efficiently by many popular convex optimization algorithms. In this paper, we propose a new nonconvex approach to better approximate the rank function. The new approximation function is actually the Moreau envelope of the rank function (MER) which has an explicit expression. The new approximation problem of low-rank matrix completion based on MER can be converted to an optimization problem with two variables. We then adapt the proximal alternating minimization algorithm to solve it. The convergence (rate) of the proposed algorithm is proved and its accelerated version is also developed. Numerical experiments on completion of low-rank random matrices and standard image inpainting problems have shown that our algorithms have better performance than some state-of-art methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Attouch, H., & Bolte, J. (2009). On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Mathematical Programming, 116(1), 5–16.
Attouch, H., Bolte, J., Redont, P., & Soubeyran, A. (2010). Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka–Łojasiewicz inequality. Mathematics of Operations Research, 35(2), 438–457.
Auslender, A. (1992). Asymptotic properties of the Fenchel dual functional and applications to decomposition problems. Journal of Optimization Theory and Applications, 73(3), 427–449.
Bauschke, H. H., & Combettes, P. L. (2011). Convex analysis and monotone operator theory in Hilbert spaces. CMS Books in Mathematics. New York: Springer.
Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1), 183–202.
Bertsekas, D. (1999). Nonlinear optimisation, 2nd ed., Athena, Belmont, Massachusetts.
Bolte, J., Sabach, S., & Teboulle, M. (2013). Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming, 146(1), 459–494.
Cai, J. F., Candès, E. J., & Shen, Z. W. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982.
Candès, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717–772.
Candès, E. J., Romberg, J., & Tao, T. (2006). Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8), 1207–1223.
Candès, E. J., & Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Transactions on Information Theory, 56(5), 2053–2080.
Candès, E. J., & Wakin, M. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30.
Cao, W. F., Sun, J., & Xu, Z. B. (2013). Fast image deconvolution using closed-form thresholding formulas of \(L_q\) \((q = 1/2, 2/3)\) regularization. Journal of Visual Communication and Image Representation, 24(1), 31–41.
Chen, P., & Suter, D. (2004). Recovering the missing components in a large noisy low-rank matrix: Application to SFM source. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 1051–1063.
Fan, J. Q., & Li, R. Z. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.
Fazel, M. (2002). Matrix rank minimization with applications. Ph.D. thesis, Stanford University.
Geng, J., Wang, L. S., & Wang, Y. F. (2015). A non-convex algorithm framework based on DC programming and DCA for matrix completion. Numerical Algorithms, 68(4), 903–921.
Goldberg, D., Nichols, D., Oki, B. M., & Terry, D. (1992). Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12), 61–70.
Hu, Y., Zhang, D. B., Ye, J. P., Li, X. L., & He, X. F. (2013). Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(9), 2117–2130.
Jin, Z. F., Wan, Z. P., Jiao, Y. L., & Lu, J. X. (2016). An alternating direction method with continuation for nonconvex low rank minimization. Journal of Scientific Computing, 66(2), 849–869.
Lai, M. J., & Wang, J. Y. (2011). An unconstrained \(\ell _q\) minimization with \(0<q\le 1\) for sparse solution of underdetermined linear systems. SIAM Journal on Optimization, 21(1), 82–101.
Li, G., & Pong, T. (2016). Calculus of the exponent of Kurdyka–Łojasiewicz inequality and its applications to linear convergence of first-order methods. arXiv:1602.02915v1.
Li, G., Mordukhovich, B. S., & Pham, T. S. (2015). New fractional error bounds for polynomial systems with applications to Hölderian stability in optimization and spectral theory of tensors. Mathematical Programming, 153(2), 333–362.
Liu, Z., & Vandenberghe, L. (2009). Interior-point method for nuclear norm approximation with application to system identification. SIAM Journal on Matrix Analysis and Applications, 31(3), 1235–1256.
Lu, Z. S., & Zhang, Y. (2015). Schatten-\(p\) quasi-norm regularized matrix optimization via iterative reweighted singular value minimization. http://www.optimization-online.org/DB_HTML/2015/11/5215.html.
Lu, C. Y., Tang, J. H., Yan, S. C., & Lin, Z. C. (2015). Nonconvex nonsmooth low-rank minimization via iteratively reweighted nuclear norm. arXiv:1510.06895.
Ma, S. Q., Goldfarb, D., & Chen, L. F. (2011). Fixed point and Bregman iterative methods for matrix rank minimization. Mathematical Programming, 128(1), 321–353.
Mohammadi, M. M., Zadeh, M. B., Amini, A., & Jutten, C. (2013). Recovery of low-rank matrices under affine constraints via a smoothed rank function. arXiv:1308.2293.
Mohammadi, M. M., Zadeh, M. B., & Jutten, C. (2009). A fast approach for overcomplete sparse decomposition based on smoothed \(\ell _0\)-norm. IEEE Transactions on Signal Processing, 57(1), 289–301.
Recht, B., Fazel, M., & Parrilo, P. A. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3), 471–501.
Rockafellar, R. (1970). Convex analysis. Princeton: Princeton University Press.
Singer, A. (2008). A remark on global positioning from local distances. Proceedings of the National Academy of Sciences of the United States of America, 105(28), 9507–9511.
Srebro, N. (2004). Learning with matrix factorizations. Ph.D. thesis, Massachusetts Institute of Technology.
Toh, K. C., & Yun, S. W. (2012). An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pacific Journal of Optimization, 6(3), 615–640.
Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2), 137C154.
Tseng, P. (2001). Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of Optimization Theory and Applications, 109(3), 475–494.
Tütüncü, R. H., Toh, K. C., & Todd, M. J. (2003). Solving semidefinite-quadratic-linear programs using SDPT3. Mathematical Programming, 95(2), 189–217.
Yang, J. F., & Yuan, X. M. (2013). Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 82(281), 301–329.
Zhang, S., & Xin, J. (2014). Minimization of transformed \(\ell _1\) penalty: Closed form representation and iterative thresholding algorithms. arXiv preprint arXiv:1412.5240.
Zhang, S., Yin, P. H., & Xin, J. (2015). Transformed Schatten-1 iterative thresholding algorithms for matrix rank minimization and applications. arXiv:1506.04444.
Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2), 894–942.
Zhang, T. (2010). Analysis of multi-stage convex relaxation for sparse regularization. Journal of Machine Learning Research, 11(Mar), 1081–1107.
Zhao, Y. B. (2012). An approximation theory of matrix rank minimization and its application to quadratic equations. Linear Algebra and Its Applications, 437(1), 77–93.
Acknowledgements
The authors would like to thank the editor-in chief and the anonymous reviewers for their insightful and constructive comments, which help to enrich the content and improve the presentation of this paper. This research was supported by the National Natural Science Foundation of China under the grants 11771347, 91730306, 41390454, and 11271297, and supported by the Nanhu Scholars Program for Young Scholars of XYNU.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yu, Y., Peng, J. & Yue, S. A new nonconvex approach to low-rank matrix completion with application to image inpainting. Multidim Syst Sign Process 30, 145–174 (2019). https://doi.org/10.1007/s11045-018-0549-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-018-0549-5