Skip to main content
Springer Nature Link
Log in

Augmented Lagrangian alternating direction method for low-rank minimization via non-convex approximation

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This paper concerns the low-rank minimization problems which consist of finding a matrix of minimum rank subject to linear constraints. Many existing approaches, which used the nuclear norm as a convex surrogate of the rank function, usually result in a suboptimal solution. To seek a tighter rank approximation, we develop a non-convex surrogate to approximate the rank function based on the Laplace function. An iterative algorithm based on the augmented Lagrangian multipliers method is developed. Empirical studies for practical applications including robust principal component analysis and low-rank representation demonstrate that our proposed algorithm outperforms many other state-of-the-art convex and non-convex methods developed recently in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The contribution of zero singular values in Laplace norm is the same as the true rank function and the nuclear norm.

References

  1. Candès, E.J., Li, X., Ma, Y., et al.: Robust principal component analysis? J. ACM 58(3), 11 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  2. Wright, J., Ganesh, A., Rao, S., et al.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: 23rd Annual Conference on Advances in Neural Information Processing Systems 22, Vancouver, 7–10 Dec 2009 (2009)

  3. De la Torre, F., Black, M.J.: Robust principal component analysis for computer vision. ICCV 1, 362–369 (2001)

    Google Scholar 

  4. Xu, H., Caramanis, C., Sanghavi, S.: Robust PCA via outlier pursuit. In: Proceedings of the 23rd International Conference on Neural Information Processing Systems, NIPS’10, pp. 2496–2504, Curran Associates Inc, Vancouver, 6–9 Dec 2010 (2010)

  5. Liu, G., Lin, Z., Yan, S., et al.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)

    Article  Google Scholar 

  6. Zhang, Z., Liang, X., Ganesh, A., et al.: TILT: transform invariant low-rank textures. Int. J. Comput. Vis. 99(1), 1–24 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen, C., He, B., Yuan, X.: Matrix completion via an alternating direction method. IMA J. Numer. Anal. 32(1), 227–245 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  9. Vidal, R., Favaro, P.: Low rank subspace clustering (LRSC). Pattern Recognit. Lett. 43, 47–61 (2014)

    Article  Google Scholar 

  10. Zhou, X., Yang, C., Zhao, H., et al.: Low rank modeling and its applications in image analysis. ACM Comput. Surv. 47(2), 36 (2015)

    Google Scholar 

  11. Xu, Y., Yin, W., Wen, Z., et al.: An alternating direction algorithm for matrix completion with nonnegative factors. Front. Math. China 7(2), 365–384 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  12. Otazo, R., Candès, E.J., Sodickson, K.: Low-rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components. Magn. Reson. Med. 73(3), 1125–1136 (2015)

    Article  Google Scholar 

  13. Mardani, M., Mateos, G., Giannakis, G.: Recovery of low-rank plus compressed sparse matrices with application to unveiling traffic anomalies. IEEE Trans. Inf. Theory 59(8), 5186–5205 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Sun, Q., Xiang, S., Ye, J.: Robust principal component analysis via capped norms. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 311–319 (2013)

  15. Hu, Y., Zhang, D., Ye, J., et al.: Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2117–2130 (2013)

    Article  Google Scholar 

  16. Marjanovic, G., Solo, V.: On optimization and matrix completion. IEEE Trans. Signal Process. 60(11), 5714–5724 (2012)

    Article  MathSciNet  Google Scholar 

  17. Peng, C., Kang, Z., Li, H., et al.: Subspace clustering using log-determinant rank approximation. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 925–934 (2015)

  18. Wang, Y., Liu, P., Li, Z., et al.: Data regularization using Gaussian beams decomposition and sparse norms. J. Inverse Ill Posed Probl. 21(1), 1–23 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Tao, P., An, L.: Convex analysis approach to dc programming: theory, algorithms and applications. Acta Math. Vietnam. 22(1), 289–355 (1997)

    MathSciNet  MATH  Google Scholar 

  20. Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Academic press, Cambridge (1982)

    MATH  Google Scholar 

  21. Boyd, S., Parikh, N., Chu, E., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  22. Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055 (2010)

  23. Bazaraa, M., Sherali, H., Shetty, C.: Nonlinear Programming: Theory and Algorithms. Wiley, Hoboken (1993)

    MATH  Google Scholar 

  24. Huang, J., Nie, F., Huang, H., et al.: Robust manifold nonnegative matrix factorization. ACM Trans. Knowl. Discov. Data (TKDD) 8(3), 11 (2014)

    Google Scholar 

  25. Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B 68(1), 49–67 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  26. Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  27. Tao, M., Yuan, X.: Recovering low-rank and sparse components of matrices from incomplete and noisy observation. SIAM J. Optim. 21(1), 57–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  28. Zhou, T., Tao, D.: Godec: Randomized low-rank and sparse matrix decomposition in noisy case. In: 28th International Conference on Machine Learning (ICML), pp. 33–40, Bellevue, 28 June–2 July 2011 (2011)

  29. Netrapalli, P., Niranjan, U.N., Sanghavi, S., et al.: Non-convex robust PCA. In: Ghahraman, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1107–1115. Curran Associates, Inc. (2014)

  30. Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)

    Article  Google Scholar 

  31. Yan, J., Pollefeys, M.: A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate. ECCV 2006. Springer, Berlin (2006)

    Google Scholar 

  32. Chen, G., Lerman, G.: Spectral curvature clustering (SCC). Int. J. Comput. Vis. 81(3), 317–330 (2009)

    Article  Google Scholar 

  33. Li, L., Huang, W., Gu, I., et al.: Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans. Image Process. 13(11), 1459–1472 (2004)

    Article  Google Scholar 

  34. Goldfarb, D., Ma, S., Scheinberg, K.: Fast alternating linearization methods for minimizing the sum of two convex functions. Math. Program. 141(1–2), 349–382 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  35. Peng, Y., Ganesh, A., Wright, J., et al.: RASL: robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2233–2246 (2012)

    Article  Google Scholar 

  36. Lee, K.C., Ho, J., Kriegman, D.J.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 684–698 (2005)

    Article  Google Scholar 

  37. Agarwal, P.K., Mustafa, N.H.: k-Means projective clustering. In: Proceedings of the Twenty-Third ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 155–165 (2004)

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant No. 11241005) and Research Project Supported by Shanxi Scholarship Council of China (2015-093).

Author information

Authors and Affiliations

  1. College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao, 266590, China

    Yongyong Chen & Yongli Wang

  2. School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China

    Mingqiang Li

  3. Shandong Academy of Science, Jinan, 250014, China

    Guoping He

Authors
  1. Yongyong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongli Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoping He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongli Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Wang, Y., Li, M. et al. Augmented Lagrangian alternating direction method for low-rank minimization via non-convex approximation. SIViP 11, 1271–1278 (2017). https://doi.org/10.1007/s11760-017-1084-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-017-1084-9

Keywords

Search

Navigation