Skip to main content
SpringerLink
Log in

Robust Recovery of Low Rank Matrix by Nonconvex Rank Regularization

  • Conference paper
  • First Online:
Image and Graphics (ICIG 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12889))

Included in the following conference series:

Abstract

As we know, nuclear norm based regularization methods have the real-world applications in pattern recognition and computer vision. However, there exists a biased estimator when nuclear norm relaxes the rank function. To solve this issue, we focus on studying nonconvex rank regularization problems for both robust matrix completion (RMC) and low rank representation (LRR), respectively. By extending both to a general low rank matrix minimization problem, we develop a nonconvex alternating direction method of multipliers (ADMM). Moreover, the convergence results, i.e., the variable sequence generated by the nonconvex ADMM is bounded and its subsequence converges to a stationary point. Meanwhile, its limiting point satisfies the Karush-Kuhn-Tucher (KKT) conditions provided under some milder assumptions. Numerical experiments can verify the convergence properties of the theoretical results and the performance shows its superiority on both image inpainting and subspace clustering.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For \(\ell _p\)-norm, if \(\sigma _i=0\), then \(\partial f_{\lambda }(\sigma _i)=\{+\infty \}\), we can guarantee from the iteration rules for \(\mathbf {X}_{k+1}\) in subproblem (8) that the rank of the generated sequence \(\{\mathbf {X}_{k+1}\}\) is nonincreasing.

  2. 2.

    http://vision.ucsd.edu/leekc/ExtYaleDatabase/ExtYale.

References

  1. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137(1–2), 91–129 (2013)

    Article  MathSciNet  Google Scholar 

  2. Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM. J. Optim. 17(4), 1205–1223 (2007)

    Article  Google Scholar 

  3. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1-2), 459–494 (2014)

    Google Scholar 

  4. Brbić, M., Kopriva, I.: \(\ell _0\)-motivated low-rank sparse subspace clustering. IEEE Trans. Cybern. 50(4), 1711–1725 (2020)

    Article  Google Scholar 

  5. Candes, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 1–37 (2011)

    Article  MathSciNet  Google Scholar 

  6. Candes, E., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Infor. Theo. 56(5), 2053–2080 (2010)

    Article  MathSciNet  Google Scholar 

  7. Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1–2), 57–79 (2016)

    Article  MathSciNet  Google Scholar 

  8. Chen, Y., Jalali, A., Sanghavi, S., Caramanis, C.: Low-rank matrix recovery from errors and erasures. IEEE Trans. Infor. Theo. 59(7), 4324–4337 (2013)

    Article  Google Scholar 

  9. Clarke, F.: Optimization and nonsmooth analysis. Society for Industrial and Applied Mathematics (1990)

    Google Scholar 

  10. Dong, W., Wu, X.: Robust low rank subspace segmentation via joint \(\ell _{21} \)-norm minimization. Neural Process. Lett. 48(1), 299–312 (2018)

    Article  Google Scholar 

  11. Friedman, J.H.: Fast sparse regression and classification. Int. J. Forecasting 28(3), 722–738 (2012)

    Article  Google Scholar 

  12. Gao, C., Wang, N., Yu, Q., Zhang, Z.: A feasible nonconvex relaxation approach to feature selection. In: Proceedings Association Advancement Artificial Intelligence (AAAI), pp. 356–361 (2011)

    Google Scholar 

  13. Geman, D., Yang, C.: Nonlinear image recovery with half-quadratic regularization. IEEE Trans. Image. Process. 4(7), 932–946 (1994)

    Article  Google Scholar 

  14. Gu, S., Xie, Q., Meng, D., Zuo, W., Feng, X., Zhang, L.: Weighted nuclear norm minimization and its applications to low level vision. Int. J. Comput. Vis. 121(2), 183–208 (2017)

    Article  Google Scholar 

  15. Hu, Y., Zhang, D., Ye, J., Li, X., He, X.: Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans. Pattern. Anal. Mach. Intell. 35(9), 2117–2130 (2013)

    Article  Google Scholar 

  16. Hu, Z., Nie, F., Tian, L., Li, X.: A comprehensive survey for low rank regularization. arXiv preprint arXiv:1808.04521 (2018)

  17. Kang, Z., Peng, C., Cheng, Q.: Robust subspace clustering via tighter rank approximation. In: Proceedings ACM Conference Information and Knowledge Management (CIKM), pp. 655–661 (2012)

    Google Scholar 

  18. Lan, X., Zhang, S., Yuen, P.C., Chellappa, R.: Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans. Image. Process. 27(4), 2022–2037 (2018)

    Article  MathSciNet  Google Scholar 

  19. Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern. Anal. Mach. Intell. 35(1), 171–184 (2013)

    Article  Google Scholar 

  20. Lu, C., Lin, Z., Yan, S.: Smoothed low rank and sparse matrix recovery by iteratively reweighted least squares minimization. IEEE Trans. Image. Process. 24(2), 646–654 (2015)

    Article  MathSciNet  Google Scholar 

  21. Lu, C., Tang, J., Yan, S., Lin, Z.: Nonconvex nonsmooth low-rank minimization via iteratively reweighted nuclear norm. IEEE Trans. Image. Process. 25(2), 829–839 (2016)

    Article  MathSciNet  Google Scholar 

  22. Lu, C., Zhu, C., Xu, C., Yan, S., Lin, Z.: Generalized singular value thresholding. In: Proceedings Association Advances Artificial Intelligence (AAAI), pp. 1805–1811 (2015)

    Google Scholar 

  23. Nie, F., Wang, H., Huang, H., Ding, C.: Joint Schatten-\(p\) norm and \(\ell _p\)-norm robust matrix completion for missing value recovery. Knowl. Infor. Syst. 42(3), 525–544 (2015)

    Article  Google Scholar 

  24. Oh, T.H., Tai, Y., Bazin, J.C., Kim, H., Kweon, I.S.: Partial sum minimization of singular values in robust PCA: algorithm and applications. IEEE Trans. Pattern. Anal. Mach. Intell. 38(4), 744–758 (2015)

    Article  Google Scholar 

  25. Trzasko, J., Manduca, A.: Highly undersampled magnetic resonance image reconstruction via homotopic \(\ell _p\)-minimization. IEEE Trans. Med. Image. 28(1), 106–121 (2009)

    Article  Google Scholar 

  26. Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. 78(1), 29–63 (2019)

    Article  MathSciNet  Google Scholar 

  27. Wei, L., Wang, X., Wu, A., Zhou, R., Zhu, C.: Robust subspace segmentation by self-representation constrained low-rank representation. Neural. Process. Lett. 48(3), 1671–1691 (2018)

    Article  Google Scholar 

  28. Wen, F., Chu, L., Liu, P., Qiu, R.C.: A survey on nonconvex regularization-based sparse and low-rank recovery in signal processing, statistics, and machine learning. IEEE Access. 6, 69883–69906 (2018)

    Article  Google Scholar 

  29. Xie, Y., Gu, S., Liu, Y., Zuo, W., Zhang, W., Zhang, L.: Weighted Schatten-\(p\) norm minimization for image denoising and background subtraction. IEEE Trans. Image. Process. 25(10), 4842–4857 (2016)

    Article  MathSciNet  Google Scholar 

  30. Yang, L., Pong, T., Chen, X.: Alternating direction method of multipliers for nonconvex background/foreground extraction. SIAM J. Imag. Sci. 10(1), 74–110 (2017)

    Article  Google Scholar 

  31. Yao, Q., Kwok, J.T., Gao, F., Chen, W., Liu, T.: Efficient inexact proximal gradient algorithm for nonconvex problems. In: Proceedings Association Advancement Artificial Intelligence (AAAI), pp. 3308–3314 (2017)

    Google Scholar 

  32. Zhang, H., Gong, C., Qian, J., Zhang, B., Xu, C., Yang, J.: Efficient recovery of low-rank matrix via double nonconvex nonsmooth rank minimization. IEEE Trans. Neural Netw. Learn. Syst. 30(10), 2916–2925 (2019)

    Article  MathSciNet  Google Scholar 

  33. Zhang, H., Qian, F., Shang, F., Du, W., Qian, J., Yang, J.: Global convergence guarantees of (A)GIST for a family of nonconvex sparse learning problems. IEEE Trans. Cybern. https://doi.org/10.1109/TCYB.2020.3010960 (2020)

  34. Zhang, H., Qian, J., Gao, J., Yang, J., Xu, C.: Scalable proximal Jacobian iteration method with global convergence analysis for nonconvex unconstrained composite optimizations. IEEE Trans. Neural Netw. Learn. Syst. 30(9), 2825–2839 (2019)

    Article  MathSciNet  Google Scholar 

  35. Zhang, H., Yang, J., Qian, J., Luo, W.: Nonconvex relaxation based matrix regression for face recognition with structural noise and mixed noise. Neurocomput. 269(20), 188–198 (2017)

    Article  Google Scholar 

  36. Zhang, H., Yang, J., Shang, F., Gong, C., Zhang, Z.: LRR for subspace segmentation via tractable Schatten-\(p\) norm minimization and factorization. IEEE Trans. Cybern. 49(5), 1722–1734 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their valuable comments. This work was supported in part by the National Natural Science Fund for Distinguished Young Scholars under Grant 61725301, in part by the National Science Fund of China under Grant 61973124, 61702197, 61876083, and 61906067, in part by the China Postdoctoral Science Foundation under Grant 2019M651415 and 2020T130191, and in part by the University of Macau under UM Macao Talent Programme (UMMTP-2020-01).

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Macau, Macau, 999078, People’s Republic of China

    Hengmin Zhang & Bob Zhang

  2. School of Information Science and Engineering, Key Laboratory of Advanced Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, 200237, People’s Republic of China

    Hengmin Zhang & Wenli Du

  3. College of Mathematics and Informatics, South China Agricultural University, Guangzhou, 510642, People’s Republic of China

    Wei Luo

  4. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Jianjun Qian & Jian Yang

Authors
  1. Hengmin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenli Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianjun Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bob Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bob Zhang .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Yuxin Peng

  2. Tsinghua University, Beijing, China

    Shi-Min Hu

  3. Tampere University, Tampere, Finland

    Moncef Gabbouj

  4. Zhejiang University, Hangzhou, China

    Kun Zhou

  5. Technion – Israel Institute of Technology, Haifa, Israel

    Michael Elad

  6. Tsinghua University, Beijing, China

    Kun Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, H., Luo, W., Du, W., Qian, J., Yang, J., Zhang, B. (2021). Robust Recovery of Low Rank Matrix by Nonconvex Rank Regularization. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12889. Springer, Cham. https://doi.org/10.1007/978-3-030-87358-5_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87358-5_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87357-8

  • Online ISBN: 978-3-030-87358-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation