Skip to main content
SpringerLink
Log in

An implementable proximal point algorithmic framework for nuclear norm minimization

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

The nuclear norm minimization problem is to find a matrix with the minimum nuclear norm subject to linear and second order cone constraints. Such a problem often arises from the convex relaxation of a rank minimization problem with noisy data, and arises in many fields of engineering and science. In this paper, we study inexact proximal point algorithms in the primal, dual and primal-dual forms for solving the nuclear norm minimization with linear equality and second order cone constraints. We design efficient implementations of these algorithms and present comprehensive convergence results. In particular, we investigate the performance of our proposed algorithms in which the inner sub-problems are approximately solved by the gradient projection method or the accelerated proximal gradient method. Our numerical results for solving randomly generated matrix completion problems and real matrix completion problems show that our algorithms perform favorably in comparison to several recently proposed state-of-the-art algorithms. Interestingly, our proposed algorithms are connected with other algorithms that have been studied in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alfakih A.Y., Khandani A., Wolkowicz H.: Saving Euclidean distance matrix completion problems via semidefinite programming. Comp. Optim. Appl. 12, 13–30 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  2. Alizadeh F., Goldfarb D.: Second-order cone programming. Math. Program. 95, 3–51 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  3. Ames, B.P.W., Vavasis, S.A.: Nuclear norm minimization for the planted clique and biclique problems. preprint (2009)

  4. Barvinok A.: Problems of distance geometry and convex properties of quadratic maps. Discret. Comput. Geom. 13, 189–202 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  5. Beck A., Teboulle M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 1, 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  6. Bertsekas D.P.: On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Autom. Control 21, 174–184 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bertsekas D.P.: Nonlinear Programming. 2nd edn. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  8. Burer S., Monteiro R.D.C.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95, 329–357 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  9. Burer S., Monteiro R.D.C.: Local minima and convergence in low-rank semidefinite programs. Math. Program. 103, 427–444 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  10. Cai, J.-F., Candès, E.J., Shen, Z.W.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. (to appear)

  11. Candès, E.J.: Compressive sampling. In: International Congress of Mathematicians. vol. III, Eur. Math. Soc., Zöurich, pp. 1433–1452 (2006)

  12. Candès, E.J., Becker, S.: Singular value thresholding—codes for the SVT algorithm to minimize the nuclear norm of a matrix, subject to linear constraints, April 2009. Available at http://svt.caltech.edu/code.html

  13. Candès E.J., Tao T.: Nearly optimal signal recovery from random projections: universal encoding strategies. IEEE Trans. Info. Theory 52, 5406–5425 (2006)

    Article  Google Scholar 

  14. Candès E.J., Recht B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 712–772 (2009)

    Article  Google Scholar 

  15. Donoho D.: Compressed sensing. IEEE Trans. Info. Theory 52, 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  16. Faraut U., Korányi A.: Analysis on Symmetric Cones, Oxford Mathematical Monographs. Oxford University Press, New York (1994)

    Google Scholar 

  17. Fazel, M.: Matrix Rank Minimization with Applications. Ph.D. thesis, Stanford University (2002)

  18. Fazel, M., Hindi, H., Boyd, S.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the American Control Conference (2001)

  19. Fazel, M., Hindi, H., Boyd, S.: Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices. In: Proceedings of the American Control Conference (2003)

  20. Gafni E.H., Bertsekas D.P.: Two-metric projection methods for constrained optimization. SIAM J. Control Optim. 22, 936–964 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  21. Ghaoui L.E., Gahinet P.: Rank minimiztion under LMI constraints: a framework for output feedback problems. In: Proceedings of the European Control Conference (1993)

  22. Goldstein A.A.: Convex programming in Hilbert space. Bull. Amer. Math. Soc. 70, 709–710 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hiriart-Urruty J.-B., Lemaréchal C.: Convex Analysis and Minimization Algorithms, vols. 1 and 2. Springer, Berlin (1993)

    Google Scholar 

  24. Larsen, R.M.: PROPACK–Software for large and sparse SVD calculations. Available from http://sun.stanford.edu/~rmunk/PROPACK/

  25. Lemaréchal C., Sagastizábal C.: Practical aspects of the Moreau-Yosida regularization I: theoretical preliminaries. SIAM J. Optim. 7, 367–385 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  26. Levitin E.S., Polyak B.T.: Constrained minimization problems. USSR Comput. Math. Math. Phys. 6, 1–50 (1966)

    Article  Google Scholar 

  27. Linial N., London E., Rabinovich Y.: The geometry of graphs and some of its algorithmic applications. Combinatorica 15, 215–245 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  28. Liu Z., Vandenberghe L.: Interior-point method for nuclear norm approximation with application to system identification. SIAM J. Matrix Anal. Appl. 31, 1235–1256 (2009)

    Article  MathSciNet  Google Scholar 

  29. Lu F., Keles S., Wright S.J., Wahba G.: A framework for kernel regularization with application to protein clustering. Proc. Natl. Acad. Sci. 102, 12332–12337 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  30. Lu, Z., Monteiro, R.D.C., Yuan, M.: Convex optimization methods for dimension reduction and coefficient estimation in multivariate linear regression. preprint (2009)

  31. Ma, S.Q., Goldfarb, D., Chen, L.F.: Fixed point and Bregman iterative methods for matrix rank minimization. Math. Program. (to appear)

  32. Martinet B.: Régularisation d’inédquations variationelles par approximations successives. Rev. Francaise Inf. Rech. Oper. 4, 154–159 (1970)

    MathSciNet  Google Scholar 

  33. Moreau J.J.: Proximite et dualite dans un espace hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)

    MathSciNet  MATH  Google Scholar 

  34. Nemirovski A.: Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15, 229–251 (2005)

    Article  MathSciNet  Google Scholar 

  35. Nesterov Y.E.: A method for unconstrained convex minimization problem with the rate of convergence O(1/k 2). Doklady AN SSSR 269, 543–547 (1983)

    MathSciNet  Google Scholar 

  36. Nesterov Y.E.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer, Dordrecht (2004)

    MATH  Google Scholar 

  37. Nesterov Y.E.: Smooth minimization of nonsmooth functions. Math. Program. 103, 127–152 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  38. Nesterov Y.E.: Gradient Methods for Minimizing Composite Objective Function. Report, CORE, Catholic University of Louvain, Louvain-la-Neuve, Belgium (September 2007)

  39. Netflix Prize: http://www.netflixprize.com/

  40. Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. (to appear)

  41. Rockafellar R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    MATH  Google Scholar 

  42. Rockafellar R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  43. Rockafellar R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  44. Sturm J.F.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones. Optim. Methods Softw. 11(12), 625–653 (1999)

    Article  MathSciNet  Google Scholar 

  45. Toh, K.C., Yun, S.W.: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. (to appear)

  46. Tseng, P.: On Accelerated Proximal Gradient Methods for Convex-Concave Optimization. preprint, University of Washington (2008)

  47. Tütüncü R.H., Toh K.C., Todd M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95, 189–217 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  48. Yosida K.: Functional Analysis. Springer, Berlin (1964)

    Google Scholar 

  49. Yuan M., Ekici A., Lu Z., Monteiro R.D.C.: Dimension reduction and coefficient estimation in multivariate linear regression. J. R. Statist. Soc. B 69, 329–346 (2007)

    Article  MathSciNet  Google Scholar 

  50. Zhao X.Y., Sun D.F., Toh K.C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Science, Shenyang Aerospace University, 110136, Shenyang, People’s Republic of China

    Yong-Jin Liu

  2. Department of Mathematics and Risk Management Institute, National University of Singapore, 10 Lower Kent Ridge Road, Singapore, 119076, Singapore

    Defeng Sun

  3. Department of Mathematics, National University of Singapore, 10 Lower Kent Ridge Road, Singapore, 119076, Singapore

    Kim-Chuan Toh

  4. Singapore-MIT Alliance, 4 Engineering Drive 3, Singapore, 117576, Singapore

    Kim-Chuan Toh

Authors
  1. Yong-Jin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Defeng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kim-Chuan Toh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Defeng Sun.

Additional information

Part of this work was done while Y.-J. Liu was with the Singapore-MIT Alliance, 4 Engineering Drive 3, Singapore 117576. Y.-J. Liu’s research was supported in part by the National Young Natural Science Foundation of China under project grant No. 11001180. D. Sun’s research was supported in part by Academic Research Fund under grant R-146-000-104-112.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, YJ., Sun, D. & Toh, KC. An implementable proximal point algorithmic framework for nuclear norm minimization. Math. Program. 133, 399–436 (2012). https://doi.org/10.1007/s10107-010-0437-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-010-0437-8

Keywords

Mathematics Subject Classification (2000)

Search

Navigation