Skip to main content

Advertisement

SpringerLink
Log in

A partial proximal point algorithm for nuclear norm regularized matrix least squares problems

  • Full Length Paper
  • Published:
Mathematical Programming Computation Aims and scope Submit manuscript

Abstract

We introduce a partial proximal point algorithm for solving nuclear norm regularized matrix least squares problems with equality and inequality constraints. The inner subproblems, reformulated as a system of semismooth equations, are solved by an inexact smoothing Newton method, which is proved to be quadratically convergent under a constraint non-degeneracy condition, together with the strong semi-smoothness property of the singular value thresholding operator. Numerical experiments on a variety of problems including those arising from low-rank approximations of transition matrices show that our algorithm is efficient and robust.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Available at http://www.mathworks.com/moler/ncmfilelist.html.

  2. Available at: http://www.cs.toronto.edu/~tsap/experiments/datasets/index.html.

References

  1. Alfakih, A.Y., Khandani, A., Wolkowicz, H.: Solving Euclidean distance matrix completion problems via semidefinite programming. Comput. Optim. Appl. 12, 13–30 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  2. Ames, B., Vavasis, S.: Nuclear norm minimization for the planted clique and biclique problems. Math. Progr. 129, 69–89 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bai, Z., Freund, R.: A partial Padé-via-Lanczos method for reduced-order modeling. Linear Algebra Appl. 332–334, 139–164 (2001)

    Article  MathSciNet  Google Scholar 

  4. Bee, M.: Estimating rating transition probabilites with missing data. Stat. Methods Appl. 14, 127–141 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  5. Benczúr, A.A., Csalogány, K., Sarlós, T.: On the feasibility of low-rank approximation for personalized PageRank. In: Special interest tracks and posters of the 14th international conference on World Wide Web, pp. 972–973. ACM, New York (2005)

  6. Bertsekas, D., Tseng, P.: Partial proximal minimization algorithms for convex programming. SIAM J. Optim. 4, 551–572 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  7. Bhatia, R.: Matrix Analysis. Springer, New York (1997)

    Book  Google Scholar 

  8. Bonacich, P.: Factoring and weighting approaches to status scores and clique identification. J. Math. Sociol. 2, 113–120 (1972)

    Article  Google Scholar 

  9. Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20, 1956–1982 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  10. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  11. Chennubhotla, S.: Spectral methods for multi-scale feature extraction and data clustering, PhD thesis, University of Toronto, (2004)

  12. Chu, M., Funderlic, R., Plemmons, R.: Structured low rank approximation. Linear Algebra Appl. 366, 157–172 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  13. Bai, Z.-J., Chu, D., Tan, R.C.E.: Computing the nearest doubly stochastic matrix with a prescribed entry. SIAM J. Sci. Comput. 29, 635–655 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  14. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

    MATH  Google Scholar 

  15. Cottle, R.W., Pang, J.-S., Stone, R.E.: The Linear Complementarity Problem. Aacdemic Press, Boston (1992)

    MATH  Google Scholar 

  16. Dolezal, V.: Monotone operators and applications in Control and Network Theory, vol. 2. Elsevier Scientific Pub. Co., Amsterdam (1979)

  17. Eaves, B.C.: On the basic theorem of complementarity. Math. Progr. 1, 68–75 (1971)

    Article  MATH  MathSciNet  Google Scholar 

  18. Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrika 1, 211–218 (1936)

    Article  MATH  Google Scholar 

  19. Eckstein, J., Bertsekas, D.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Progr. 55, 293–318 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  20. Fazel, M.: Matrix Rank Minimization with Applications. PhD thesis, Stanford University (2002)

  21. Fazel, M., Hindi, H., Boyd, S.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the American Control Conference (2001)

  22. Fischer, A.: Solution of monotone complementarity problems with locally Lipschitzian functions. Math. Progr. 76, 513–532 (1997)

    MATH  Google Scholar 

  23. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  24. Gao, Y., Sun, D.F.: Calibrating least squares covariance matrix problems with equality and inequality constraints. SIAM J. Matrix Anal. Appl. 31, 1432–1457 (2009)

    Article  MathSciNet  Google Scholar 

  25. Gao, Y., Sun, D.F.: A majorized penalty approach for calibrating rank constrained correlation matrix problems (2010)

  26. Glowinski, R., Marrocco, A.: Sur l’approximation par éléments finis d’ordre un, et la résolution par pénalisation-dualité d’une classe de problèmes de Dirichlet nonlinéaires. Revue Française d’Automatique, Informatique, Recherche Opérationnelle 2, 41–76 (1975)

    MathSciNet  Google Scholar 

  27. Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)

    Book  MATH  Google Scholar 

  28. Golub, G.H., van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)

  29. Golub, G.H., Hoffman, A., Stewart, G.W.: A generalization of the Eckart–Young–Mirsky matrix approximation theorem. Linear Algebra Appl. 88, 317–327 (1987)

    Article  MathSciNet  Google Scholar 

  30. Ha, C.: A generalization of the proximal point algorithm. SIAM J. Control Optim. 28, 503–512 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  31. Ho, N.D., Van Dooren, P.: Non-negative matrix factorization with fixed row and column sums. Linear Algebra Appl. 429, 1020–1025 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  32. Ibaraki, S., Fukushima, M.: Partial proximal method of multipliers for convex programming problems. J Oper. Res. Soc. Japan-Keiei Kagaku 39, 213–229 (1996)

    MATH  MathSciNet  Google Scholar 

  33. Jiang K.F.: Algorithms for large scale nuclear norm minimization and convex quadratic semidefinite programming problems, PhD thesis, National University of Singapore, August (2011)

  34. Juan, K.: Lin. Reduced rank approximations of transition matrices. In: Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics (2003)

  35. Liu, Y.J. , Sun, D.F., Toh, K.C.: An implementable proximal point algorithmic framework for nuclear norm minimization. Math. Progr. (2010)

  36. Löwner, K.: Über monotone matrixfunktionen. Mathematische Zeitschrift 38, 177–216 (1934)

    Article  MathSciNet  Google Scholar 

  37. Luo, Y.: A smoothing Newton-BICGStab method for least squares matrix nuclear norm problems, Master thesis, National University of Singapore, January (2010)

  38. Ma, S., Goldfarb, D., Chen, L.: Fixed point and bregman iterative methods for matrix rank minimization. Math. Progr. 128, 321–353 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  39. Meng, F., Sun, D., Zhao, G.: Semismoothness of solutions to generalized equations and the Moreau–Yosida regularization. Math. Progr. 104, 561–581 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  40. Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 959–972 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  41. Mirsky, L.: Symmetric gauge functions and unitarily invariant norms. Q. J. Math. 11, 50–59 (1960)

    Article  MATH  MathSciNet  Google Scholar 

  42. Moreau, J.J.: Proximite et dualite dans un espace hilbertien. Bulletin de la Societe Mathematique de France 93, 273–299 (1965)

    MATH  MathSciNet  Google Scholar 

  43. Page, L., Brin, S., Motwani, R. Winograd, T.: The pagerank citation ranking: Bringing order to the web, Technical report, Stanford Digital Library Technologies Project. Available at http://dbpubs.stanford.edu/pub/1999-66 (1999)

  44. Pong, T.K., Tseng, P., Ji, S., Ye, J.: Trace norm regularization: reformulations, algorithms, and multi-task learning. SIAM J. Optim. 20, 3465–3489 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  45. Qi, H.D., Sun, D.F.: A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl. 28, 360–385 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  46. Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Progr. 58, 353–367 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  47. Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  48. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    MATH  Google Scholar 

  49. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  50. Rockafellar, R.T.: Augmented Lagrangains and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  51. Sinkhorn, R., Knopp, P.: Concerning nonnegative matrices and doubly stochastic matrices. Pac. J. Math. 21, 343–348 (1967)

    Article  MATH  MathSciNet  Google Scholar 

  52. Sturm, J.F.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones. Optim. Methods Softw. 11, 625–653 (1999)

    Article  MathSciNet  Google Scholar 

  53. Sun, D.F., Sun, J.: Semismooth matrix valued functions. Math. Oper. Res. 27, 150–169 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  54. Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3—a Matlab software package for semidefinite programming. Optim. Methods Softw. 11, 545–581 (1999)

    Article  MathSciNet  Google Scholar 

  55. Toh, K.C., Yun, S.W.: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. 6, 615–640 (2010)

    MATH  MathSciNet  Google Scholar 

  56. Van der Vorst, H.A.: SIAM J. Sci. Stat. Comput. Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems 13, 631–644 (1992)

    MATH  Google Scholar 

  57. Warga, J.: Fat homeomorphisms and unbounded derivate containers. J. Math. Anal. Appl. 81, 545–560 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  58. Watson, G.A.: Characterization of the subdifferential of some matrix norms. Linear Algebra Appl. 170, 33–45 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  59. Yosida, K.: Functional Analysis. Springer, Berlin (1964)

    Google Scholar 

  60. Zarantonello E.H.: Projections on convex sets in Hilbert space and spectral theory I and II. In: Zarantonello, E.H. (ed.) Contributions to Nonlinear Functional Analysis, pp. 237–424. Academic Press, New York (1971)

  61. Zhao, X.Y., Sun, D.F., Toh, K.C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, National University of Singapore, 10 Lower Kent Ridge Road, Singapore , 119076, Singapore

    Kaifeng Jiang & Kim-Chuan Toh

  2. Department of Mathematics and Risk Management Institute, National University of Singapore, 10 Lower Kent Ridge Road, Singapore , 119076, Singapore

    Defeng Sun

Authors
  1. Kaifeng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Defeng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kim-Chuan Toh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kim-Chuan Toh.

Additional information

Kim-Chuan Toh: Research support in part by Academic Research Fund under grant R-146-000-168-112.

Defeng Sun: Research supported in part by Academic Research Fund under grant R-146-000-149-112.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jiang, K., Sun, D. & Toh, KC. A partial proximal point algorithm for nuclear norm regularized matrix least squares problems. Math. Prog. Comp. 6, 281–325 (2014). https://doi.org/10.1007/s12532-014-0069-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12532-014-0069-8

Mathematics Subject Classification

Search

Navigation