Skip to main content
SpringerLink
Log in

A Riemannian symmetric rank-one trust-region method

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

The well-known symmetric rank-one trust-region method—where the Hessian approximation is generated by the symmetric rank-one update—is generalized to the problem of minimizing a real-valued function over a \(d\)-dimensional Riemannian manifold. The generalization relies on basic differential-geometric concepts, such as tangent spaces, Riemannian metrics, and the Riemannian gradient, as well as on the more recent notions of (first-order) retraction and vector transport. The new method, called RTR-SR1, is shown to converge globally and \(d+1\)-step q-superlinearly to stationary points of the objective function. A limited-memory version, referred to as LRTR-SR1, is also introduced. In this context, novel efficient strategies are presented to construct a vector transport on a submanifold of a Euclidean space. Numerical experiments—Rayleigh quotient minimization on the sphere and a joint diagonalization problem on the Stiefel manifold—illustrate the value of the new methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Absil, P.A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  2. Absil, P.A., Mahony, R., Sepulchre, R.: Optimization algorithms on matrix manifolds. Princeton University Press, Princeton, NJ (2008)

    Book  MATH  Google Scholar 

  3. Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM Journal on Optimization 22(1), 135–158 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  4. Adler, R.L., Dedieu, J.P., Margulies, J.Y.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  5. Afsari, B., Tron, R., Vidal, R.: On the convergence of gradient descent for finding the Riemannian center of mass. SIAM J. Control Optim. 51(3), 2230–2260 (2013). ArXiv:1201.0925v1

    Article  MATH  MathSciNet  Google Scholar 

  6. Boothby, W.M.: An Introduction to Differentiable Manifolds and Riemannian Geometry, 2nd edn. Academic Press, London (2003)

    Google Scholar 

  7. Borsdorf, R.: Structured Matrix Nearness Problems: Theory and Algorithms. Ph.D. thesis, The University of Manchester (2012)

  8. Boumal, N., Absil, P.A.: RTRMC: A Riemannian trust-region method for low-rank matrix completion. Adv. Neural Inf. Process. Syst. 24(NIPS), 406–414 (2011)

    Google Scholar 

  9. Boumal, N., Mishra, B.: The Manopt toolbox. http://www.manopt.org

  10. Brace, I., Manton, J.H.: An improved BFGS-on-manifold algorithm for computing weighted low rank approximations. In Proceedings of 17th International Symposium on Mathematical Theory of Networks and Systems, pp. 1735–1738 (2006)

  11. Byrd, R.H., Khalfan, H.F., Schnabel, R.B.: Analysis of a symmetric rank-one trust region method. SIAM J. Optim. 6(4), 1025–1039 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  12. Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63(1–3), 129–156 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  13. do Carmo, M.P.: Riemannian geometry. Mathematics: Theory & Applications (1992)

  14. Chavel, I.: Riemannian Geometry: A Modern Introduction, 2nd ed. Cambridge Studies in Advanced Mathematics (2006)

  15. Conn, A.R., Gould, N.I.M., Toint, P.L.: Convergence of quasi-Newton matrices generated by the symmetric rank one update. Math. Program. 50(1–3), 177–195 (1991). doi:10.1007/BF01594934

    Article  MATH  MathSciNet  Google Scholar 

  16. Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-region methods. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, MPS/SIAM Series on Optimization (2000)

  17. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998). doi:10.1137/S0895479895290954

    Article  MATH  MathSciNet  Google Scholar 

  18. Gabay, D.: Minimizing a differentiable function over a differential manifold. J. Optim. Theory Appl. 37(2), 177–219 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  19. Gallivan, K.A., Qi, C., Absil, P.A.: A Riemannian Dennis-Moré condition. In: Berry, M.W., Gallivan, K.A., Gallopoulos, E., Grama, A., Philippe, B., Saad, Y., Saied, F. (eds.) High-Performance Scientific Computing, pp. 281–293. Springer, London (2012)

    Chapter  Google Scholar 

  20. Huang, W.: Optimization algorithms on Riemannian manifolds with applications. Ph.D. thesis, Department of Mathematics, Florida State University (2013)

  21. Ishteva, M., Absil, P.A., Van Huffel, S., De Lathauwer, L.: Best low multilinear rank approximation of higher-order tensors, based on the Riemannian trust-region scheme. SIAM J. Matrix Anal. Appl. 32(1), 115–135 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  22. Jiang, B., Dai, Y.H.: A framework of constraint preserving update schemes for optimization on the Stiefel manifold (2013). ArXiv:1301.0172

  23. Journée, M., Bach, F., Absil, P.A., Sepulchre, R.: Low-rank optimization on the cone of positive semidefinite matrices. SIAM J. Optim. 20(5), 2327–2351 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  24. Khalfan, H.F., Byrd, R.H., Schnabel, R.B.: A theoretical and experimental study of the symmetric rank-one update. SIAM J. Optim. 3(1), 1–24 (1993). doi:10.1137/0803001

    Article  MATH  MathSciNet  Google Scholar 

  25. Kleinsteuber, M., Shen, H.: Blind source separation with compressively sensed linear mixtures. IEEE Signal Process. Lett. 19(2), 107–110 (2012). ArXiv:1110.2593v1

    Article  Google Scholar 

  26. Mishra, B., Meyer, G., Bach, F., Sepulchre, R.: Low-rank optimization with trace norm penalty (2011). ArXiv:1112.2318v2

  27. Nocedal, J., Wrigh, S.J.: Numerical Optimization, 2nd edn. Springer, Berlin (2006)

    MATH  Google Scholar 

  28. O’Neill, B.: Semi-Riemannian geometry. Academic Press Incorporated [Harcourt Brace Jovanovich Publishers] (1983)

  29. Qi, C.: NuMerical OptiMization Methods on RieMannian Manifolds. Ph.D. thesis, DepartMent of Mathematics, Florida State University (2011)

  30. Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012). doi:10.1137/11082885X

    Article  MATH  MathSciNet  Google Scholar 

  31. Sato, H., Iwai, T.: Convergence analysis for the Riemannian conjugate gradient method (2013). ArXiv:1302.0125v1

  32. Savas, B., Lim, L.H.: Quasi-Newton methods on Grassmannians and multilinear approximations of tensors. SIAM J. Sci. Comput. 32(6), 3352–3393 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  33. Seibert, M., Kleinsteuber, M., Hüper, K.: Properties of the BFGS method on Riemannian manifolds. Mathematical System Theory C Festschrift in Honor of Uwe Helmke on the Occasion of his Sixtieth Birthday pp. 395–412 (2013)

  34. Selvan, S.E., Amato, U., Gallivan, K.A., Qi, C.: Descent algorithms on oblique manifold for source-adaptive ICA contrast. IEEE Trans. Neural Netw. Learn. Syst. 23(12), 1930–1947 (2012)

    Article  Google Scholar 

  35. Stiefel, E.: Richtungsfelder und fernparallelismus in n-dimensionalen mannigfaltigkeiten. Commentarii Mathematici Helvetici 8(1), 305–353 (1935). doi:10.1007/BF01199559

    Article  MathSciNet  Google Scholar 

  36. Theis, F.J., Cason, T.P., Absil, P.A.: Soft dimension reduction for ICA by joint diagonalization on the Stiefel manifold. In: Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation, vol. 5441, pp. 354–361 (2009)

  37. Turaga, P., Veeraraghavan, A., Srivastava, A., Chellappa, R.: Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(11), 2273–86 (2011). doi:10.1109/TPAMI.2011.52

    Article  Google Scholar 

  38. Vandereycken, B., Vandewalle, S.: A Riemannian optimization approach for computing low-rank solutions of Lyapunov equations. SIAM J. Matrix Anal. Appl. 31(5), 2553–2579 (2010). doi:10.1137/090764566

    Article  MATH  MathSciNet  Google Scholar 

  39. Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Mathematical Programming, Published online (2012). doi:10.1007/s10107-012-0584-1

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Florida State University, 208 Love Building, 1017 Academic Way, Tallahassee, FL, 32306-4510, USA

    Wen Huang & K. A. Gallivan

  2. Department of Mathematical Engineering, ICTEAM Institute, Université catholique de Louvain, 1348 , Louvain-la-Neuve, Belgium

    P.-A. Absil

Authors
  1. Wen Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P.-A. Absil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. A. Gallivan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P.-A. Absil.

Additional information

This paper presents research results of the Belgian Network DYSCO (Dynamical Systems, Control, and Optimization), funded by the Interuniversity Attraction Poles Programme initiated by the Belgian Science Policy Office. This work was financially supported by the Belgian FRFC (Fonds de la Recherche Fondamentale Collective). This work was performed in part while the third author was a Visiting Professor at the Institut de mathématiques pures et appliquées (MAPA) at Université catholique de Louvain.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huang, W., Absil, PA. & Gallivan, K.A. A Riemannian symmetric rank-one trust-region method. Math. Program. 150, 179–216 (2015). https://doi.org/10.1007/s10107-014-0765-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-014-0765-1

Keywords

Mathematics Subject Classification (2000)

Search

Navigation