Skip to main content

Advertisement

Springer Nature Link
Log in

A Riemannian subspace BFGS trust region method

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

Motivated by the subspace techniques in the Euclidean space, this paper presents a subspace BFGS trust region (RTR-SBFGS) algorithm to the problem of minimizing a smooth function defined on Riemannian manifolds. In each iteration of the RTR-SBFGS algorithm, a low-dimensional trust region subproblem is solved, which reduces the amount of computation significantly for large scale problems. A limited-memory variant of RTR-SBFGS, named LRTR-SBFGS, is introduced also. Both RTR-SBFGS and LRTR-SBFGS are proved to converge globally. Under some mild conditions, we establish the local linear convergence of these two methods. Numerical results demonstrate that, compared to the state-of-the-art algorithms, RTR-SBFGS and LRTR-SBFGS are effective methods and subspace techniques are suitable for Riemannian optimization problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7, 303–330 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  2. Absil, P.-A., Baker, C.G., Gallivan, K.A.: Accelerated line-search and trust-region methods. SIAM J. Numer. Anal. 47, 997–1018 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)

    Book  MATH  Google Scholar 

  4. Baker, C.G., Absil, P.-A., Gallivan, K.A.: An implicit trust-region method on Riemannian manifolds. IMA J. Numer. Anal. 28, 665–689 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  5. Boumal, N.: An introduction to optimization on smooth manifolds. Cambridge University Press, Cambridge (2022)

    MATH  Google Scholar 

  6. Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: Manopt, a Matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014)

    MATH  Google Scholar 

  7. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its application to quasiNewton methods. Math. Comput. 28, 549–560 (1974)

    Article  MATH  Google Scholar 

  8. do Carmo, M. P.: Riemannian Geometry. Translated from the second Portuguese edition by Francis Flaherty. Mathematics: Theory & Applications. Birkh\(\ddot{a}\)user Boston Inc., Boston, MA (1992)

  9. Gill, P.E., Leonard, M.W.: Reduced-Hessian quasi-Newton methods for unconstrained optimization. SIAM J. Optim. 12, 209–237 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. Huang, W.: Optimization Algorithms on Riemannian Manifolds with Applications. Florida State University, Florida (2013)

    Google Scholar 

  11. Huang, W., Absil, P.-A., Gallivan, K.A.: A Riemannian symmetric rank-one trust-region method. Math. Program. 150, 179–216 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Hosseini, S., Uschmajew, A.: A Riemannian gradient sampling algorithm for nonsmooth optimization on manifolds. SIAM J. Optim. 27, 173–189 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Nocedal, J., Wright, S. J.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering, Springer, New York, second edition, (2006)

  14. Sato, H.: Riemannian Optimization and Its Applications. Springer, Berlin (2021)

    Book  MATH  Google Scholar 

  15. Sato, H.: Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analyses. SIAM J. Optim. 32, 2690–2717 (2021)

  16. Siegel, D.: Implementing and modifying Broyden class updates for large scale optimization. ReportDAMPT 1992/NA12, University of Cambridge, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, England (1992)

  17. Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23, 1214–1236 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  18. Wang, X., Ma, S., Goldfarb, D., Liu, W.: Stochastic quasi-newton methods for nonconvex stochastic optimization. SIAM J. Optim. 27, 927–956 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  19. Wang, Z., Yuan, Y.: A subspace implementation of quasi-Newton trust region methods for unconstrained optimization. Numer. Math. 104, 241–269 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  20. Wei, H., Yang, W.H.: A Riemannian subspace limited-memory SR1 trust region method. Optim. Lett. 10, 1705–1723 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  21. Zhu, X., Sato, H.: Riemannian conjugate gradient methods with inverse retraction. Comput. Optim. Appl. 77, 779–810 (2020)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The work of Wei Hong Yang was supported by the National Natural Science Foundation of China grant 11971118. The authors are grateful to the associate editor and the two anonymous referees for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. School of Statistics and Mathematics, Shanghai Lixin University of Accounting and Finance, 995 Shangchuan Road, Shanghai, 201209, China

    Hejie Wei

  2. School of Mathematical Sciences, Fudan University, Shanghai, 200433, China

    Wei Hong Yang & Yinsheng Chai

Authors
  1. Hejie Wei
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Wei Hong Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yinsheng Chai
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Hejie Wei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, H., Yang, W.H. & Chai, Y. A Riemannian subspace BFGS trust region method. Optim Lett 17, 1889–1914 (2023). https://doi.org/10.1007/s11590-022-01964-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-022-01964-9

Keywords