Skip to main content
SpringerLink
Log in

Proximal gradient method for nonconvex and nonsmooth optimization on Hadamard manifolds

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

In this paper, we address the minimizing problem of the nonconvex and nonsmooth functions on Hadamard manifolds, and develop an improved proximal gradient method. First, by utilizing the geometric structure of non-positive curvature manifolds, we propose a monotone proximal gradient algorithm with fixed step size on Hadamard manifolds. Then, a convergence theorem of the proposed method has been established under the reasonable definition of proximal gradient mapping on manifolds. If the function further satisfies the Riemannian Kurdyka-Łojasiewicz (KL) property with an exponent, the local convergence rate is given. Finally, numerical experiments on a special Hadamard manifold, named symmetric positive definite matrix manifold, show the advantages of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2009)

    MATH  Google Scholar 

  2. Alimisis, F., Orvieto, A., Bécigneul, G., Lucchi, A.: Practical accelerated optimization on Riemannian manifolds. arXiv preprint arXiv:2002.04144 (2020)

  3. Ansari, Q.H., Babu, F., Yao, J.C.: Regularization of proximal point algorithms in Hadamard manifolds. J. Fixed Point Theory Appl. 21(1), 25 (2019)

    Article  MathSciNet  Google Scholar 

  4. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)

    Article  MathSciNet  Google Scholar 

  5. Axen, S.D., Baran, M., Bergmann, R., Rzecki, K.: Manifolds. jl: an extensible Julia framework for data analysis on manifolds. arXiv preprint arXiv:2106.08777 (2021)

  6. Bacák, M.: Convex analysis and optimization in Hadamard spaces, vol. 22. Walter de Gruyter GmbH, Berlin/Boston (2014)

  7. Beck, A.: First-order methods in optimization, vol. 25. SIAM, Philadelphia, (2017)

  8. Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18(11), 2419–2434 (2009)

    Article  MathSciNet  Google Scholar 

  9. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  10. Bento, G., Neto, J., Oliveira, P.: Convergence of inexact descent methods for nonconvex optimization on Riemannian manifolds. arXiv preprint arXiv:1103.4828 (2011)

  11. Bento, G.C., Ferreira, O.P., Melo, J.G.: Iteration-complexity of gradient, subgradient and proximal point methods on Riemannian manifolds. J. Optim. Theory Appl. 173(2), 548–562 (2017)

    Article  MathSciNet  Google Scholar 

  12. Bento, G.C., Ferreira, O.P., Oliveira, P.R.: Proximal point method for a special class of nonconvex functions on Hadamard manifolds. Optimization 64(2), 289–319 (2015)

    Article  MathSciNet  Google Scholar 

  13. Bhatia, R.: Positive Definite Matrices, vol. 24. Princeton University Press, Princeton (2009)

    Book  Google Scholar 

  14. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)

    Article  MathSciNet  Google Scholar 

  15. Boumal, N.: An introduction to optimization on smooth manifolds. Available online (2020). http://www.nicolasboumal.net/book

  16. Boumal, N., Mishra, B., Absil, P.A., Sepulchre, R.: Manopt, a Matlab toolbox for optimization on manifolds. J. Mach. Learn. Res 15, 1455–1459 (2014)

    MATH  Google Scholar 

  17. de Carvalho Bento, G., Bitar, S.D.B., da Cruz Neto, J.X., Oliveira, P.R., de OliveiraSouza, J.C.: Computing Riemannian center of mass on Hadamard manifolds. J. Optim. Theory Appl. 183(3), 977–992 (2019)

    Article  MathSciNet  Google Scholar 

  18. de Carvalho Bento, G., da Cruz Neto, J.X., Oliveira, P.R.: A new approach to the proximal point method: convergence on general Riemannian manifolds. J. Optim. Theory Appl. 168(3), 743–755 (2016)

    Article  MathSciNet  Google Scholar 

  19. Cheeger, J., Ebin, D.: Comparison theorems in Riemannian geometry. AMS Chelsea Publishing, Providence (2008)

  20. Chen, S., Ma, S., Man-Cho So, A., Zhang, T.: Proximal gradient method for nonsmooth optimization over the Stiefel manifold. SIAM J. Optim. 30(1), 210–239 (2020)

    Article  MathSciNet  Google Scholar 

  21. Colao, V., López, G., Marino, G., Martín-Márquez, V.: Equilibrium problems in Hadamard manifolds. J. Math. Anal. Appl. 388(1), 61–77 (2012)

    Article  MathSciNet  Google Scholar 

  22. Ferreira, O., Louzeiro, M., Prudente, L.: Gradient method for optimization on Riemannian manifolds with lower bounded curvature. SIAM J. Optim. 29(4), 2517–2541 (2019)

    Article  MathSciNet  Google Scholar 

  23. Ferreira, O.P., Oliveira, P.R.: Proximal point algorithm on Riemannian manifolds. Optimization 51(2), 257–270 (2002)

    Article  MathSciNet  Google Scholar 

  24. Frankel, P., Garrigos, G., Peypouquet, J.: Splitting methods with variable metric for kurdyka- lojasiewicz functions and general convergence rates. J. Optim. Theory Appl. 165(3), 874–900 (2015)

    Article  MathSciNet  Google Scholar 

  25. Gabay, D.: Minimizing a differentiable function over a differential manifold. J. Optim. Theory Appl. 37(2), 177–219 (1982)

    Article  MathSciNet  Google Scholar 

  26. Hosseini, S.: Convergence of nonsmooth descent methods via Kurdyka–Lojasiewicz inequality on Riemannian manifolds. Hausdorff Center for Mathematics and Institute for Numerical Simulation, University of Bonn (2015,(INS Preprint No. 1523)) (2015)

  27. Hosseini, S., Mordukhovich, B.S., Uschmajew, A.: Nonsmooth optimization and its applications. Springer, New York (2019)

    Book  Google Scholar 

  28. Hu, J., Liu, X., Wen, Z.W., Yuan, Y.X.: A brief introduction to manifold optimization. J. Oper. Res. Soc. China 8(2), 199–248 (2020)

    Article  MathSciNet  Google Scholar 

  29. Huang, W., Wei, K.: An extension of FISTA to Riemannian optimization for sparse PCA. arXiv preprint arXiv:1909.05485 (2019)

  30. Huang, W., Wei, K.: Riemannian proximal gradient methods. Math. Program. (2021). https://doi.org/10.1007/s10107-021-01632-3

  31. Kurdyka, K., Mostowski, T., Parusinski, A.: Proof of the gradient conjecture of R. Thom. Ann. Math. 152(3), 763–792 (2000)

    Article  MathSciNet  Google Scholar 

  32. Lang, S.: Fundamentals of differential geometry, vol. 191. Springer, New York (2012)

    MATH  Google Scholar 

  33. Ledyaev, Y., Zhu, Q.: Nonsmooth analysis on smooth manifolds. Trans. Am. Math. Soc. 359(8), 3687–3732 (2007)

    Article  MathSciNet  Google Scholar 

  34. Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. Adv. Neural information Process. Syst. 28, 379–387 (2015)

    Google Scholar 

  35. Lin, Z., Li, H., Fang, C.: Accelerated Optimization for Machine Learning. Springer, New York (2020)

    Book  Google Scholar 

  36. Liu, Y.Y., Shang, F.H., Cheng, J., Cheng, H., Jiao, L.: Accelerated first-order methods for geodesically convex optimization on Riemannian manifolds. In: Advances in Neural Information Processing Systems, pp. 4868–4877. ACM (2017)

  37. Luenberger, D.G.: The gradient projection method along geodesics. Manag. Sci. 18(11), 620–631 (1972)

    Article  MathSciNet  Google Scholar 

  38. Moakher, M., Zéraï, M.: The Riemannian geometry of the space of positive-definite matrices and its application to the regularization of positive-definite matrix-valued data. J. Math. Imag. Vis. 40(2), 171–187 (2011)

    Article  MathSciNet  Google Scholar 

  39. Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \({O}(1/k^{2})\). Dokl. akad. nauk Sssr 269, 543–547 (1983)

    MathSciNet  Google Scholar 

  40. Neto, J.X.C., Oliveira, P.R., Soares, P.A., Jr., Soubeyran, A.: Learning how to play Nash, potential games and alternating minimization method for structured nonconvex problems on Riemannian manifolds. J. Convex Anal. 20(2), 395–438 (2013)

    MathSciNet  MATH  Google Scholar 

  41. Pennec, X., Sommer, S., Fletcher, T.: Riemannian Geometric Statistics in Medical Image Analysis. Academic Press, USA (2019)

    MATH  Google Scholar 

  42. Pennec, X., Sommer, S., Fletcher, T.: Riemannian Geometric Statistics in Medical Image Analysis. Academic Press, San Diego (2019)

  43. Sakai, T.: Riemannian geometry, vol. 149. American Mathematical Society, Providence (1996)

  44. Tang, J.L., Liu, H.: Unsupervised feature selection for linked social media data. In: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 904–912. ACM (2012)

  45. Udriste, C.: Convex Functions and Optimization Methods on Riemannian Manifolds, vol. 297. Springer, New York (1994)

    Book  Google Scholar 

  46. Wang, J., Li, C., Lopez, G., Yao, J.C.: Convergence analysis of inexact proximal point algorithms on Hadamard manifolds. J. Glob. Optim. 61(3), 553–573 (2015)

    Article  MathSciNet  Google Scholar 

  47. Ying, S., Wen, Z., Shi, J., Peng, Y., Peng, J., Qiao, H.: Manifold preserving: an intrinsic approach for semisupervised distance metric learning. IEEE Trans. Neural Netw. Learn. Syst. 29(7), 2731–2742 (2018)

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, School of Sciences, Shanghai University, Shanghai, 200444, China

    Shuailing Feng, Lele Song & Shihui Ying

  2. School of Mathematical Sciences, Xiamen University, Xiamen, 361005, China

    Wen Huang

  3. Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong

    Tieyong Zeng

Authors
  1. Shuailing Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lele Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shihui Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tieyong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shihui Ying.

Additional information

This research is supported by the National Natural Science Foundation of China (11971296) and The Capacity Construction Project of Local Universities in Shanghai (18010500600)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, S., Huang, W., Song, L. et al. Proximal gradient method for nonconvex and nonsmooth optimization on Hadamard manifolds. Optim Lett 16, 2277–2297 (2022). https://doi.org/10.1007/s11590-021-01822-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-021-01822-0

Keywords

Search

Navigation