Skip to main content
SpringerLink
Log in

Some accelerated alternating proximal gradient algorithms for a class of nonconvex nonsmooth problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, we study a class of nonconvex and nonsmooth optimization problems, whose objective function can be split into two separable terms and one coupling term. Alternating proximal gradient methods combining with extrapolation are proposed to solve such problems. Under some assumptions, we prove that every cluster point of the sequence generated by our algorithms is a critical point. Furthermore, if the objective function satisfies Kurdyka–Łojasiewicz property, the generated sequence is globally convergent to a critical point. In order to make the algorithm more effective and flexible, we also use some strategies to update the extrapolation parameter and solve the problems with unknown Lipschitz constant. Numerical experiments demonstrate the effectiveness of our algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets.

References

  1. Paatero, P., Tapper, U.: Positive matrix factorization: a nonnegative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111–126 (1994)

    Article  Google Scholar 

  2. Lee, D.D., Seung, H.S.: Learning the parts of objects by nonnegative matrix factorization. Nature 401, 788–791 (1999)

    Article  MATH  Google Scholar 

  3. Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding and evaluating blind deconvolution algorithms. In: Proceedings of Computer Vision and Patter Recognition (CVPR) (2009)

  4. Zhang, X., Zhang, X., Li, X., Li, Z., Wang, S.: Classify social image by integrating multimodal content multimed. Tools Appl. 77, 7469–7485 (2017)

    Article  Google Scholar 

  5. Bugeau, A., Ta, V.T., Papadakis, N.: Variational exemplar-based image colorization. IEEE Trans. Image Process. 23, 298–307 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Pierre, F., Aujol, J.F., Bugeau, A., Papadakis, N., Ta, V.T.: Luminance-chrominance model for image colorization. SIAM J. Imaging Sci. 8, 536–563 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bertsekas, D.P.: Nolinear Programming. Athena Scientific, Belmont (1995)

    Google Scholar 

  8. Auslender, A.: Asymptotic properties of the Fenchel dual functional and applications to decomposition problems. J. Optim. Theory Appl. 73, 427–499 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  9. Attouch, H., Redont, P., Soubeyran, A.: A new class of alternating proximal minimization algorithms with costs-to-move. SIAM J. Optim. 18, 1061–1081 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  10. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  11. Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6, 1758–1789 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  12. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. Ser. B 116, 5–16 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  13. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  14. Pock, T., Sabach, S.: Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems. SIAM J. Imaging Sci. 9, 1756–1787 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gao, X., Cai, X., Han, D.: A Gauss-Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems. J. Global Optim. 76, 863–887 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  16. Nikolova, M., Tan, P.: Alternating structure-adapted proximal gradient descent for nonconvex nonsmooth block-regularized problems. SIAM J. Optim. 29, 2053–2078 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  17. Tan, P., Pierre, F., Nikolova, M.: Inertial alternating generalized Forward-Backward splitting for image colorization. J. Math. Imaging Vis. 61, 672–690 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  18. Rockafellar, R.T., Wets, J.B.: Variational Analysis. Springer, New York (1998)

    Book  MATH  Google Scholar 

  19. Nesterov, Y.: Lectures on Convex Optimization. Springer, New York (2018)

    Book  MATH  Google Scholar 

  20. Bolte, J., Daniilidis, A., Lewis, A.: The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17, 1205–1223 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  21. Nesterov, Y.: A method of solving a convex programming problem with convergence rate \(O( \frac{1}{k^2 }\)). Sov. Math. Dokl. 27, 372–376 (1983)

    Google Scholar 

  22. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  23. Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. In: Advances in Neural Information Processing Systems, pp. 379–387 (2015)

  24. Li, Q., Zhou, Y., Liang, Y., Varshney, P.K.: Convergence analysis of proximal gradient with momentum for nonconvex optimization. In: Proceedings of the 34th International Conference on Machine Learning, 2111-2119 (2017)

  25. Wu, Z., Li, M.: General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems. Comput. Optim. Appl. 73, 129–158 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  26. Gong, P., Zhang, C., Lu, Z., Huang, J., Ye, J.: A general iterative shrinkage and thresholding algorithm for nonconvex regularized optimization problems. In: International Conference on Leadership and Management, 37–45 (2013)

  27. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  28. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137, 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  29. Ban, G.Y., Karoui, N.E., Lim, A.E.B.: Machine learning and portfolio optimization. Manag. Sci. 64, 1136–1154 (2016)

    Article  Google Scholar 

  30. Chen, X., Peng, J., Zhang, S.: Sparse solutions to random standard quadratic optimization problems. Math. Program. 141, 273–293 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. Genkin, A., Lewis, D.D., Madigan, D.: Large-scale Bayesian logistic regression for text categorization. Technometrics 49, 291–304 (2007)

    Article  MathSciNet  Google Scholar 

  32. Shevade, S.K., Keerthi, S.S.: A simple and efficient algorithm for gene selection using sparse logistic regression. Bioinformatics 19, 2246–2253 (2003)

    Article  Google Scholar 

  33. Fridman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Ann. Stat. 28, 337–407 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  34. Zhang, T.: Analysis of multi-stage convex relaxation for sparse regularization. J. Mach. Learn. Res. 11, 1081–1107 (2010)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Zhejiang University, Hangzhou, 310027, China

    Xin Yang

  2. School of Mathematical Sciences, Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing, 210023, China

    Lingling Xu

Authors
  1. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingling Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lingling Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The work is supported by Grant National Natural Science Foundation of China (NSFC) 11971238.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Xu, L. Some accelerated alternating proximal gradient algorithms for a class of nonconvex nonsmooth problems. J Glob Optim 87, 939–964 (2023). https://doi.org/10.1007/s10898-022-01214-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-022-01214-3

Keywords

Search

Navigation