Skip to main content
SpringerLink
Log in

Convergence rate analysis of an extrapolated proximal difference-of-convex algorithm

  • Original Research
  • Published:
Journal of Applied Mathematics and Computing Aims and scope Submit manuscript

Abstract

This paper considers a class of difference-of-convex (DC) optimization problems, whose objective function is the sum of a convex smooth function and a possibly nonsmooth DC function. We first propose a new extrapolated proximal difference-of-convex algorithm, which incorporates a more general setting of the extrapolation parameters \(\{\beta _k\}\). Then we prove the subsequential convergence of the proposed method to a stationary point of the DC problem. Based on the Kurdyka–Łojasiewicz inequality, the global convergence and convergence rate of the whole sequence generated by our method have been established. Finally, some numerical experiments on the DC regularized least squares problems have been performed to demonstrate the efficiency of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Alvarado, A., Scutari, G., Pang, J.S.: A new decomposition method for multiuser DC programming and its applications. IEEE Trans. Signal Process. 62, 2984–2998 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions invoving analytic features. Math. Program. 116, 5–16 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Łojasiewicz inequality. Math. Oper. Res. 3, 438–457 (2010)

    Article  MATH  Google Scholar 

  4. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137, 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Candès, E.J., Wakin, M., Boyd, S.: Enhancing spasity by reweighted \(\ell _{1}\) minimization. J. Fourier Anal. Appl. 14, 877–905 (2008)

  7. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chambolle, A., Dossal, C.: On the convergence of the iterates of “FISTA’’. J. Optim. Theory Appl. 166, 25 (2015)

    MATH  Google Scholar 

  9. Gaso, G., Rakotomamonjy, A., Canu, S.: Recovering sparse signals with a certain family of nonconvex penalties and DC programming. IEEE Trans. Signal Process. 57, 4686–4698 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  10. Gong, P., Zhang, C., Lu, Z., Huang, J.Z., Ye, J.: A general iterative shinkage and thresholding algorithm for non-convex regularized optimization problems. In: ICML (2013)

  11. Gotoh, J., Takeda, A., Tono, K.: DC formulations and algorithms for sparse optimization problems. Preprint, METR 2015-27, Department of Mathematical Informatics, University of Tokyo. http://www.keisu.t.u-tokyo.ac.jp/research/techrep/index.html

  12. Le Thi, H.A., Nguyen, M.C.: DCA based algorithms for feature selection in multi-class support vector machine. Ann. Oper. Res. 249, 273–300 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Le Thi, H.A., Pham, D.T.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133, 23–46 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  14. Le Thi, H.A., Pham, D.T.: DC programming and DCA: thirty years of developments. Math. Program. 169, 5–68 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  15. Le Thi, H.A., Pham, D.T., Le, H.M., Vo, X.Y.: DC approximation approaches for sparse optimization. Eur. J. Oper. Res. 244, 26–46 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  16. Lin, Y., Li, S., Zhang, Y.Z.: Convergence rate analysis of accelerated forward–backward algorithm with generalized Nesterov momentum scheme. arXiv: 2112.05873

  17. Lin, Y., Schmidtlein, C.R., Li, Q., Li, S., Xu, Y.: A Krasnoselskii–Mann algorithm with an improved EM preconditioner for PET image reconstruction. IEEE Trans. Med. Imaging 38, 2114–2126 (2019)

    Article  Google Scholar 

  18. Liu, T., Pong, T.K.: Further properties of the forward–backward envelope with applications to difference-of-convex programming. Comput. Optim. Appl. 67, 489–520 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  19. Liu, T., Pong, T.K., Takeda, A.: A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems. Math. Program. 176, 339–367 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  20. Liu, T., Pong, T.K., Takeda, A.: A refined convergence analysis of pDCA\(_e\) with applications to simultaneous sparse recovery and outlier detection. Math. Program. 176, 339–367 (2019)

  21. Lou, Y., Zeng, T., Osher, S., Xin, J.: A weighted difference of anisotropic and isotropic total variation model for image processing. SIAM J. Imaging Sci. 8, 1798–1823 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. Lu, Z., Zhou, Z., Sun, Z.: Enhanced proximal DC algorithms with extrapolation for a class of structured nonsmooth DC minimization. Math. Program. 176, 369–401 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  23. Lu, Z., Zhou, Z.: Nonmonotone enhanced proximal DC algorithms for a class of structured nonsmooth DC programming. SIAM J. Optim. 29, 2725–2752 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  24. O’Donoghue, B., Candès, E.J.: Adaptive restart for accelerated gradient schemes. Found. Comput. Math. 15, 715–732 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Pang, J.-S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42, 95–118 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  26. Pham, D.T., Le Thi, H.A.: Convex analysis approach to D.C. programming: theory, algorithms and applications. Acta Math. Vietnam. 22, 289–355 (1997)

    MathSciNet  MATH  Google Scholar 

  27. Pham, D.T., Le Thi, H.A.: A D.C. optimization algorithm for solving the trust-region subproblem. SIAM J. Optim. 8, 476–505 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  28. Rockafellar, R.T., Wets, R.: Variational Analysis. Grundlehren der Mathematischen Wissenschaften. Springer, Berlin (1998)

    Google Scholar 

  29. Sanjabi, M., Razaviyayn, M., Luo, Z.-Q.: Optimal joint base station assignment and beamforming for heterogeneous networks. IEEE Trans. Signal Process. 62, 1950–1961 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  30. Sun, K., Sun, X.A.: Algorithms for difference-of-convex (DC) programs based on difference-of-Moreau-envelopes smoothing. arXiv: 2104.01470

  31. Wen, B., Chen, X., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 62, 297–324 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  32. Yin, P., Lou, Y., He, Q., Xin, J.: Minimization of \(\ell _{1-2}\) for compressed sensing. SIAM J. Sci. Comput. 37, A536–A563 (2015)

Download references

Acknowledgements

The authors would like to thank the editors and anonymous reviewers for their insight and helpful comments and suggestions which improve the quality of the paper. This work is also supported in part by NSFC11801131, Natural Science Foundation of Hebei Province (Grant No. A2019202229), a scientific grant of Hebei Educational Committee (QN2018101).

Author information

Authors and Affiliations

  1. School of Science, Hebei University of Technology, Tianjin, People’s Republic of China

    Lejia Gao

  2. Institute of Mathematics, Hebei University of Technology, Tianjin, People’s Republic of China

    Bo Wen

Authors
  1. Lejia Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Wen.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest with regards to the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This author’s work was supported in part by NSFC11801131, Natural Science Foundation of Hebei Province, (Grant No. A2019202229), a scientific grant of Hebei Educational Committee (QN2018101)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, L., Wen, B. Convergence rate analysis of an extrapolated proximal difference-of-convex algorithm. J. Appl. Math. Comput. 69, 1403–1429 (2023). https://doi.org/10.1007/s12190-022-01797-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12190-022-01797-w

Keywords

Mathematics Subject Classification

Search

Navigation