Skip to main content
SpringerLink
Log in

Acceleration techniques for level bundle methods in weakly smooth convex constrained optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We develop a unified level-bundle method, called accelerated constrained level-bundle (ACLB) algorithm, for solving constrained convex optimization problems. where the objective and constraint functions can be nonsmooth, weakly smooth, and/or smooth. ACLB employs Nesterov’s accelerated gradient technique, and hence retains the iteration complexity as that of existing bundle-type methods if the objective or one of the constraint functions is nonsmooth. More importantly, ACLB can significantly reduce iteration complexity when the objective and all constraints are (weakly) smooth. In addition, if the objective contains a nonsmooth component which can be written as a specific form of maximum, we show that the iteration complexity of this component can be much lower than that for general nonsmooth objective function. Numerical results demonstrate the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Within Sect. 1.2, we do not separate the objective function as in (1) and refer f to the entire objective function of the optimization.

References

  1. Bertsekas, D.P.: Nonlinear Programming. Athena Scientific Belmont (1999)

  2. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  Google Scholar 

  3. Chen, Y., Zhang, W.: Accelerated bundle level methods with inexact oracle. Sci. Sin. Math. 47(10), 1119–1142 (2017). (in Chinese)

    Article  Google Scholar 

  4. Chen, Y., Lan, G., Ouyang, Y., Zhang, W.: Fast bundle-level methods for unconstrained and ball-constrained convex optimization. Comput. Optim. Appl. 73(1), 159–199 (2019)

    Article  MathSciNet  Google Scholar 

  5. Cruz, J.Y.B., de Oliveira, W.: Level bundle-like algorithms for convex optimization. J. Global Optim. 59(4), 787–809 (2014)

    MathSciNet  MATH  Google Scholar 

  6. de Oliveira, W.: Target radius methods for nonsmooth convex optimization. Oper. Res. Lett. 45(6), 659–664 (2017)

    Article  MathSciNet  Google Scholar 

  7. de Oliveira, W., Sagastizábal, C.: Bundle methods in the XXIst century: a bird’s-eye view. Pesqui. Oper. 34(3), 647–670 (2014)

    Article  Google Scholar 

  8. Fábián, C.I.: Bundle-type methods for inexact data. CEJOR 8(1), 35–55 (2000)

    MathSciNet  MATH  Google Scholar 

  9. Fábián, C.I., Wolf, C., Koberstein, A., Suhl, L.: Risk-averse optimization in two-stage stochastic models: computational aspects and a study. SIAM J. Optim. 25(1), 28–52 (2015)

    Article  MathSciNet  Google Scholar 

  10. Jacob, L., Obozinski, G., Vert, J.-P.: Group lasso with overlap and graph lasso. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 433–440. ACM (2009)

  11. Karas, E., Ribeiro, A., Sagastizábal, C., Solodov, M.: A bundle-filter method for nonsmooth convex constrained optimization. Math. Program. 116(1), 297–320 (2009)

    Article  MathSciNet  Google Scholar 

  12. Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable minimization. Math. Program. 46(1), 105–122 (1990)

    Article  MathSciNet  Google Scholar 

  13. Kiwiel, K.C.: Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequalities. Math. Program. 69(1), 89–109 (1995)

    MathSciNet  MATH  Google Scholar 

  14. Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization, vol. 1133. Springer, Berlin (2006)

    MATH  Google Scholar 

  15. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    Article  MathSciNet  Google Scholar 

  16. Lan, G.: Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization. Math. Program. 149(1–2), 1–45 (2015)

    Article  MathSciNet  Google Scholar 

  17. Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69(1), 111–147 (1995)

    Article  MathSciNet  Google Scholar 

  18. Mairal, J., Jenatton, R., Obozinski, G., Bach, F.: Convex and network flow optimization for structured sparsity. J. Mach. Learn. Res. 12(Sep), 2681–2720 (2011)

    MathSciNet  MATH  Google Scholar 

  19. Nemirovsky, A., Yudin, D., Dawson, E.: Problem Complexity and Method Efficiency in Optimization. Wiley, London (1983)

    Google Scholar 

  20. Nesterov, Y.: A method for unconstrained convex minimization problem with the rate of convergence o (1/k\(^{2}\)). Dokl. SSSR 269, 543–547 (1983)

    Google Scholar 

  21. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer, Berlin (2004)

    Book  Google Scholar 

  22. Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)

    Article  MathSciNet  Google Scholar 

  23. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer, Berlin (2013)

    MATH  Google Scholar 

  24. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1), 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  25. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused lasso. J. R. Stat. Soc.: Ser. B (Statistical Methodology) 67(1), 91–108 (2005)

    Article  MathSciNet  Google Scholar 

  26. Tomioka, R., Suzuki, T., Hayashi, K., Kashima, H.: Statistical performance of convex tensor decomposition. In: Advances in Neural Information Processing Systems, pp. 972–980 (2011)

  27. van Ackooij, W.: Level bundle methods for constrained convex optimization with various oracles. Comput. Optim. Appl. 57(3), 555–597 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This research was partially supported by NSF grants DMS-1319050, DMS-1620342, DMS-1719932, CMMI-1745382, DMS-1818886 and DMS-1925263.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Florida, Gainesville, FL, 32611, USA

    Yunmei Chen & Wei Zhang

  2. Department of Mathematics and Statistics, George State University, Atlanta, GA, 30303, USA

    Xiaojing Ye

Authors
  1. Yunmei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaojing Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Ye, X. & Zhang, W. Acceleration techniques for level bundle methods in weakly smooth convex constrained optimization. Comput Optim Appl 77, 411–432 (2020). https://doi.org/10.1007/s10589-020-00208-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-020-00208-9

Keywords

Search

Navigation