Skip to main content
SpringerLink
Log in

A simple version of bundle method with linear programming

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Bundle methods have been well studied in nonsmooth optimization. In most of the bundle methods developed thus far (traditional bundle methods), at least one quadratic programming subproblem needs to be solved in each iteration. In this paper, a simple version of bundle method with linear programming is proposed. In each iteration, a cutting-plane model subject to a constraint constructed by an infinity norm is minimized. Without line search or trust region techniques, the convergence of the method can be shown. Additionally, the infinity norm in the constraint can be generalized to \(p\)-norm. Preliminary numerical experiments show the potential advantage of the proposed method for solving large scale problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andreani, R., Martinez, J.M., Schuverdt, M.L.: On the relation between constant positive linear dependence condition and quasinormality constraint qualification. J. Optim. Theory Appl. 125(2), 473–483 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  2. Astorino, A., Frangioni, A., Gaudioso, M., Gorgone, E.: Piecewise-quadratic approximations in convex numerical optimization. SIAM J. Optim. 21(4), 1418–1438 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bagirov, A.M., Karasözen, B., Sezer, M.: Discrete gradient method: derivative-free method for nonsmooth optimization. J. Optim. Theory Appl. 137(2), 317–334 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bonnans, J.F., Gilbert, J.C., Lemarechal, C., Sagastizábal, C.A.: Numerical Optimization: Theoretical and Practical Aspects. Universitext. Springer, Berlin (2006)

    MATH  Google Scholar 

  5. Cheney, E.W., Goldstein, A.A.: Newton’s method for convex programming and tchebycheff approximation. Numer. Math. 1(1), 253–268 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  6. Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(1–3), 261–275 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  7. de Oliveira, W.: Regularized optimization methods for convex minlp problems. TOP 24(3), 665–692 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex proximal bundle methods in depth: a unified analysis for inexact oracles. Math. Program. 148(1), 241–277 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  9. Frangioni, A.: Generalized bundle methods. SIAM J. Optim. 13(1), 117–156 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  10. Fuduli, A., Gaudioso, M., Nurminski, E.A.: A splitting bundle approach for non-smooth non-convex minimization. Optimization 64(5), 1131–1151 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Haarala, M., Miettinen, K., Mäkelä, M.M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. 19(6), 673–692 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  12. Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hare, W., Sagastizábal, C., Solodov, M.: A proximal bundle method for nonsmooth nonconvex functions with inexact information. Comput. Optim. Appl. 63(1), 1–28 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II, Grundlehren Der Mathematischen Wissenschaften, vol. 306. Springer, Berlin (1993)

    MATH  Google Scholar 

  15. Kelley Jr., J.E.: The cutting-plane method for solving convex programs. J. Soc. Ind. Appl. Math. 8, 703–712 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  16. Kim, S., Chang, K.N., Lee, J.Y.: A descent method with linear programming subproblems for nondifferentiable convex optimization. Math. Program. 71(1), 17–28 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  17. Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Springer, Berlin (1985)

    Book  MATH  Google Scholar 

  18. Lemaréchal, C.: An extension of davidon methods to non differentiable problems. In: Balinski, M.L., Wolfe, P. (eds.) Nondifferentiable Optimization, Mathematical Programming Studies, vol. 3, pp. 95–109. Springer, Berlin (1975)

    Chapter  Google Scholar 

  19. Lemaréchal, C.: Bundle methods in nonsmooth optimization. In: Lemaréchal, C., Mifflin, R. (eds.) Nonsmooth Optimization: Proceedings of a IIASA Workshop, March 28-April 8, 1977, Iiasa Proceedings Series, vol. 3, pp. 79–102. International Institute for Applied Systems Analysis, Pergamon Press (1978)

  20. Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69(1), 111–147 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  21. Linderoth, J., Wright, S.: Decomposition algorithms for stochastic programming on a computational grid. Comput. Optim. Appl. 24(2–3), 207–250 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  22. Lukšan, L., Vlček, J.: A bundle-newton method for nonsmooth unconstrained minimization. Math. Program. 83(3, Ser.A), 373–391 (1998)

    MathSciNet  MATH  Google Scholar 

  23. Lukšan, L., Vlček, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Tech. Rep. 798, Institute of Computer Science of Academy of Sciences of the Czech Republic (2000)

  24. Makela, M.M., Neittaanmaki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific, Singapore (1992)

    Book  MATH  Google Scholar 

  25. Mifflin, R.: A modification and an extension of lemarechal’s algorithm for nonsmooth minimization. In: Sorensen, D.C., Wets, RJb (eds.) Nondifferential and Variational Techniques in Optimization, Mathematical Programming Studies, vol. 17, pp. 77–90. Springer, Berlin (1982)

    Chapter  Google Scholar 

  26. Mifflin, R., Sagastizábal, C.: A science fiction story in nonsmooth optimization originating at iiasa. this volume (2012)

  27. Noll, D., Prot, O., Rondepierre, A.: A proximity control algorithm to minimize nonsmooth and nonconvex functions. Pacific J. Optim. 4(3), 569–602 (2008)

    MathSciNet  MATH  Google Scholar 

  28. Rockafellar, R.T., Wets, R.J.B.: Variational Analysis: Grundlehren Der Mathematischen Wissenschaften, vol. 317. Springer, Berlin (1998)

    Book  Google Scholar 

  29. Ruszczyński, A.P.: Nonlinear Optimization. No. v. 13 in Nonlinear Optimization. Princeton University Press, Princeton (2006)

    MATH  Google Scholar 

  30. Sagastizábal, C., Solodov, M.: An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter. SIAM J. Optim. 16(1), 146–169 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  31. Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2(1), 121–152 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  32. Solodov, M.V.: On approximations with finite precision in bundle methods for nonsmooth optimization. J. Optim. Theory Appl. 119(1), 151–165 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  33. Solodov, M.V.: A bundle method for a class of bilevel nonsmooth convex minimization problems. SIAM J. Optim. 18(1), 242–259 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  34. van Ackooij, W., Frangioni, A., de Oliveira, W.: Inexact stabilized benders’ decomposition approaches with application to chance-constrained problems with finite support. Comput. Optim. Appl. 65(3), 637–669 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wolfe, P.: A method of conjugate subgradients for minimizing nondifferentiable functions. In: Balinski, M.L., Wolfe, P. (eds.) Nondifferentiable Optimization, Mathematical Programming Studies, vol. 3, pp. 145–173. Springer, Berlin (1975)

    Chapter  Google Scholar 

Download references

Acknowledgements

The author would like to give thanks to Prof. Michal Kočvara for the Fortran codes of the BT algorithm and a mex file for running in Matlab. He also thanks Prof. José Mario Martinez for comments on the CPLD constraint qualification and Prof. Claudia Sagastizábal for suggesting the extension to \(p\)-norm of the subproblem. The majority of this research was conducted during his PhD study under the supervision of Prof. Andrew Eberhard. He is immensely grateful to Andrew for his comments on an earlier version of the manuscript. He is also thankful to the reviewers for their valuable suggestions.

Author information

Authors and Affiliations

  1. IMECC/UNICAMP, Rua Sérgio Buarque de Holanda, 651, Campinas, SP, 13083-859, Brazil

    Shuai Liu

Authors
  1. Shuai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Liu.

Additional information

This research was partially supported by ARC Grant DP12100567 and FAPESP Grant 2017 / 15936-2.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S. A simple version of bundle method with linear programming. Comput Optim Appl 72, 391–412 (2019). https://doi.org/10.1007/s10589-018-0048-5

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-0048-5

Keywords

Mathematics Subject Classification

Search

Navigation