Skip to main content
SpringerLink
Log in

Globally convergent limited memory bundle method for large-scale nonsmooth optimization

  • Published:
Mathematical Programming Submit manuscript

Abstract

Many practical optimization problems involve nonsmooth (that is, not necessarily differentiable) functions of thousands of variables. In the paper [Haarala, Miettinen, Mäkelä, Optimization Methods and Software, 19, (2004), pp. 673–692] we have described an efficient method for large-scale nonsmooth optimization. In this paper, we introduce a new variant of this method and prove its global convergence for locally Lipschitz continuous objective functions, which are not necessarily differentiable or convex. In addition, we give some encouraging results from numerical experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bacaud, L., Lemaréchal, C., Renaud, A., Sagastizábal, C.: Bundle methods in stochastic optimal power management: A disaggregated approach using preconditioners. Comput. Optim. Appl. 20 (3), 227–244 (2001)

    Article  MathSciNet  Google Scholar 

  2. Bihain, A.: Optimization of upper semidifferentiable functions. J. Optimization Theory Appl. 4, 545–568 (1984)

    Article  MathSciNet  Google Scholar 

  3. Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  4. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley-Interscience, New York, 1983

  5. Clarke, F.H., Ledyaev, Y.S., Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control Theory. Springer, New York, 1998

  6. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  7. Fletcher, R.: Practical Methods of Optimization, 2nd edn. John Wiley and Sons, Chichester, 1987

  8. Gilbert, J.C., Lemaréchal, C.: Some numerical experiments with variable-storage quasi-Newton algorithms. Math. Program. 45, 407–435 (1989)

    Article  MATH  Google Scholar 

  9. Gröwe-Kuska, N., Kiwiel, K.C., Nowak, M.P., Römisch, W., Wegner, I.: Power management in a hydro-thermal system under uncertainty by Lagrangian relaxation. In: C. Greengard, A. Ruszczynski (eds.) Decision Making under Uncertainty: Energy and Power, IMA Volumes in Mathematics and its Applications, vol. 128, Springer, New York, 2002, pp. 39–70

  10. Haarala, M.: Large-scale nonsmooth optimization: Variable metric bundle method with limited memory. Ph.D. thesis, University of Jyväskylä, Department of Mathematical Information Technology (2004)

  11. Haarala, M., Miettinen, K., Mäkelä, M.M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. 19 (6), 673–692 (2004)

    MathSciNet  Google Scholar 

  12. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II. Springer-Verlag, Berlin, 1993

  13. Kärkkäinen, T., Majava, K., Mäkelä, M.M.: Comparison of formulations and solution methods for image restoration problems. Reports of the Department of Mathematical Information Technology, Series B. Scientific Computing, B 14/2000 University of Jyväskylä, Jyväskylä (2000)

  14. Kärkkäinen, T., Majava, K., Mäkelä, M.M.: Comparison of formulations and solution methods for image restoration problems. Inverse Probl. 17 (6), 1977–1995 (2001)

    Google Scholar 

  15. Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics 1133. Springer-Verlag, Berlin, 1985

  16. Kiwiel, K.C.: A method for solving certain quadratic programming problems arising in nonsmooth optimization. IMA J. Numer. Anal. 6, 137–152 (1986)

    MATH  MathSciNet  Google Scholar 

  17. Lemaréchal, C.: Nondifferentiable optimization. In: G.L. Nemhauser, A.H.G. Rinnooy Kan, M.J. Todd (eds.) Optimization. Elsevier North-Holland, Inc., New York, 1989, pp. 529–572

  18. Lemaréchal, C., Strodiot, J.J., Bihain, A.: On a bundle algorithm for nonsmooth optimization. In: O.L. Mangasarian, R.R. Mayer, S.M. Robinson (eds.) Nonlinear Programming. Academic Press, New York, 1981, pp. 285–281

  19. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)

    Article  MATH  Google Scholar 

  20. Lukšan, L., Vlček, J.: Globally convergent variable metric method for convex nonsmooth unconstrained minimization. J. Optimization Theory Appl. 102, 593–613 (1999)

    Article  Google Scholar 

  21. Mäkelä, M.M.: Issues of implementing a Fortran subroutine package NSOLIB for nonsmooth optimization. Technical Report 5/1993, Department of Mathematics, Laboratory of Scientific Computing, University of Jyväskylä (1993)

  22. Mäkelä, M.M.: Survey of bundle methods for nonsmooth optimization. Optim. Methods Softw. 17 (1), 1–29 (2002)

    Google Scholar 

  23. Mäkelä, M.M., Miettinen, M., Lukšan, L., Vlček, J.: Comparing nonsmooth nonconvex bundle methods in solving hemivariational inequalities. J. Glob. Optim. 14, 117–135 (1999)

    Article  MATH  Google Scholar 

  24. Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing Co., Singapore, 1992

  25. Mifflin, R.: A modification and an extension of Lemaréchal's algorithm for nonsmooth minimization. Math. Program. Study 17, 77–90 (1982)

    MATH  MathSciNet  Google Scholar 

  26. Mistakidis, E.S., Stavroulakis, G.E.: Nonconvex Optimization in Mechanics. Smooth and Nonsmooth Algorithms, Heuristics and Engineering Applications by the F.E.M. Kluwert Academic Publishers, Dordrecht, 1998

  27. Moreau, J.J., Panagiotopoulos, P.D., Strang, G. (eds.): Topics in Nonsmooth Mechanics. Birkhäuser Verlag, Basel, 1988

  28. Naniewicz, Z., Panagiotopoulos, P.D.: Mathematical Theory of Hemivariational Inequalities and Applications. Marcel Dekker, New York, 1995

  29. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35 (151), 773–782 (1980)

    MathSciNet  Google Scholar 

  30. Nowak, M.P., Nürnberg, R., Römisch, W., Schultz, R., Westphalen, M.: Stochastic programming for power production and trading under uncertainty. In: W. Jäger, H.J. Krebs (eds.) Mathematics –- Key Technology for the Future. Springer, Berlin, 2003, pp. 623–636

  31. Outrata, J., Kočvara, M., Zowe, J.: Nonsmooth Approach to Optimization Problems With Equilibrium Constraints. Theory, Applications and Numerical Results. Kluwert Academic Publisher, Dordrecht, 1998

  32. Panagiotopoulos, P.D.: Hemivariational Inequalities. Springer-Verlag, New York, 1993

  33. Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: Conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2 (1), 121–152 (1992)

    Article  MathSciNet  Google Scholar 

  34. Vlček, J., Lukšan, L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optimization Theory Appl. 111 (2), 407–430 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computational & Applied Mathematics, University of the Witwatersrand, Private Bag 3, Wits, 2050, Johannesburg, South Africa

    Napsu Haarala

  2. Helsinki School of Economics, P.O. Box 1210, 00101, Helsinki, Finland

    Kaisa Miettinen

  3. Department of Mathematical Information Technology, University of Jyväskylä, P.O. Box 35 (Agora), 40014, Finland

    Marko M. Mäkelä

Authors
  1. Napsu Haarala
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaisa Miettinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marko M. Mäkelä
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Napsu Haarala.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haarala, N., Miettinen, K. & Mäkelä, M. Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109, 181–205 (2007). https://doi.org/10.1007/s10107-006-0728-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-006-0728-2

Keywords

Search

Navigation