Skip to main content
SpringerLink
Log in

Damped Techniques for the Limited Memory BFGS Method for Large-Scale Optimization

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

This paper is aimed to extend a certain damped technique, suitable for the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method, to the limited memory BFGS method in the case of the large-scale unconstrained optimization. It is shown that the proposed technique maintains the global convergence property on uniformly convex functions for the limited memory BFGS method. Some numerical results are described to illustrate the important role of the damped technique. Since this technique enforces safely the positive definiteness property of the BFGS update for any value of the steplength, we also consider only the first Wolfe–Powell condition on the steplength. Then, as for the backtracking framework, only one gradient evaluation is performed on each iteration. It is reported that the proposed damped methods work much better than the limited memory BFGS method in several cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, Chichester (1987). Reprinted in 2000

    MATH  Google Scholar 

  2. Al-Baali, M.: Convergence analysis of a class of damped quasi-Newton methods for nonlinear optimization. Research Report, DOMAS 11/2, Sultan Qaboos University, Oman (2011)

  3. Powell, M.J.D.: Algorithms for nonlinear constraints that use Lagrange functions. Math. Program. 14, 224–248 (1978)

    Article  MATH  Google Scholar 

  4. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773–782 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  5. Al-Baali, M., Grandinetti, L.: On practical modifications of the quasi-Newton BFGS method. Adv. Model. Optim. 11, 63–76 (2009)

    MATH  MathSciNet  Google Scholar 

  6. Al-Baali, M.: Quasi-Wolfe conditions for quasi-Newton methods for large-scale optimization. In: 40th Workshop on Large Scale Nonlinear Optimization, Erice, Italy, June 22–July 1 (2004)

  7. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  8. Kelley, C.T.: Iterative Methods for Optimization, 2nd edn. Siam, Philadelphia (1999)

    Book  MATH  Google Scholar 

  9. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, London (1999)

    Book  MATH  Google Scholar 

  10. Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  11. Al-Baali, M.: Variational quasi-Newton methods for unconstrained optimization. J. Optim. Theory Appl. 77, 127–143 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  12. Al-Baali, M.: Improved Hessian approximations for the limited memory BFGS method. Numer. Algorithms 22, 99–112 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  13. Nash, S.G., Nocedal, J.: A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization. SIAM J. Optim. 1, 358–372 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  14. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    Article  MATH  Google Scholar 

  15. Toint, P.L.: Test problems for partially separable optimization and results for the routine PSPMIN. Technical Report No. 83/4, Department of Mathematics, Faculties University de Namur, Namur, Belgium (1983)

  16. Grandinetti, L.: Some investigations in a new algorithm for nonlinear optimization based on conic models of the objective function. J. Optim. Theory Appl. 43, 1–21 (1984)

    MATH  MathSciNet  Google Scholar 

  17. Siegel, D.: Implementing and modifying Broyden class updates for large scale optimization. Technical Report NA12, Department of Applied Mathematics and Theoretical Physics, Cambridge University, England (1992)

  18. Toint, Ph.L.: Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math. Comput. 32, 839–851 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  19. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  20. Malmedy, V., Toint, Ph.L.: Approximating Hessians in unconstrained optimization arising from discretized problems. Comput. Optim. Appl. 50, 1–22 (2011)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, Sultan Qaboos University, Muscat, Oman

    Mehiddin Al-Baali

  2. Department of Computer Engineering, Electronics and Systems, University of Calabria, Rende, Italy

    Lucio Grandinetti

  3. Information Engineering Department, Polytechnic University of Marche, Ancona, Italy

    Ornella Pisacane

Authors
  1. Mehiddin Al-Baali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lucio Grandinetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ornella Pisacane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lucio Grandinetti.

Additional information

Communicated by David Hull.

Some results were presented by M. Al-Baali at 40th Workshop on Large Scale Nonlinear Optimization, Erice, Italy, June 22–July 1, 2004.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Al-Baali, M., Grandinetti, L. & Pisacane, O. Damped Techniques for the Limited Memory BFGS Method for Large-Scale Optimization. J Optim Theory Appl 161, 688–699 (2014). https://doi.org/10.1007/s10957-013-0448-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-013-0448-8

Keywords

Search

Navigation