Skip to main content
SpringerLink
Log in

A regularized limited memory BFGS method for nonconvex unconstrained minimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The limited memory BFGS method (L-BFGS) is an adaptation of the BFGS method for large-scale unconstrained optimization. However, The L-BFGS method need not converge for nonconvex objective functions and it is inefficient on highly ill-conditioned problems. In this paper, we proposed a regularization strategy on the L-BFGS method, where the used regularization parameter may play a compensation role in some sense when the condition number of Hessian approximation tends to become ill-conditioned. Then we proposed a regularized L-BFGS method and established its global convergence even when the objective function is nonconvex. Numerical results show that the proposed method is efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Al-Baali, M.: Improved Hessian approximations for limited memory BFGS method. Numer. Algoritm. 22, 99–112 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  2. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)

    MATH  MathSciNet  Google Scholar 

  3. Byrd, R.H., Nocedal, J., Yuan, Y.X.: Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24, 1171–1190 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  4. Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  5. Chen, X.J.: Convergence of the BFGS method for \(LC^{1}\) convex constrained optimization. SIAM J. Control Optim. 34, 2051–2063 (1996)

    Article  MathSciNet  Google Scholar 

  6. Dai, Y.H.: Convergence properties of the BFGS algorithm. SIAM J. Optim. 13, 693–701 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  7. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28, 549–560 (1974)

    Article  MATH  Google Scholar 

  8. Dennis, J.E., Moré, J.J.: Quasi-Newton methods, motivation and theory. SIAM Rev. 19, 46–89 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  9. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, Chichester (1987)

    MATH  Google Scholar 

  11. Li, D.H., Fukushima, M.: On the global convergence of the BFGS method for nonconvex unconstrained problems. SIAM J. Optim. 11, 1054–1064 (2001a)

    Article  MATH  MathSciNet  Google Scholar 

  12. Li, D.H., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. 129, 15–35 (2001b)

    Article  MATH  MathSciNet  Google Scholar 

  13. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large-sclae optimization. Math. Program. 45, 503–528 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  14. Liu, T.W., Li, D.H.: A pratical update criterion for SQP method. Optim. Methods Soft. 2, 253–266 (2007)

    Article  MATH  Google Scholar 

  15. Mascarenhas, W.F.: The BFGS method with exact line searches fails for nonconvex objective functions. Math. Program. A 99, 49–61 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  16. moré, J.J., Garbow, B.S., Hillstrome, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Soft. 7, 17–41 (1981)

    Article  MATH  Google Scholar 

  17. Morales, J.L.: A numerical study of limited memory BFGS methods. Appl. Math. Lett. 15, 481–487 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  18. Nash, S.H., Nocedal, J.: A numerical study of the limited memory BFGS method and the truncated-Newton method for large-scale optimization. SIAM J. Optim. 1, 358–372 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  19. Nocedal, J.: Updating quasi-Newton matrices with limited storge. Math. Comput. 35, 773–782 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  20. Nocedal, J.: Theory of algorithms for unconstrained optimization. Acta Numer. 1, 199–242 (1992)

    Article  MathSciNet  Google Scholar 

  21. Nocedal, J., St. Wright, J.: Numerical Optimization. Springer, New York (2000)

    Google Scholar 

  22. Powell, M.J.D.: Quadratic termination properties of minimization algorithm, Part I and Part II . J. Inst. Math. Appl. 10, 333–357 (1972)

    Article  MATH  MathSciNet  Google Scholar 

  23. Powell, M.J.D.: Some convergence properties of a variable mertric algorithm for minimization without exact line search. In: Cottle, R.W., Lemke, C.E. (eds.) Nonlinear Programming, SIAM-AMS Proceedings, vol. IX, pp 53–72. SIAM, Philadelphia (1976)

    Google Scholar 

  24. Powell, M.J.D.: On the convergence of the DEP algorithm for unconstrained optimization when there are only two variables. Math. Program. B 87, 281–301 (2000)

    Article  MATH  Google Scholar 

  25. Pu, D., Yu, W.: On the convergence of the DFP algorithm. Ann. Oper. Res. 24, 175–184 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  26. Wei, Z.X., Yu, G.H., Yuan, G.L., et al: The superlinear convergence of a modified BFGS-type method for unconstrained optimization. J. Comput. Optim. Appl. 29, 315–332 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  27. Xiao, Y.H., Wei, Z.X., Wang, Z.G.: A limited memory BFGS-type method for large-scale unconstrained optimization. Comput. Math. Appl. 56, 1001–1009 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  28. Yu, G.H., Wei, Z.X., Guan, L.T.: A modified BFGS-trust region method for nonconvex minimization. Pac. J. Optim. 2, 119–133 (2006)

    MATH  MathSciNet  Google Scholar 

  29. Yang, Y.T., Xu, C.X.: A compact limited memory method for large-sclae unconstrained optimization. Eur. Oper. Res. 180, 48–56 (2007)

    Article  MATH  Google Scholar 

  30. Yuan, Y.: Numerical Methods for Nonlinear Programming. Shanghai Scientific and Technical Publishers, Shanghai (1993). In Chinese

    Google Scholar 

  31. Zhou, W., Zhang, L.: Global convergence of a regularized factorized quasi-Newton method for nonlinear least squares problems. Comput. Appl. Math. 29, 195–214 (2010)

    MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Mathematics and Econometrics, Hunan University, 410082, Changsha, China

    Tao-Wen Liu

Authors
  1. Tao-Wen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao-Wen Liu.

Additional information

This work was partially supported by the National Natural Sciences Foundation of China (No.11271115) and the Hunan Natural Sciences Foundation of China (No.13JJ3040)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, TW. A regularized limited memory BFGS method for nonconvex unconstrained minimization. Numer Algor 65, 305–323 (2014). https://doi.org/10.1007/s11075-013-9706-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-013-9706-y

Keywords

Mathematics Subject Classfication (2010)

Search

Navigation