Skip to main content
SpringerLink
Log in

A limited memory BFGS algorithm for non-convex minimization with applications in matrix largest eigenvalue problem

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

This study aims to present a limited memory BFGS algorithm to solve non-convex minimization problems, and then use it to find the largest eigenvalue of a real symmetric positive definite matrix. The proposed algorithm is based on the modified secant equation, which is used to the limited memory BFGS method without more storage or arithmetic operations. The proposed method uses an Armijo line search and converges to a critical point without convexity assumption on the objective function. More importantly, we do extensive experiments to compute the largest eigenvalue of the symmetric positive definite matrix of order up to 54,929 from the UF sparse matrix collection, and do performance comparisons with EIGS (a Matlab implementation for computing the first finite number of eigenvalues with largest magnitude). Although the proposed algorithm converges to a critical point, not a global minimum theoretically, the compared results demonstrate that it works well, and usually finds the largest eigenvalue of medium accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Absil P-A, Mahony R, Andrews B (2005) Convergence of the iterates of descent methods for analytic cost functions. SIAM J Optim 16:531–547

    Article  MathSciNet  MATH  Google Scholar 

  • Auchmuty G (1983) Duality for non-convex variational principles. J Differ Equ 50:80–145

    Article  MathSciNet  MATH  Google Scholar 

  • Auchmuty G (1989) Unconstrained variational principles for eigenvalues of real symmetric matrices. SIAM J Math Anal 20:1186–1207

    Article  MathSciNet  MATH  Google Scholar 

  • Bai Z, Dongarra J, Ruhe A, van der Vorst H (2000) Templates for the solution of algebraic eigenvalue problems: a practical guide. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  • Byrd RH, Nocedal J (1989) A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J Numer Anal 26:727–739

    Article  MathSciNet  MATH  Google Scholar 

  • Byrd RH, Nocedal J, Yuan Y (1987) Global convergence of a class of quasi-Newton methods on convex problems. SIAM J Numer Anal 24:1171–1189

    Article  MathSciNet  MATH  Google Scholar 

  • Conn AR, Gould NIM, Toint PhL (1995) CUTE: constrained and unconstrained testing environment. ACM Trans Math Softw 21:123–160

    Article  MATH  Google Scholar 

  • Dai YH (2002) Convergence properties of the BFGS algorithm. SIAM J Optim 13:693–701

    Article  MathSciNet  MATH  Google Scholar 

  • Davis TA, Hu Y (2011) The university of Florida sparse matrix collection. ACM Trans Math Softw 38:1–25

    MathSciNet  Google Scholar 

  • Demmel J, Veselić K (1992) Jacobis method is more accurate than QR. SIAM J Matrix Anal Appl 13:1204C1245

    MathSciNet  MATH  Google Scholar 

  • Demmel J, Marques OA, Parlett BN, Vömel C (2008) Performance and accuracy of LAPACK’s symmetric tridiagonal eigensolvers. SIAM J Sci Comput 30:1508–1526

    Article  MathSciNet  MATH  Google Scholar 

  • Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program 91:201–213

    Article  MathSciNet  MATH  Google Scholar 

  • Gao H, Dai YH, Tong X (2015) Barzilai–Borwein-like methods for the extreme eigenvalue problem. J Ind Manag Optim 11:999–1019

    Article  MathSciNet  MATH  Google Scholar 

  • Golub GH, Van Loan CF (1996) Matrix computation, 3rd edn. John Hopkins University Press, Baltimore

    MATH  Google Scholar 

  • Jiang B, Dai YH (2013) Feasible Barzilai–Borwein-like methods for extreme symmetric eigenvalue problems. Optim Methods Softw 28:756–784

    Article  MathSciNet  MATH  Google Scholar 

  • Li DH, Fukushima M (2001a) A modified BFGS method and its global convergence in nonconvex minimization. J Comput Appl Math 129:15–35

    Article  MathSciNet  MATH  Google Scholar 

  • Li DH, Fukushima M (2001b) On the global convergence of the BFGS method for nonconvex unconstrained optimization problems. SIAM J Optim 11:1054–1064

    Article  MathSciNet  MATH  Google Scholar 

  • Liu D, Nocedal J (1989) On the limited memory BFGS method for large-scale optimization. Math Program 45:503–528

    Article  MathSciNet  MATH  Google Scholar 

  • Mascarenhas WF (2004) The BFGS method with exact line searchs fails for non-convex objective functions. Math Program 99:49–61

    Article  MathSciNet  MATH  Google Scholar 

  • Mongeau M, Torki M (2004) Computing eigenelements of real symmetric Matrices via optimization. Comput Optim Appl 29:263–287

    Article  MathSciNet  MATH  Google Scholar 

  • Nocedal J (1980) Updating quasi-Newton matrices with limited storage. Math Comput 35:773–782

    Article  MathSciNet  MATH  Google Scholar 

  • Sameh AH, Wisniewski JA (1982) A trace minimization algorithm for the generalized eigenvalue problem compuations. SIAM J Numer Anal 19:1243–1259

    Article  MathSciNet  MATH  Google Scholar 

  • Wei Z, Yu G, Yuan G, Lian Z (2004) The superlinear convergence of a modified BFGS-type method for uconstrained optimization. Comput Optim Appl 29:315–332

    Article  MathSciNet  MATH  Google Scholar 

  • Wei Z, Li G, Qi LQ (2006) New quasi-Newton methods for unconstrained optimization problems. Appl Math Comput 175:1156–1188

    Article  MathSciNet  MATH  Google Scholar 

  • Wei Z, Li G, Qi LQ (2008) Global convergence of the Polak–Ribiere–Polyak conjugate gradient method with inexact line searches for nonconvex unconstrained optimization problems. Math Comput 77:2173–2193

    Article  MathSciNet  MATH  Google Scholar 

  • Xiao Y, Li T, Wei Z (2013) Global convergence of a modified limited memory BFGS method for non-convex minimization. Acta Math Appl Sin E 29:555–566

    Article  MathSciNet  MATH  Google Scholar 

  • Xiao Y, Wang Q, Wang D (2010) Notes on the Dai–Yuan–Yuan modified spectral gradient method. J Comput Appl Math 234:2986–2992

    Article  MathSciNet  MATH  Google Scholar 

  • Xiao Y, Wei Z, Wang Z (2008) A limited memory BFGS-type method for large-scale unconstrained optimization. Comput Math Appl 56:1001–1009

    Article  MathSciNet  MATH  Google Scholar 

  • Xu CX, Zhang JZ (2001) A survey of quasi-Newton equations and quasi-Newton methods for optimization. Optimization and numerical algebra. Ann Oper Res 103:213–234

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang L (2005) A globally convergent BFGS method for nonconvex minimization without line searches. Optim Method Softw 20:737–747

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang JZ, Deng NY, Chen LH (1999) New quasi-Newton equation and related methods for unconstrained optimization. J Optim Theory Appl 102:147–167

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang J, Xiao Y, Wei Z (2009) Nonlinear conjugate gradient methods with sufficient descent condition for large-scale unconstrained optimization. Math Prob Eng Article ID 243290:16

  • Zhou W, Zhang L (2009) Global convergence of the nonmonotone MBFGS method for nonconvex unconstrained minimization. J Comput Appl Math 223:40–47

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

We are grateful to the associate editor and two anonymous referees for their valuable comments and suggestions which have helped us improve the presentation of this paper greatly. The work was supported by the Major State Basic Research Development Program of China (973 Program) (Grant No. 2015CB856003), the National Natural Science Foundation of China (Grant No. 11471101), and the Program for Science and Technology Innovation Talents in Universities of Henan Province (Grant No. 13HASTIT050).

Author information

Authors and Affiliations

  1. College of Mathematics and Statistics, Henan University, Kaifeng, 475000, China

    Zhanwen Shi & Guanyu Yang

  2. Institute of Applied Mathematics, College of Mathematics and Statistics, Henan University, Kaifeng, 475000, China

    Yunhai Xiao

Authors
  1. Zhanwen Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanyu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunhai Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunhai Xiao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Z., Yang, G. & Xiao, Y. A limited memory BFGS algorithm for non-convex minimization with applications in matrix largest eigenvalue problem. Math Meth Oper Res 83, 243–264 (2016). https://doi.org/10.1007/s00186-015-0527-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-015-0527-8

Keywords

Mathematics Subject Classification

Search

Navigation