Abstract
We derive compact representations of BFGS and symmetric rank-one matrices for optimization. These representations allow us to efficiently implement limited memory methods for large constrained optimization problems. In particular, we discuss how to compute projections of limited memory matrices onto subspaces. We also present a compact representation of the matrices generated by Broyden's update for solving systems of nonlinear equations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J. Barnes, “An algorithm for solving nonlinear equations based on the secant method,”Computer Journal 8 (1965) 66–67.
L. Biegler, J. Nocedal and C. Schmid, “Reduced Hessian methods for large scale constrained optimization,” Technical Report, Department of Electrical Engineering and Computer Science, Northwestern University (Evanston, IL, 1993).
C.G. Broyden, “A class of methods for solving nonlinear simultaneous equations,”Mathematics of Computation 19 (1965) 577–593.
A. Buckley and A. LeNir, “QN-like variable storage conjugate gradients,”Mathematical Programming 27 (1983) 103–119.
R.H. Byrd, P. Lu and J. Nocedal, “A limited memory algorithm for bound constrained optimization,” Technical Report, Department of Electrical Engineering and Computer Science, Northwestern University (Evanston, IL, 1993).
A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Testing a class of methods for solving minimization problems with simple bounds on the variables,”Mathematics of Computation 50/182 (1988) 399–430.
J.E. Dennis Jr. and R.B. Schnabel,Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Prentice-Hall, Englewood Cliffs, NJ, 1983).
R. Fletcher,Practical Methods of Optimization (Wiley, Chichester, 1987, 2nd ed.).
D.M. Gay and R.B. Schnabel, “Solving systems of nonlinear equations by Broyden's method with projected updates,” in: O.L. Mangasarian, R.R. Meyer and S.M. Robinson, eds.,Nonlinear Programming 3 (Academic Press, New York, 1978) pp. 245–281.
J.C. Gilbert and C. Lemaréchal, “Some numerical experiments with variable storage quasi-Newton algorithms,”Mathematical Programming 45 (1989) 407–436.
J.C. Gilbert and J. Nocedal, “The limited memory step computation and automatic differentiation,”Applied Math Letters 6(3) (1993) 47–50.
A. Griewank, “On automatic differentiation,” in: M. Iri and K. Tanabe, eds.,Mathematical Programming (Kluwer Academic Publishers, Tokyo, 1989) pp. 83–107.
H. Fayez Khalfan, “Topics in quasi-Newton methods for unconstrained optimization,” Ph.D. thesis, Department of Mathematics, University of Colorado (Boulder, CO, 1989).
H. Fayez Khalfan, R.H. Byrd, and R.B. Schnabel, “A theoretical and experimental study of the symmetric rank one update,”SIAM Journal on Optimization 3 (1993) 1–24.
D.C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization,”Mathematical Programming 45 (1989) 503–528.
D.Q. Mahidhara and L. Lasdon, “An SQP algorithm for large sparse nonlinear programs,” Technical report, MSIS Department, School of Business Administration, University of Texas (Austin, TX, 1990).
H. Matthies and G. Strang, “The solution of nonlinear finite element equations,”International Journal of Numerical Methods in Engineering 14 (1979) 1613–1626.
J. Nocedal, “Updating quasi-Newton matrices with limited storage,”Mathematics of Computation 35 (1980) 773–782.
J.M. Ortega and W.C. Rheinboldt,Iterative Solution of Nonlinear Equations in Several Variables (Academic Press, New York, 1970).
R.B. Schnabel, “Quasi-Newton methods using multiple secant equations,” Technical Report CU-CS-247-83, Department of Computer Science, University of Colorado (Boulder, CO, 1983).
H.F. Walker, “Implementation of the GMRES method using Householder transformations,”SIAM Journal on Scientific and Statistical Computing 9(1) (1988) 152–163.
Author information
Authors and Affiliations
Additional information
These authors were supported by the Air Force Office of Scientific Research under Grant AFOSR-90-0109, the Army Research Office under Grant DAAL03-91-0151 and the National Science Foundation under Grants CCR-8920519 and CCR-9101795.
This author was supported by the U.S. Department of Energy, under Grant DE-FG02-87ER25047-A001, and by National Science Foundation Grants CCR-9101359 and ASC-9213149.
Rights and permissions
About this article
Cite this article
Byrd, R.H., Nocedal, J. & Schnabel, R.B. Representations of quasi-Newton matrices and their use in limited memory methods. Mathematical Programming 63, 129–156 (1994). https://doi.org/10.1007/BF01582063
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01582063