Abstract
Quasi-Newton algorithms for unconstrained nonlinear minimization generate a sequence of matrices that can be considered as approximations of the objective function second derivatives. This paper gives conditions under which these approximations can be proved to converge globally to the true Hessian matrix, in the case where the Symmetric Rank One update formula is used. The rate of convergence is also examined and proven to be improving with the rate of convergence of the underlying iterates. The theory is confirmed by some numerical experiments that also show the convergence of the Hessian approximations to be substantially slower for other known quasi-Newton formulae.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Global convergence of a class of trust region algorithms for optimization with simple bounds,”SIAM Journal on Numerical Analysis 25 (1988) 433–460 (with a correction given inSIAM Journal on Numerical Analysis 26 (1989) 764–767).
A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Testing a class of methods for solving minimization problems with simple bounds on the variables,”Mathematics of Computation 50 (1988) 399–430.
J.E. Dennis and J.J. Moré, “Quasi-Newton methods, motivation and theory,”SIAM Review 19 (1977) 46–89.
J.E. Dennis and R.B. Schnabel,Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Prentice-Hall, Englewood Cliffs, NJ, 1983).
A.V. Fiacco and G.P. McCormick,Nonlinear Programming (Wiley, New York, 1968).
R. Fletcher,Practical Methods of Optimization: Unconstrained Optimization (Wiley, Chichester, 1980).
R.P. Ge and M.J.D. Powell, “The convergence of variable metric matrices in unconstrained optimization,”Mathematical Programming 27 (1983) 123–143.
P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, New York, 1981).
A. Griewank and Ph.L. Toint, “Partitioned variable metric updates for large structured optimization problems,”Numerische Mathematik 39 (1982) 119–137.
J.J. Moré, “Recent developments in algorithms and software for trust region methods,” in: A. Bachem, M. Grötschel and B. Korte, eds.,Mathematical Programming: The State of the Art (Springer, Berlin, 1983).
J.M. Ortega and W.C. Rheinboldt,Iterative Solution of Nonlinear Equations in Several Variables (Academic Press, New York, 1970).
M.J.D. Powell, “A new algorithm for unconstrained optimization,” in: J.B. Rosen, O.L. Mangasarian and K. Ritter, eds.,Nonlinear Programming (Academic Press, New York, 1970).
G. Schuller, “On the order of convergence of certain quasi-Newton methods,”Numerische Mathematik 23 (1974) 181–192.
D.C. Sorensen, “An example concerning quasi-Newton estimates of a sparse Hessian,”SIGNUM Newsletter 16 (1981) 8–10.
Ph.L. Toint, “On the superlinear convergence of an algorithm for solving a sparse minimization problem,”SIAM Journal on Numerical Analysis 16 (1979) 1036–1045.
Author information
Authors and Affiliations
Additional information
The work of this author was supported by the National Sciences and Engineering Research Council of Canada, and by the Information Technology Research Centre, which is funded by the Province of Ontario.
Rights and permissions
About this article
Cite this article
Conn, A.R., Gould, N.I.M. & Toint, P.L. Convergence of quasi-Newton matrices generated by the symmetric rank one update. Mathematical Programming 50, 177–195 (1991). https://doi.org/10.1007/BF01594934
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01594934