Abstract
Quasi-Newton methods are generally held to be the most efficient minimization methods for small to medium sized problems. From these the symmetric rank one update of Broyden (Math. Comp., vol. 21, pp. 368–381, 1967) has been disregarded for a long time because of its potential failure. The work of Conn, Gould and Toint (Math. Prog., vol. 50, pp. 177–195, 1991), Kelley and Sachs (COAP, vol. 9, pp. 43–64, 1998) and Khalfan, Byrd and Schnabel (SIOPT, vol. 3, pp. 1–24, 1993; SIOPT, vol. 6, pp. 1025–1039, 1996) has renewed the interest in this method. However the question of boundedness of the generated matrix sequence has not been resolved by this work. In the present paper it is shown that a slightly modified version of this update generates bounded updates and converges superlinearly for uniformly convex functions. Numerical results support these theoretical considerations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M. AlBaali, “Highly efficient Broyden methods of minimization with variable parameter, ” Opt. Meth. and Software, vol. 1, pp. 301–310, 1992.
M. AlBaali and R. Fletcher, “An efficient line search for nonlinear least squares, ” J.O.T.A., vol. 48, pp. 359–377, 1986.
J.T. Betts, “A gradient projection multiplier method for nonlinear programming, ” J.O.T.A., vol. 24, pp. 523–548, 1978.
C.G. Broyden, “Quasi-Newton methods and their application to function minimization, ” Math. Comp. vol. 21, pp. 368–381, 1967.
R.H. Byrd, D.C. Liu, and J. Nocedal, “On the behaviour of Broyden's class of quasi-Newton methods, ” SIOPT vol. 2, pp. 533–557, 1992.
A.R. Conn, N.I.M. Gould, and Ph.L. Toint, “Convergence of quasi-Newton matrices generated by the symmetric rank one update, ” Math. Prog., vol. 50, pp. 177–195, 1991.
J.E. Dennis and R.B. Schnabel, “Numerical methods for unconstrained optimization and nonlinear equations, ” reprint Classics in Applied Mathematics 16, Philadelphia: SIAM, 1996.
A.V. Fiacco and G.P. McCormick, Nonlinear Programming: Sequential Unconstrained Minimization Techniques, Unabridged, corrected republication. Classics in Applied Mathematics 4, Philadelphia SIAM 1990.
R. Fletcher, Practical Methods of Optimization, 2nd ed., Chinchester: Wiley, 1987.
D. Goldfarb, “Sufficient conditions for the convergence of a variable metric algorithm, ” in Optimization, R. Fletcher (Ed.), London-New York: Academic Press, 1969, pp. 273–281.
D.M. Himmelblau, Applied Nonlinear Programming, New York: McGraw Hill, 1972.
W. Hock and K. Schittkowski, “Testexamples for nonlinear programming codes, ” Lect. Notes in Econ. and Math. Syst. vol. 187, Berlin-Heidelberg-New York: Springer, 1981.
C.T. Kelley and E.W. Sachs, “Local convergence of the symmetric rank-one iteration, ” COAP, vol. 9, pp. 43–64, 1998.
H.F. Khalfan, R.H. Byrd, and R.B. Schnabel, “A theoretical and experimental study of the symmetric rank one update, ” SIOPT, vol. 3, pp. 1–24, 1993.
H.F. Khalfan, R.H. Byrd, and R.B. Schnabel, “Analysis of a symmetric rank one trust region method, ” SIOPT, vol. 6, pp. 1025–1039, 1996.
H. Kleinmichel, “Quasi-Newton-Verfahren vom Rang-Eins-Typ zur Lösung unrestringierter Minimierungsaufgaben. Teil I: Verfahren und grundlegende Eigenschaften, ” Num. Math., vol. 38, pp. 219–228, 1981.
H. Kleinmichel, “Quasi-Newton-Verfahren vom Rang-Eins-Typ zur Lösung unrestringierter Minimierungsaufgaben. Teil II: n-Schritt quadratische Konvergenz für Restart-Varianten, ” Num. Math., vol. 38, pp. 229–244, 1981.
L. Lukšan, “Computational experience with improved variable metric methods for unconstrained minimization, ” Kybernetika, vol. 26, pp. 415–431, 1990.
J.J. Moré, “The Levenberg-Marquardt algorithm: Implementation and theory, ” pp. 105–116, in Lecture Notes in Math., vol. 680: Numerical Analysis, G. Watson (Ed.), Berlin-Heidelberg: New York: Springer, 1977, pp. 105–116.
St. G. Nash, “Newton type minimization via the Lanczos method, ” SINUM, vol. 21, pp. 770–788, 1984.
L.M. Nazareth, “A conjugate direction algorithm without line searches, ” J.O.T.A., vol. 23, pp. 377–387, 1977.
J.L. Nazareth and B. Smith, “Metric based symmetric rank-one updates, ” COAP, vol. 8, pp. 219–244, 1997.
M.R. Osborne and L.P. Sun, “A new approach to the symmetric rank-one updating algorithm, ” Dept. of Statistics, Australian National University, Canberra, Australia. Report NMO/01, 1988.
J.M. Ortega and W.C. Rheinboldt, “Iterative Solution of Nonlinear Equations in Several Variables, ” New York-San Francisco-London: Academic Press, 1971.
M.J.D. Powell, “Some properties of the variable metric method, ” in Numerical Methods of Optimization, F.A. Lootsma (Ed.), Academic Press, 1972, pp. 1–17.
P. Spellucci, Numerische Verfahren der nichtlinearen Optimierung, Birkhäuser: Basel, 1993.
J. Stoer, “The convergence of matrices generated by Rank-2-methods from the restricted β-class of Broyden, ” Num. Math., vol. 44, pp. 37–52, 1984.
H. Wolkowicz, “Measures for symmetric rank-one updates, ” Math. of Oper. Res., vol. 19, pp. 815–830, 1994.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Spellucci, P. A Modified Rank One Update Which Converges Q-Superlinearly. Computational Optimization and Applications 19, 273–296 (2001). https://doi.org/10.1023/A:1011259905470
Issue Date:
DOI: https://doi.org/10.1023/A:1011259905470