Abstract
LetB be a positive definite symmetric approximation to the second derivative matrix of the objective function of a minimization calculation. We study modifications of the BFGS method that apply a scaling technique to the columns of a conjugate direction matrixZ satisfyingZ T BZ = I. For a simple scaling technique similar to the ones considered by Powell (1987) and (1989) we show that, due to a two iteration cycle, linear convergence can occur when the method is applied to a quadratic function and Wolfe's line search is employed, although for exact line searches quadratic termination can be proved. We then suggest a different scaling technique that prevents certain columns thought to contain important curvature information from being scaled. For this algorithm we prove global and superlinear convergence and demonstrate the superiority of our method over the BFGS formula on a range of illconditioned optimization problems. Moreover, we present an implementation of our algorithm that requires only 3n 2 +O(n) multiplications per iteration.
Similar content being viewed by others
References
J.E. Dennis and J. Moré, “Quasi-Newton methods, motivation and theory,”SIAM Review 19 (1977) 46–89.
R. Fletcher,Practical Methods of Optimization (John Wiley & Sons, Chichester, 1987).
D. Goldfarb, “Factorized variable metric methods for unconstrained optimization,”Mathematics of Computation 30 (1976) 796–811.
S.-P. Han, “Optimization by updated conjugate subspaces,” in: eds. D.F. Griffiths and G.A. Watson,Numerical Analysis: Pitman Research Notes in Mathematics Series 140 (Longman Scientific & Techical Burnt Mill 1986) pp. 82–97.
M. Lalee and J. Nocedal, “Automatic column scaling strategies for quasi-Newton methods,” to appear inSIAM Journal on Optimization (1991).
M.J.D. Powell, “Some global convergence properties of a variable metric algorithm for minimization without exact line searches,” in: R.W. Cottle and C.E. Lemke (eds.),Nonlinear Programming, SIAM — AMS Proceedings Vol. IX (American Mathematical Society, Providence, 1976) pp. 53–72.
M.J.D. Powell, “How bad are the BFGS and DFP methods when the objective function is quadratic?,”Mathematical Programming 34 (1986) 34–37.
M.J.D. Powell, “Updating conjugate directions by the BFGS formula,”Mathematical Programming 38 (1987) 29–46.
M.J.D. Powell, “TOLMIN: a Fortran package for linearly constrained optimization calculations,” Report DAMTP 1989/NA2, University of Cambridge (1989).
D. Siegel, “Updating of conjugate direction matrices using members of Broyden's family,”Mathematical Programming 60 (2) (1993) 167–185.
P. Wolfe, “Convergence conditions for ascent methods,”SIAM Review 11 (1968) 226–235.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Siegel, D. Modifying the BFGS update by a new column scaling technique. Mathematical Programming 66, 45–78 (1994). https://doi.org/10.1007/BF01581137
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01581137