Abstract
This paper describes a new unconstrained optimisation procedure employing conjugate directions and requiring only threen-dimensional vectors. The method has been tested for computational efficiency and stability on a large set of test functions and compared with numerical data of other major methods. Results show that the method possesses strong superiority over other existing conjugate gradient methods on all problems and can out-perform or is at least as efficient as quasi-Newton methods on many tested problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
E.M.L. Beale, “A derivation of conjugate gradients”, in: F.A. Lootsma, ed.,Numerical methods for nonlinear optimisation (Academic Press, New York, 1972) pp. 39–43.
M.C. Biggs, “A note on minimisation algorithms which make use of nonquadratic properties of the objective function”,Journal of the Institute of Mathematics and its Applications 12 (1973) 337–338.
M.J. Box, “A comparison of several current optimisation methods and the use of transformations in constrained problems”,The Computer Journal 9 (1966) 67–77.
R.P. Brent,Algorithms for minimisation without derivatives (Prentice Hall, Englewood Cliffs, N.J., 1973).
K.W. Brodlie, “An assessment of two approaches to variable metric methods”,Mathematical Programming 12 (1977) 344–355.
A.G. Buckley, “A combined conjugate gradient quasi-Newton minimisation algorithm”,Mathematical Programming 15 (1978) 200–210.
A.R. Colville, “A comparative study of nonlinear programming codes”, IBM New York Scientific Centre Report 320-2949 (1968).
H. Crowder and P. Wolfe, “Linear convergence of the conjugate gradient method”, IBM Journal of Research and Development 16 (1972) 431–433.
W.C. Davidon, “Optimally conditioned optimisation algorithms without line searches”,Mathematical Programming 9 (1975) 1–30.
R. Fletcher, “A new approach to variable metric algorithms”,The Computer Journal 13 (1970) 317–322.
R. Fletcher and M.J.D. Powell, “A rapidly convergent descent method for minimisation”,The Computer Journal 6 (1963) 163–168.
R. Fletcher and C.M. Reeves, “Function minimisation by conjugate gradients,The Computer Journal 7 (1964) 149–154.
D.M. Himmelblau,Applied nonlinear programming McGraw-Hill, New York, 1972).
J. Kowalik and M.R. Osborne,Methods for unconstrained optimisation problem (American Elsevier Publishing Company, New York, 1968).
F.A. Lootsma,Numerical methods for nonlinear optimisation (Academic Press, New York, 1972).
F.A. Lootsma, “Performance evaluation of nonlinear optimization methods via multi-criteria decision analysis and via linear model analysis”, in: M.J.D. Powell, ed.,Nonlinear Optimization (Academic Press, New York, 1982) pp. 419–453.
NAG, Numerical Algorithms Group Inc.,NAG Fortran Library — Mark 8 (Downers Grove, Illinois, 1981).
S.S. Oren and E. Spedicato, “Optimal conditioning of self-scaling variable metric algorithms”,Mathematical Programming 10 (1976) 70–90.
M. Osborne and M.A. Saunders, “Descent methods for minimisation”, in: R.S. Anderssen, L.S. Jennings and D.M. Ryan, eds.,Optimisation (University of Queensland Press, Brisbane, 1972) pp. 221–237.
A. Perry, “A modified conjugate gradient algorithm”,Operations Research 26 (1978) 1073–1078.
M.J.D. Powell, “Restart procedures for the conjugate gradient method”,Mathematical Programming 12 (1977) 241–254.
H.H. Rosenbrock, “An automatic method for finding the greatest or least value of a function”,The Computer Journal 3 (1960) 175–184.
D.F. Shanno, “Conjugate gradient methods with inexact searches”,Mathematics of Operations Research 3 (1978) 244–256.
D.F. Shanno and K.H. Phua, “Matrix conditioning and non-linear optimisation”,Mathematical Programming 14 (1978) 149–160.
E. Spedicato, “Computational experience with quasi-Newton algorithms for minimisation problems of moderately large size”, in: L.C.W. Dixon and G.P. Szego, eds.,Towards global optimisation 2 (North-Holland, Amsterdam, 1978) pp. 209–219.
G.W. Stewart, “A modification of Davidon's minimisation method to accept difference approximations of derivatives”,Journal of ACM 14 (1967) 72–83.
C. Sutti, “Numerical experiences on a minimisation algorithm without derivatives”, in: L.C.W. Dixon and G.P. Szego, eds.,Towards global optimisation 2 (North-Holland, Amsterdam, 1978) pp. 255–267.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Le, D. A fast and robust unconstrained optimization method requiring minimum storage. Mathematical Programming 32, 41–68 (1985). https://doi.org/10.1007/BF01585658
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01585658