Skip to main content
Springer Nature Link
Log in

The method of successive affine reduction for nonlinear minimization

  • Published:
Mathematical Programming Submit manuscript

Abstract

The traditional development of conjugate gradient (CG) methods emphasizes notions of conjugacy and the minimization of quadratic functions. The associated theory of conjugate direction methods, strictly a branch of numerical linear algebra, is both elegant and useful for obtaining insight into algorithms for nonlinear minimization. Nevertheless, it is preferable that favorable behavior on a quadratic be a consquence of a more general approach, one which fits in more naturally with Newton and variable metric methods. We give new CG algorithms along these lines and discuss some of their properties, along with some numerical supporting evidence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D.P. Bertsekas, “Projected Newton methods for optimization problems with simple constraints“,SIAM J. Control and Optimization 20 (1982) 221–246.

    Google Scholar 

  2. A. Buckley and A. LeNir, “QN-like variable storage conjugate gradients“,Mathematical Programming 27 (1983) 155–175.

    Google Scholar 

  3. O.P. Burdakov, “Stable variants of the secant method for nonlinear equations“,Journal of Computational Mathematics and Mathematical Physics (Moscow) 23 (1983) 1027–1040 (in Russian).

    Google Scholar 

  4. J.E. Dennis and J.J. Moré, “Quasi-Newton methods, motivation and theory“,SIAM Review 19 (1977) 46–89.

    Google Scholar 

  5. M.C. Fenelon, “Preconditioned conjugate-gradient-type methods for large-scale unconstrained optimization”, Ph.D. dissertation, Department of Operations Research, Stanford University (Stanford, CA, November 1981).

    Google Scholar 

  6. R. Fletcher, “AFORTRAN subroutine for minimization by the method of conjugate gradients”, Report R7073, AERE (Harwell, UK, 1972).

    Google Scholar 

  7. M.R. Hestenes and E. Stiefel, “Methods of conjugate gradients for solving linear systems“,J. Res. Nat. Bur. Std. 48 (1952) 409–436.

    Google Scholar 

  8. K.C. Kiwiel, “An aggregate subgradient method for nonsmooth convex minimization“,Mathematical Programming 27 (1983) 320–341.

    Google Scholar 

  9. C. Lemarechal, “Note on an extension of Davidon methods to nondifferentiable functions“,Mathematical Programming 7 (1974) 384–387.

    Google Scholar 

  10. J.J. Moré, B.S. Garbow and K.E. Hillstrom, “Testing unconstrained optimization software“,ACM Trans. on Math. Software 7 (1981) 17–41.

    Google Scholar 

  11. B.A. Murtagh and M.A. Saunders, “MINOS 5.0 Users Guide”, Technical Report SOL 83-20, Systems Optimization Laboratory, Department of Operations Research, Stanford University (Stanford, CA, 1983).

    Google Scholar 

  12. J.L. Nazareth, “Conjugate gradient methods less dependent on conjugacy”,SIAM Review, to appear.

  13. J.L. Nazareth, “Hierarchical implementation of optimization methods”, in: P. Boggs, R. Byrd and B. Schnabel (editors),Numerical Optimization 1984 (SIAM, Philadelphia, 1985) pp. 199–210. J.L. Nazareth, “An algorithm based upon successive affine reduction and Newton's method”, in: R. and J.L. Lyons (eds.),Proceedings of the 7th INRIA International Conference on Computing Methods in Applied Science and Engineering, Versailles, France, December 1985 (Springer-Verlag, to appear).

    Google Scholar 

  14. A. Perry, “Amodified conjugate gradient algorithm,” Discussion Paper 229, Center for Mathematical Studies in Economics and Management Science, Northwestern University (Evanston, IL, 1976).

    Google Scholar 

  15. M.J.D. Powell, “Some properties of the variable metric algorithm“, in: F.A. Lootsma (editor),Numerical Methods for Nonlinear Optimization (Academic Press, London, 1972) pp. 1–18.

    Google Scholar 

  16. D.F. Shanno, “Conjugate gradient methods with inexact line searches“,Mathematics of Operations Research 3 (1979) 244–256.

    Google Scholar 

  17. P. Wolfe, “A method of conjugate subgradients for minimizing nondifferentiable functions“,Mathematical Programming Study 3 (1975) 145–173.

    Google Scholar 

Download references

Author information

Author notes

  1. J. L. Nazareth

    Present address: Computational Decision Support Systems, P.O. Box 4908, 94704, Berkeley, CA, USA

Authors and Affiliations

  1. International Institute for Applied Systems Analysis, A-2361, Laxenburg, Austria

    J. L. Nazareth

Authors
  1. J. L. Nazareth
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

I acknowledge partial support by ONR Contract #N00014-76-C-0013 at the Center for Pure & Applied Mathematics, University of California, Berkeley in the dissemination of this article.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nazareth, J.L. The method of successive affine reduction for nonlinear minimization. Mathematical Programming 35, 97–109 (1986). https://doi.org/10.1007/BF01589444

Download citation

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01589444

Key words

Search

Navigation