Skip to main content
Springer Nature Link
Log in

Approximate quasi-Newton methods

  • Published:
Mathematical Programming Submit manuscript

Abstract

We consider the effect of approximation on performance of quasi-Newton methods for infinite dimensional problems. In particular we study methods in which the approximation is refined at each iterate. We show how the local convergence behavior of the quasi-Newton method in the infinite dimensional setting is affected by the refinement strategy. Applications to boundary value problems and integral equations are considered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. E.L. Allgower, K. Böhmer, F.A. Potra and W.C. Rheinboldt, “A mesh-independence principle for operator equations and their discretizations,”SIAM Journal on Numerical Analysis 23 (1986) 160–169.

    Google Scholar 

  2. C.G. Broyden, “A class of methods for solving simultaneous equations,”Mathematics of Computation 19 (1965) 577–593.

    Google Scholar 

  3. C.G. Broyden, “A new double-rank minimization algorithm,”AMS Notices 16 (1969) 670.

    Google Scholar 

  4. C.G. Broyden, “The convergence of single rank quasi-Newton methods,”Mathematics of Computation 24 (1970) 365–382.

    Google Scholar 

  5. C.G. Broyden, “The convergence of an algorithm for solving sparse nonlinear systems,”Mathematics of Computation 25 (1971) 285–294.

    Google Scholar 

  6. C.G. Broyden, J.E. Dennis and J.J. Moré, “On the local and superlinear convergence of quasi-Newton methods,”Journal of the Institute of Mathematics and its Applications 12 (1973) 223–246.

    Google Scholar 

  7. R.S. Dembo, S.C. Eisenstat and T. Steihaug, “Inexact Newton methods,”SIAM Journal on Numerical Analysis 19 (1982) 400–408.

    Google Scholar 

  8. J.E. Dennis and R.B. Schnabel,Numerical Methods for Nonlinear Equations and Unconstrained Optimization (Prentice-Hall, Englewood Cliffs, NJ, 1983).

    Google Scholar 

  9. J.E. Dennis and H.F. Walker, “Convergence theorems for least change secant update methods,”SIAM Journal on Numerical Analysis 3 (1981) 949–987.

    Google Scholar 

  10. J.E. Dennis and H.F. Walker, “Least-change sparse secant updates with inaccurate secant conditions,”SIAM Journal on Numerical Analysis 22 (1985) 760–778.

    Google Scholar 

  11. J.C. Dunn, “Diagonally modified conditional gradient methods for input constrained optimal control problems,”SIAM Journal on Control and Optimization 24 (1986) 1177–1191.

    Google Scholar 

  12. J.C. Dunn and E.W. Sachs, “The effect of perturbations on the convergence rates of optimization algorithms,”Applied Mathematics and Optimization 10 (1983) 143–147.

    Google Scholar 

  13. R. Fletcher, “A new approach to variable metric methods,”Computer Journal 13 (1970) 317–322.

    Google Scholar 

  14. Z. Fortuna, “Some convergence properties of the conjugate gradient method in Hilbert space,”SIAM Journal on Numerical Analysis 16 (1979) 380–384.

    Google Scholar 

  15. D. Goldfarb, “A family of variable metric methods derived by variational means,”Mathematics of Computation 24 (1970) 23–26.

    Google Scholar 

  16. A. Griewank, “The solution of boundary value problems by Broyden based secant methods,” in J. Noye and R. May, eds.,Computational Techniques and Applications: CTAC 85, Proceedings of CTAC, Melbourne, August 1985 (North-Holland, Amsterdam, 1986) pp. 309–321.

    Google Scholar 

  17. A. Griewank, “Rates of convergence for secant methods on nonlinear problems in Hilbert space,” in: J.P. Hennart, ed.,Springer Lecture Notes, Vol. 1230 (Springer, New York, 1987) pp. 138–157.

    Google Scholar 

  18. A. Griewank, “The local convergence of Broyden-like methods on Lipschitzian problems in Hilbert space,”SIAM Journal on Numerical Analysis 24 (1987) 684–705.

    Google Scholar 

  19. A. Griewank and P.L. Toint, “Partitioned variable metric methods for large structured optimization problems,”Numerische Mathematik 39 (1982) 119–137.

    Google Scholar 

  20. R.D. Grigorieff, “Über diskrete Approximationen nichtlinearer Gleichungen 1. Art.,”Mathematische Nachrichten 69 (1975) 253–272.

    Google Scholar 

  21. W. Hackbusch, “Multigrid methods of the second kind,” in: D.J. Paddon and H. Holstein, eds.,Multigrid Methods for Integral and Differential Equations (Oxford University Press, Oxford, 1985).

    Google Scholar 

  22. W.E. Hart and S.O.W. Soul, “Quasi-Newton methods for discretized nonlinear boundary problems,”Journal of the Institute of Applied Mathematics 11 (1973) 351–359.

    Google Scholar 

  23. P. Henrici,Discrete Variable Methods in Ordinary Differential Equations (Wiley, New York, 1962).

    Google Scholar 

  24. L.V. Kantorovich and G.P. Akilov,Functional Analysis in Normed Spaces (Pergamon, New York, 1964).

    Google Scholar 

  25. C.T. Kelley and J.I. Northrup, “Pointwise quasi-Newton methods and some applications,” in: F. Kappel, K. Kunisch and W. Schappacher, eds.,Distributed Parameter Systems (Springer, New York, 1987) pp. 167–180.

    Google Scholar 

  26. C.T. Kelley and J.I. Northrup, “A pointwise quasi-Newton method for integral equations,”SIAM Journal on Numerical Analysis 25 (1988) 1138–1155.

    Google Scholar 

  27. C.T. Kelley and E.W. Sachs, “Broyden's method for approximate solution of nonlinear integral equations,”Journal of Integral Equations (1985) 25–44.

  28. C.T. Kelley and E.W. Sachs, “A quasi-Newton method for elliptic boundary value problems,SIAM Journal on Numerical Analysis 24 (1987) 516–531.

    Google Scholar 

  29. C.T. Kelley and E.W. Sachs, “Quasi-Newton methods and optimal control problems,”SIAM Journal on Control and Optimization 25 (1987) 1503–1517.

    Google Scholar 

  30. C.T. Kelley and E.W. Sachs, “A pointwise quasi-Newton method for unconstrained optimal control problems,”Numerische Mathematik 55 (1989) 159–176.

    Google Scholar 

  31. R. Klessig and E. Polak, “An adaptive precision gradient method for optimal control,”SIAM Journal on Control 11 (1973) 80–95.

    Google Scholar 

  32. E.S. Marwil, “Convergence results for Schubert's method for solving sparse nonlinear equations,”SIAM Journal on Numerical Analysis 16 (1979) 588–604.

    Google Scholar 

  33. E. Sachs, “Broyden's method in Hilbert space,”Mathematical Programming 35 (1986) 71–82.

    Google Scholar 

  34. E.W. Sachs, “Rates of convergence for adaptive Newton methods,”Journal of Optimization Theory and Applications 48 (1986) 175–190.

    Google Scholar 

  35. K. Schittkowski, “An adaptive precision method for nonlinear optimization problems,”SIAM Journal on Control 17 (1979) 82–98.

    Google Scholar 

  36. L.K. Schubert, “Modification on a quasi-Newton method for nonlinear equations with sparse Jacobian,”Mathematics of Computation 24 (1970) 27–30.

    Google Scholar 

  37. D.F. Shanno, “Conditioning of quasi-Newton methods for function minimization,”Mathematics of Computation 24 (1970) 647–657.

    Google Scholar 

  38. A.H. Sherman, “On Newton-iterative methods for the solution of systems of nonlinear equations,”SIAM Journal on Numerical Analysis 15 (1978) 755–776.

    Google Scholar 

  39. J. Stoer, “Two examples on the convergence of certain rank-2 minimization methods for quadratic functionals in Hilbert space,”Linear Algebra and Applications 28 (1984) 37–52.

    Google Scholar 

  40. F. Stummel, “Diskrete Konvergenz linearer Operatoren,”Mathematische Annalen 190 (1979) 45–92.

    Google Scholar 

  41. R.A. Tapia, “Diagonalized multiplier methods and quasi-Newton methods for constrained optimization,”Journal of Optimization Theory and Applications 22 (1977) 135–194.

    Google Scholar 

  42. Ph.L. Toint, “On the superlinear convergence of an algorithm for solving a sparse minimization problem,”SIAM Journal on Numerical Analysis 16 (1979) 1036–1045.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, North Carolina State University, P.O. Box 8205, 27695-8205, Raleigh, NC, USA

    C. T. Kelley

  2. FB-IV Mathematik, Universität Trier, Postfach 3825, 5500, Trier, FR Germany

    E. W. Sachs

Authors
  1. C. T. Kelley
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. E. W. Sachs
    View author publications

    You can also search for this author inPubMed Google Scholar

Additional information

The research of this author was supported by NSF grant DMS-8601139 and AFOSR grant AFOSR-ISSA-860074.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kelley, C.T., Sachs, E.W. Approximate quasi-Newton methods. Mathematical Programming 48, 41–70 (1990). https://doi.org/10.1007/BF01582251

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01582251

AMS(MOS) Subject Classifications

Key words