Skip to main content
SpringerLink
Log in

On a Global Complexity Bound of the Levenberg-Marquardt Method

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we investigate a global complexity bound of the Levenberg-Marquardt method (LMM) for the nonlinear least squares problem. The global complexity bound for an iterative method solving unconstrained minimization of φ is an upper bound to the number of iterations required to get an approximate solution, such that ‖∇φ(x)‖≤ε. We show that the global complexity bound of the LMM is O(ε −2).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, New York (1995)

    MATH  Google Scholar 

  2. Moré, J.J.: The Levenberg-Marquardt algorithm: implementation and theory. Numer. Anal. 630, 105–116 (1978)

    Article  Google Scholar 

  3. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  4. Osborne, M.R.: Nonlinear least squares—the Levenberg algorithm revisited. J. Aust. Math. Soc. 19, 343–357 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  5. Wright, S.J., Holt, J.N.: An inexact Levenberg-Marquardt method for large sparse nonlinear least squares. J. Aust. Math. Soc. 26, 387–403 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  6. Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. SIAM, Philadelphia (2000)

    MATH  Google Scholar 

  7. Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularization methods for unconstrained optimization. Part II: Worst-case function- and derivative-evaluation complexity. Math. Program. doi:10.1007/s10107-009-0337-y

  8. Cartis, C., Gould, N.I.M., Toint, P.L.: On the complexity of steepest descent, Newton’s and regularized Newton’s methods for nonconvex unconstrained optimization problems. Technical Report 09/14, Department of Mathematics, FUNDP—University of Namur (2009)

  9. Gratton, S., Sartenaer, A., Toint, P.L.: Recursive trust-region methods for multiscale nonlinear optimization. SIAM J. Optim. 19, 414–444 (2008)

    Article  MathSciNet  Google Scholar 

  10. Nesterov, Yu.: Introductory Lectures on Convex Optimization. Kluwer Academic, Dordrecht (2004)

    MATH  Google Scholar 

  11. Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program., Ser. A 108, 177–205 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  12. Polyak, R.A.: Regularized Newton method for unconstrained convex optimization. Math. Program., Ser. B 120, 125–145 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  13. Ueda, K., Yamashita, N.: Convergence properties of the regularized Newton method for the unconstrained nonconvex optimization. Appl. Math. Optim. 62, 27–46 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  14. Ueda, K., Yamashita, N.: A regularized Newton method without line search for unconstrained optimization. Technical Report 2009-007, Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University (2009)

  15. Nesterov, Yu.: Modified Gauss-Newton scheme with worst-case guarantees for global performance. Optim. Methods Softw. 22, 469–483 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  16. Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg-Marquardt method. Comput., Suppl. (Wien) 15, 227–238 (2001)

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Advanced Technology R&D Center, Mitsubishi Electric Corporation, Hyogo, 661-8661, Japan

    Kenji Ueda

  2. Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto, 606-8501, Japan

    Nobuo Yamashita

Authors
  1. Kenji Ueda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nobuo Yamashita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nobuo Yamashita.

Additional information

Communicated by B. Polyak.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ueda, K., Yamashita, N. On a Global Complexity Bound of the Levenberg-Marquardt Method. J Optim Theory Appl 147, 443–453 (2010). https://doi.org/10.1007/s10957-010-9731-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-010-9731-0

Keywords

Search

Navigation