Skip to main content
SpringerLink
Log in

Local Convergence Analysis of the Levenberg–Marquardt Framework for Nonzero-Residue Nonlinear Least-Squares Problems Under an Error Bound Condition

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

The Levenberg–Marquardt method is widely used for solving nonlinear systems of equations, as well as nonlinear least-squares problems. In this paper, we consider local convergence properties of the method, when applied to nonzero-residue nonlinear least-squares problems under an error bound condition, which is weaker than requiring full rank of the Jacobian in a neighborhood of a stationary point. Differently from the zero-residue case, the choice of the Levenberg–Marquardt parameter is shown to be dictated by (i) the behavior of the rank of the Jacobian and (ii) a combined measure of nonlinearity and residue size in a neighborhood of the set of (possibly non-isolated) stationary points of the sum of squares function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Dennis Jr., J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Classics in Applied Mathematics, vol. 16. SIAM, Philadelphia (1996)

    Book  Google Scholar 

  2. Fan, J., Yuan, Y.: On the quadratic convergence of the Levenberg–Marquardt method without nonsingularity assumption. Computing 74(1), 23–29 (2005)

    Article  MathSciNet  Google Scholar 

  3. Gratton, S., Lawless, A.S., Nichols, N.K.: Approximate Gauss–Newton methods for nonlinear least squares problems. SIAM J. Optim. 18, 106–132 (2007)

    Article  MathSciNet  Google Scholar 

  4. Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt method. In: Alefeld, G., Chen, X. (eds.) Topics in Numerical Analysis, pp. 239–249. Springer, Viena (2001)

    Chapter  Google Scholar 

  5. Bellavia, S., Riccietti, E.: On an elliptical trust-region procedure for ill-posed nonlinear least-squares problems. J. Optim. Theory Appl. 178(3), 824–859 (2018)

    Article  MathSciNet  Google Scholar 

  6. Cornelio, A.: Regularized nonlinear least squares methods for hit position reconstruction in small gamma cameras. Appl. Math. Comput. 217(12), 5589–5595 (2011)

    MATH  Google Scholar 

  7. Deidda, G., Fenu, C., Rodriguez, G.: Regularized solution of a nonlinear problem in electromagnetic sounding. Inverse Probl. 30(12), 27 (2014). https://doi.org/10.1088/0266-5611/30/12/125014

    Article  MathSciNet  MATH  Google Scholar 

  8. Henn, S.: A Levenberg–Marquardt scheme for nonlinear image registration. BIT Numer. Math. 43(4), 743–759 (2003)

    Article  MathSciNet  Google Scholar 

  9. Landi, G., Piccolomini, E.L., Nagy, J.G.: A limited memory BFGS method for a nonlinear inverse problem in digital breast tomosynthesis. Inverse Probl. 33(9), 21 (2017). https://doi.org/10.1088/1361-6420/aa7a20

    Article  MathSciNet  MATH  Google Scholar 

  10. López, D.C., Barz, T., Korkel, S., Wozny, G.: Nonlinear ill-posed problem analysis in model-based parameter estimation and experimental design. Comput. Chem. Eng. 77(Supplement C), 24–42 (2015). https://doi.org/10.1016/j.compchemeng.2015.03.002

    Article  Google Scholar 

  11. Mandel, J., Bergou, E., Gürol, S., Gratton, S., Kasanickỳ, I.: Hybrid Levenberg–Marquardt and weak-constraint ensemble Kalman smoother method. Nonlinear Process. Geophys. 23, 59–73 (2016)

    Article  Google Scholar 

  12. Tang, L.M.: A regularization homotopy iterative method for ill-posed nonlinear least squares problem and its application. In: Applied Mechanics and Materials. Advances in Civil Engineering, ICCET 2011, vol. 90, pp. 3268–3273. Trans Tech Publications (2011). http://www.scientific.net/AMM.90-93.3268

    Article  Google Scholar 

  13. Dennis Jr., J.E.: Nonlinear least squares and equations. In: Jacobs, D.A.H. (ed.) The State of the Art in Numerical Analysis, pp. 269–312. Academic Press, London (1977)

    Google Scholar 

  14. Li, D., Fukushima, M., Qi, L., Yamashita, N.: Regularized Newton methods for convex minimization problems with singular solutions. Comput. Optim. Appl. 28(2), 131–147 (2004)

    Article  MathSciNet  Google Scholar 

  15. Behling, R., Fischer, A.: A unified local convergence analysis of inexact constrained Levenberg–Marquardt methods. Optim. Lett. 6(5), 927–940 (2012)

    Article  MathSciNet  Google Scholar 

  16. Behling, R., Iusem, A.: The effect of calmness on the solution set of systems of nonlinear equations. Math. Program. 137(1–2), 155–165 (2013)

    Article  MathSciNet  Google Scholar 

  17. Dan, H., Yamashita, N., Fukushima, M.: Convergence properties of the inexact Levenberg–Marquardt method under local error bound conditions. Optim. Methods Softw. 17(4), 605–626 (2002)

    Article  MathSciNet  Google Scholar 

  18. Fan, J.: Convergence rate of the trust region method for nonlinear equations under local error bound condition. Comput. Optim. Appl. 34(2), 215–227 (2006)

    Article  MathSciNet  Google Scholar 

  19. Fan, J., Pan, J.: Convergence properties of a self-adaptive Levenberg–Marquardt algorithm under local error bound condition. Comput. Optim. Appl. 34(1), 723–751 (2016)

    MathSciNet  Google Scholar 

  20. Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for computing the regularization parameter of the Levenberg–Marquardt method. Comput. Optim. Appl. 65(3), 723–751 (2016)

    Article  MathSciNet  Google Scholar 

  21. Ipsen, I.C.F., Kelley, C.T., Poppe, S.R.: Rank-deficient nonlinear least squares problems and subset selection. SIAM J. Numer. Anal. 49(3), 1244–1266 (2011)

    Article  MathSciNet  Google Scholar 

  22. Bergou, E., Diouane, Y., Kungurtsev, V.: Convergence and Iteration Complexity Analysis of a Levenberg–Marquardt Algorithm for Zero and Non-zero Residual Inverse Problems. http://www.optimization-online.org/DB_HTML/2017/05/6005.html. Accessed 5 July 2019

  23. Edwards Jr., C.H.: Advanced Calculus of Several Variables. Academic Press, New York (1994)

    Book  Google Scholar 

  24. Bellavia, S., Cartis, C., Gould, N.I.M., Morini, B., Toint, P.L.: Convergence of a regularized Euclidean residual algorithm for nonlinear least-squares. SIAM J. Numer. Anal. 48(1), 1–29 (2010)

    Article  MathSciNet  Google Scholar 

  25. Hiriart-Urruty, J.-B., Le, H.Y.: A variational approach of the rank function. TOP 21, 207–240 (2013)

    Article  MathSciNet  Google Scholar 

  26. Fischer, A.: Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Program. 94(1), 91–124 (2002)

    Article  MathSciNet  Google Scholar 

  27. Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems, Classics in Applied Mathematics, vol. 15. SIAM, Philadelphia (1995)

    Book  Google Scholar 

  28. Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. Johns Hopkins University Press, Baltimore (2013)

    MATH  Google Scholar 

Download references

Acknowledgements

We are thankful to two anonymous referees, whose suggestions helped to improve the first version of this paper. This work was partially supported by the Brazilian research agencies CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico) and FAPESP (Fundação de Amparo à Pesquisa do Estado de São Paulo): R. Behling Grants 304392/2018-9 and 429915/2018-7, D. S. Gonçalves Grant 421386/2016-9, S. A. Santos Grants 302915/2016-8, 2018/24293-0 and 2013/07375-0. The first author wants to thank the Federal University of Santa Catarina and remarks that his contribution to the present article was predominantly carried out at this institution.

Author information

Authors and Affiliations

  1. Fundação Getúlio Vargas, Rio de Janeiro, RJ, Brazil

    Roger Behling

  2. Federal University of Santa Catarina, Florianópolis, SC, Brazil

    Douglas S. Gonçalves

  3. University of Campinas, Campinas, SP, Brazil

    Sandra A. Santos

Authors
  1. Roger Behling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Douglas S. Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sandra A. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Douglas S. Gonçalves.

Additional information

Communicated by Nobuo Yamashita.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Behling, R., Gonçalves, D.S. & Santos, S.A. Local Convergence Analysis of the Levenberg–Marquardt Framework for Nonzero-Residue Nonlinear Least-Squares Problems Under an Error Bound Condition. J Optim Theory Appl 183, 1099–1122 (2019). https://doi.org/10.1007/s10957-019-01586-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-019-01586-9

Keywords

Mathematics Subject Classification

Search

Navigation