Skip to main content
SpringerLink
Log in

A modified Levenberg–Marquardt method for solving system of nonlinear equations

  • Original Research
  • Published:
Journal of Applied Mathematics and Computing Aims and scope Submit manuscript

Abstract

A modified Levenberg–Marquardt methods for solving system of nonlinear equations is described and analysed in this paper. Specifically, we propose a convex combination of \(\Vert F_k\Vert ^\delta \) and \(\left\| J_k^TF_k\right\| ^\delta \) with \(\delta \in [1,2]\) for the LM parameter and analyse the convergence of this modified Levenberg–Marquardt method under the \(\gamma \)-Hölderian local error bound of the underlying function and the v-Hölderian continuity of its Jacobian. The results show that, under some suitable relationships of exponents v, \(\gamma \) and \(\delta \), the modified Levenberg–Marquardt method converges to the solution set of the system of nonlinear equations at least superlinearly. Numerical experiments show the new algorithm is efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Heidelberg (1999)

    Book  MATH  Google Scholar 

  2. Sun, W., Yuan, Y.: Optimization Theory and Methods: Nonlinear Programming. Springer, Heidelberg (2006)

    MATH  Google Scholar 

  3. Andrei, N.: Modern Numerical Nonlinear Optimization. Springer, Cham (2022)

    Book  MATH  Google Scholar 

  4. Xie, L., Ding, J., Ding, F.: Gradient based iterative solutions for general linear matrix equations. Comput. Math. Appl. 58(7), 1441–1448 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Xie, L., Liu, Y., Yang, H.: Gradient based and least squares based iterative algorithms for matrix equations \(AXB+CX^TD=F\). Appl. Math. Comput. 217(5), 2191–2199 (2010)

    MathSciNet  MATH  Google Scholar 

  6. Li, M., Liu, X.: Iterative identification methods for a class of bilinear systems by using the particle filtering technique. Int. J. Adapt. Control Signal Process. 35(10), 2056–2074 (2021)

    Article  MathSciNet  Google Scholar 

  7. Ding, F., Chen, T.: Parameter estimation of dual-rate stochastic systems by using an output error method. IEEE Trans. Automat. Control 50(9), 1436–1441 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  8. Li, M., Liu, X.: Maximum likelihood hierarchical least squares-based iterative identification for dual-rate stochastic systems. Int. J. Adapt. Control Signal Process. 35(2), 240–261 (2020)

    Article  MathSciNet  Google Scholar 

  9. Ding, F., Ling, X., Meng, D., Jin, X.-B., Alsaedi, A., Hayat, T.: Gradient estimation algorithms for the parameter identification of bilinear systems using the auxiliary model. J. Comput. Appl. Math. 369, 112575 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  10. Ding, F., Chen, T.: Combined parameter and output estimation of dual-rate systems using an auxiliary model. Automatica 40(10), 1739–1748 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  11. Liu, Y., Ding, F., Shi, Y.: An efficient hierarchical identification method for general dual-rate sampled-data systems. Automatica 50(3), 962–970 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  12. Ding, F., Liu, X., Ma, X.: Kalman state filtering based least squares iterative parameter estimation for observer canonical state space systems using decomposition. J. Comput. Appl. Math. 301, 135–143 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  13. Ding, F., Liu, X.P., Liu, G.: Identification methods for Hammerstein nonlinear systems. Digital Signal Process. 21(2), 215–238 (2011)

    Article  Google Scholar 

  14. Deepho, J., Abubakar, A.B., Malik, M., Argyros, I.K.: Solving unconstrained optimization problems via hybrid cd-dy conjugate gradient methods with applications. J. Comput. Appl. Math. 405, 113823 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chen, L., Ma, Y.: Shamanskii-like Levenberg–Marquardt method with a new line search for systems of nonlinear equations. J. Syst. Sci. Complexity 33(5), 1694–1707 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  16. Chen, L.: A modified Levenberg–Marquardt method with line search for nonlinear equations. Comput. Optim. Appl. 65(3), 753–779 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Levenberg, K.: A method for the solution of certain nonlinear problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    Article  MathSciNet  MATH  Google Scholar 

  18. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  19. Yuan, Y.: Recent advances in trust region algorithms. Math. Program. 151(1), 249–281 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  20. Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt method. In: Alefeld, G., Chen, X. (eds.) Topics in Numerical Analysis: With Special Emphasis on Nonlinear Problems, pp. 239–249. Springer Vienna, Vienna (2001)

    Chapter  MATH  Google Scholar 

  21. Moré, J.J., Garbow, B.S., Hillstrom, K.H.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  22. Ahookhosh, M., Aragón Artacho, F.J., Fleming, Ronan M. T., Vuong, P.T.: Local convergence of the Levenberg–Marquardt method under Hölder metric subregularity. Adv. Comput. Math. 45(5–6), 2771–2806 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  23. Wang, H., Fan, J.: Convergence rate of the Levenberg–Marquardt method under Hölderian local error bound. Optim. Methods Softw. 35(4), 767–786 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  24. Guo, L., Lin, G.-H., Jane, J.Y.: Solving mathematical programs with equilibrium constraints. J. Optim. Theory Appl. 166(1), 234–256 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Zheng, L., Chen, L., Ma, Y.: A variant of the Levenberg–Marquardt method with adaptive parameters for systems of nonlinear equations. AIMS Math. 7(1), 1241–1256 (2022)

    Article  MathSciNet  Google Scholar 

  26. Zheng, L., Chen, L., Tang, Y.: Convergence rate of the modified Levenberg–Marquardt method under Hölderian local error bound. Open Math. 20(1), 998–1012 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  27. Fan, J., Yuan, Y.: On the convergence of a new Levenberg–Marquardt method. In Technical Report, AMSS, Chinese Academy of Sciences (2001)

  28. Dennis, J.E., Jr., Schnable, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Society for Industrial and Applied Mathematics, Philadelphia (1983)

    Google Scholar 

  29. Fischer, A.: Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Program. 94, 91–124 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  30. Ma, C., Jiang, L.: Some research on Levenberg–Marquardt method for the nonlinear equations. Appl. Math. Comput. 184(2), 1032–1040 (2007)

    MathSciNet  MATH  Google Scholar 

  31. Fan, J., Pan, J.: A note on the Levenberg–Marquardt parameter. Appl. Math. Comput. 207(2), 351–359 (2009)

    MathSciNet  MATH  Google Scholar 

  32. Huang, B., Ma, C.: A shamanskii-like self-adaptive Levenberg–Marquardt method for nonlinear equations. Comput. Math. Appl. 77(2), 357–373 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  33. Amini, K., Rostami, F., Caristi, G.: An efficient Levenberg–Marquardt method with a new LM parameter for systems of nonlinear equations. Optimization 67(5), 637–650 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  34. Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for computing the regularization parameter of the Levenberg–Marquardt method. Comput. Optim. Appl. 65(3), 1–29 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  35. Musa, Y.B., Waziri, M.Y., Halilu, A.S.: On computing the regularization parameter for the Levenberg–Marquardt method via the spectral radius approach to solving systems of nonlinear equations. J. Numer. Math. Stochast. 9(1), 80–94 (2017)

    MathSciNet  MATH  Google Scholar 

  36. Fan, J., Yuan, Y.: On the quadratic convergence of the Levenberg–Marquardt method without nonsingularity assumption. Computing 74(1), 23–39 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  37. Chen, L., Ma, Y., Su, C.: An efficient m-step Levenberg–Marquardt method for nonlinear equations. ChinaXiv, p. 15, (2016)

  38. Yuan, Y.-X.: Problems on convergence of unconstrained optimization algorithms. In: Numerical Linear Algebra and Optimization, pp. 95–107. Science Press, Beijing, New York (1999)

    Google Scholar 

  39. Schnabel, R.B., Frank, P.D.: Tensor methods for nonlinear equations. SIAM J. Numer. Anal. 21(5), 815–843 (1984)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research was supported by NFSC Grant 11901061, the University Natural Science Research Project of Anhui Province Grant KJ2020ZD008 and the Natural Science Foundation of Anhui province Grant 2108085MF204. Part of this work was completed during authors’ visit to University of Texas at Arlington. They would like to thank Prof. Ren-Cang Li for his hospitality during the visit.

Author information

Authors and Affiliations

  1. School of Sciences, Changzhou Institute of Technology, Changzhou, 213032, Jiangsu, People’s Republic of China

    Liang Chen

  2. School of Computer Science and Information Engineering, Changzhou Institute of Technology, Changzhou, 213032, Jiangsu, People’s Republic of China

    Yanfang Ma

  3. School of Mathematical Sciences, Huaibei Normal University, Huaibei, 235000, Anhui, People’s Republic of China

    Liang Chen

Authors
  1. Liang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanfang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liang Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no known conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Ma, Y. A modified Levenberg–Marquardt method for solving system of nonlinear equations. J. Appl. Math. Comput. 69, 2019–2040 (2023). https://doi.org/10.1007/s12190-022-01823-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12190-022-01823-x

Keywords

Mathematics Subject Classification

Search

Navigation