Abstract
We propose an adaptive multi-step Levenberg–Marquardt (LM) method for nonlinear equations. The adaptive scheme can decide automatically whether an iteration should evaluate the Jacobian matrix at the current iterate to compute an LM step, or use the latest evaluated Jacobian to compute an approximate LM step, so that not only the Jacobian evaluation but also the linear algebra work can be saved. It is shown that the adaptive multi-step LM method converges superlinearly under the local error bound condition, which does not require the full column rank of the Jacobian at the solution. Numerical experiments demonstrate the efficiency of the adaptive multi-step LM method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Behling, R., Iusem, A.: The effect of calmness on the solution set of systems of nonlinear equations. Math. Program. 137, 155–165 (2013)
Fan, J.Y.: The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence. Math. Comput. 81, 447–466 (2012)
Fan, J.Y.: A Shamanskii-like Levenberg–Marquardt method for nonlinear equations. Comput. Optim. Appl. 56, 63–80 (2013)
Fan, J.Y., Yuan, Y.X.: On the quadratic convergence of the Levenberg–Marquardt method without nonsingularity assumption. Computing 74, 23–39 (2005)
Kelley, C.T.: Solving Nonlinear Equations with Newton’s Method. Fundamentals of Algorithms. SIAM, Philadelphia (2003)
Levenberg, K.: A method for the solution of certain nonlinear problems in least squares. Q. Appl. Math. 2, 164–166 (1944)
Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear inequalities. SIAM J. Appl. Math. 11, 431–441 (1963)
Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. (TOMS) 7, 17–41 (1981)
Moré, J.J.: The Levenberg–Marquardt Algorithm: Implementation and Theory. In: Watson, G.A. (ed.) Lecture Notes in Mathematics 630: Numerical Analysis, pp. 105–116. Springer, Berlin (1978)
Powell, M.J.D.: Convergence properties of a class of minimization algorithms. Nonlinear Program. 2, 1–27 (1975)
Schnabel, R.B., Frank, P.D.: Tensor methods for nonlinear equations. SIAM J. Numer. Anal. 21, 815–843 (1984)
Stewart, G.W., Sun, J.-G.: Matrix Perturbation Theory, (Computer Science and Scientific Computing). Academic Press, Boston (1990)
Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt method. Comput. Suppl. 15, 237–249 (2001)
Yuan, Y.X.: Recent advances in trust region algorithms. Math. Program. Ser. B 151, 249–281 (2015)
Author information
Authors and Affiliations
Corresponding author
Additional information
The first author was supported in part by NSFC Grant 11571234. The third author was supported in part by NSFC Grant 11371145 and 11771148, and Science and Technology Commission of Shanghai Municipality Grant 13dz2260400.
Rights and permissions
About this article
Cite this article
Fan, J., Huang, J. & Pan, J. An Adaptive Multi-step Levenberg–Marquardt Method. J Sci Comput 78, 531–548 (2019). https://doi.org/10.1007/s10915-018-0777-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10915-018-0777-8