Abstract
This paper presents a class of Levenberg–Marquardt methods for solving the nonlinear least-squares problem. Explicit algebraic rules for computing the regularization parameter are devised. In addition, convergence properties of this class of methods are analyzed. We prove that all accumulation points of the generated sequence are stationary. Moreover, q-quadratic convergence for the zero-residual problem is obtained under an error bound condition. Illustrative numerical experiments with encouraging results are presented.
Similar content being viewed by others
References
Birgin, E.G., Gentil, J.M.: Evaluating bound-constrained minimization software. Comput. Optim. Appl. 53(2), 347–373 (2012)
Dan, H., Yamashita, N., Fukushima, M.: Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions. Optim. Methods Softw. 17(4), 605–626 (2002)
Dennis Jr., J.E., Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations. In: Classics in Applied Mathematics, vol 16. Society for Industrial and Applied Mathematics (SIAM). Philadelphia, PA (1996). Corrected reprint of the 1983 original
Fan, J.: Accelerating the modified Levenberg-Marquardt method for nonlinear equations. Math. Compd. 83(287), 1173–1187 (2014)
Fan, J., Pan, J.: Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition. Comput. Optim. Appl. 34(1), 47–62 (2006)
Fan, J., Yuan, Y.-X.: On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption. Computing 74(1), 23–39 (2005)
Fischer, A.: Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Progr. 94((1, Ser. A)), 91–124 (2002)
Fischer, A., Shukla, P.K., Wang, M.: On the inexactness level of robust Levenberg-Marquardt methods. Optimization 59(2), 273–287 (2010)
Gould, N., Orban, D., Toint, P.: Cutest: a constrained and unconstrained testing environment with safe threads. Technical Report RAL-TR-2013-005. Rutherford Appleton Laboratory, Oxfordshire (2013)
Gratton, S., Lawless, A.S., Nichols, N.K.: Approximate Gauss-Newton methods for nonlinear least squares problems. SIAM J. Optim. 18(1), 106–132 (2007)
Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for quadratic regularization of Newton’s method. Comput. Optim. Appl. 60, 343–376 (2014)
Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quart. Appl. Math. 2, 164–168 (1944)
Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)
Moré, J.J.: The Levenberg-Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.), Lecture Notes in Mathematics 630: Numerical Analysis, pp. 105–116. Springer, New York (1978)
Morrison, D.D.: Methods for nonlinear least squares problems and convergence proofs. In: Lorell, J., Yagi, F. (eds.), Proceedings of the Seminar on Tracking Programs and Orbit Determination, pp. 1–9. Jet Propulsion Laboratory, Pasadena (1960)
Ueda, K., Yamashita, N.: On a global complexity bound on the Levenberg-Marquardt method. J. Optimiz. Theory App. 147, 443–453 (2010)
Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg-Marquardt method. In: Alefeld, G., Chen, X. (eds.) Topics in Numerical Analysis: With Special Emphasis on Nonlinear Problems, Computing Supplement, vol. 15, pp. 239–249. Springer, Vienna (2001)
Yuan, Y.-X.: Recent advances in numerical methods for nonlinear equations and nonlinear least squares. Numer. Algebra Control Optim. 1(1), 15–34 (2011)
Acknowledgments
The authors are thankful to Abel S. Siqueira for installing the CUTEst and preparing our machine to run it within Matlab. We are also grateful to the anonymous referees whose suggestions led to improvements in the paper. Elizabeth W. Karas was partially supported by CNPq Grants 477611/2013-3 and 308957/2014-8. Sandra A. Santos was partially supported by CNPq Grant 304032/2010-7, FAPESP Grants 2013/05475-7 and 2013/07375-0 and PRONEX Optimization. Benar F. Svaiter was partially supported by CNPq Grants 302962/2011-5, 474944/2010-7, FAPERJ Grant E-26/102.940/2011 and PRONEX Optimization.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
The complete computational results are presented next. The outcomes of the four solvers CH, QR, FP and LM, respectively, are displayed row by row for each problem.
Rights and permissions
About this article
Cite this article
Karas, E.W., Santos, S.A. & Svaiter, B.F. Algebraic rules for computing the regularization parameter of the Levenberg–Marquardt method. Comput Optim Appl 65, 723–751 (2016). https://doi.org/10.1007/s10589-016-9845-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-016-9845-x
Keywords
- Nonlinear least-squares problems
- Levenberg–Marquardt method
- Regularization
- Global convergence
- Local convergence
- Computational results