Skip to main content
SpringerLink
Log in

Nonlinear Stepsize Control Algorithms: Complexity Bounds for First- and Second-Order Optimality

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

A nonlinear stepsize control (NSC) framework has been proposed by Toint (Optim Methods Softw 28:82–95, 2013) for unconstrained optimization, generalizing several trust-region and regularization algorithms. More recently, worst-case complexity bounds to achieve approximate first-order optimality were proved by Grapiglia, Yuan and Yuan (Math Program 152:491–520, 2015) for the generic NSC framework. In this paper, improved complexity bounds for first-order optimality are obtained. Furthermore, complexity bounds for second-order optimality are also provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. The inequality in (6) typically results from an error bound on Taylor series and a bounded \(H_{k}\).

  2. For details, see Section 2 in [3].

References

  1. Shultz, G.A., Schnabel, R.B., Byrd, R.H.: A family of trust-region-based algorithms for unconstrained minimization with strong global convergence properties. SIAM J. Numer. Anal. 22, 47–67 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  2. Toint, PhL: Global convergence of a class of trust region methods for nonconvex minimization in Hilbert space. IMA J. Numer. Anal. 8, 231–252 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  3. Toint, PhL: Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization. Optim. Methods Softw. 28, 82–95 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  4. Conn, A.R., Gould, N.I.M., Toint, PhL: Trust-Region Methods. SIAM, Philadelphia (2000)

    Book  MATH  Google Scholar 

  5. Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming 2. Academic Press, New York (1975)

    Google Scholar 

  6. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic overestimation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127, 245–295 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function-and derivative-evaluation complexity. Math. Program. 130, 295–319 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Nesterov, Yu.: Modified Gauss–Newton scheme with worst-case guarantees for global performance. Optim. Methods Softw. 22, 469–483 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bellavia, S., Cartis, C., Gould, N.I.M., Morini, B., Toint, PhL: Convergence of a regularized euclidean residual algorithm for nonlinear least-squares. SIAM J. Numer. Anal. 48, 1–29 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  10. Fan, J., Yuan, Y.: A new trust region algorithm with trust region radius converging to zero. In: Li, D. (ed.) Proceeding of the 5th International Conference on Optimization: Techiniques and Applications, pp. 786–794. Hong Kong (2001)

  11. Zhang, J., Wang, Y.: A new trust region method for nonlinear equations. Math. Methods Oper. Res. 58, 283–298 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  12. Fan, J.: Convergence rate of the trust region method for nonlinear equations under local error bound condition. Comput. Optim. Appl. 34, 215–227 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Lu, X., Ni, Q.: A quasi-Newton trust-region method with a new conic model for the unconstrained optimization. Appl. Math. Comput. 204, 373–384 (2008)

    MathSciNet  MATH  Google Scholar 

  14. Grapiglia, G.N., Yuan, J., Yuan, Y.: On the worst-case complexity of nonlinear stepsize control algorithms for convex unconstrained optimization. Optim. Methods Softw. 31, 591–604 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  15. Grapiglia, G.N., Yuan, J., Yuan, Y.: On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization. Math. Program. 152, 491–520 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  16. Powell, M.J.D.: On the global convergence of trust region algorithms for unconstrained minimization. Math. Program. 29, 297–303 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  17. Cartis, C., Gould, N.I.M., Toint, P.L.: Worst-case evaluation complexity of regularization methods for smooth unconstrained optimization using Hölder continuous gradients. Technical Report NA-14-21, Mathematical Institute, University of Oxford (2014)

  18. Cartis, C., Gould, N.I.M., Toint, PhL: Complexity bounds for second-order optimality in unconstrained optimization. J. Complex. 28, 93–108 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  19. Nesterov, Yu.: Introductory Lectures on Convex Optimization. Applied Optimization. Kluwer, Dordrecht (2004)

    Book  MATH  Google Scholar 

  20. Gratton, S., Sartenaer, A., Toint, Ph L.: Recursive trust-region methods for multiscale nonlinear optimization. SIAM J. Optim. 19, 414–444 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  21. Cartis, C., Sampaio, PhR, Toint, PhL: Worst-case evaluation complexity of non-monotone gradient-related algorithms for unconstrained optimization. Optimization 64, 1349–1361 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. Ueda, K., Yamashita, N.: On a global complexity bound of the Levenberg–Marquardt method. J. Optim. Theory Appl. 147, 443–453 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  23. Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108, 177–205 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  24. Curtis, F.E., Robinson, D.P., Samadi, M.: A trust region algorithm with a worst-case iteration complexity of \(O(\epsilon ^{-3/2})\) for nonconvex optimization. Math. Program. doi:10.1007/s10107-016-1026-2

  25. Martínez, J.M., Raydan, M.: Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization (2015). http://www.optimization-online.org/DB_FILE/2015/11/5203.pdf

  26. Birgin, E.G., Gardenghi, J.L., Martínez, J.M., Toint, Ph.L.: Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models. Technical Report naXys-05-2015, Namur Center for Complex Systems (naXys), University of Namur, Namur, Belgium (2015)

Download references

Acknowledgments

The authors are grateful to two anonymous referees for their valuable comments and suggestions to improve the paper. G.N. Grapiglia was supported by the Coordination for Improvement of Higher Education Personnel (CAPES), Brazil. J. Yuan was partially supported by the Coordination of the Improvement of Higher Education Personnel (CAPES) and by the National Council for Scientific and Technological Development (CNPq), Brazil. Y. Yuan was partially supported by the Natural Science Foundation of China (NSFC), China.

Author information

Authors and Affiliations

  1. Departamento de Matemática, Universidade Federal do Paraná, Centro Politécnico, Cx. Postal 19.081, Curitiba, Paraná, 81531-980, Brazil

    Geovani Nunes Grapiglia & Jinyun Yuan

  2. State Key Laboratory of Scientific/Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Zhong Guan Cun Donglu 55, Beijing, 100190, People’s Republic of China

    Ya-xiang Yuan

Authors
  1. Geovani Nunes Grapiglia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinyun Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ya-xiang Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geovani Nunes Grapiglia.

Additional information

Communicated by Johannes O. Royset.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Grapiglia, G.N., Yuan, J. & Yuan, Yx. Nonlinear Stepsize Control Algorithms: Complexity Bounds for First- and Second-Order Optimality. J Optim Theory Appl 171, 980–997 (2016). https://doi.org/10.1007/s10957-016-1007-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-016-1007-x

Keywords

Mathematics Subject Classification

Search

Navigation