Skip to main content
SpringerLink
Log in

A nonmonotone inexact Newton method for unconstrained optimization

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

It is well known that the Newton method has a second order rate of convergence and that it is widely used to solve optimization problems and nonlinear equations which arise from computational science, engineering analysis and other applications. However, two big disadvantages hinder its application: high computational cost for large scale problems and poor global performance in some complicated and difficult problems. Some inexact Newton methods have emerged over time. Among them, the Newton preconditioned conjugate gradient method is the most efficient and popular approach to overcome the first shortcoming while keeping rapid convergence. In this paper, we have improved the global performance of the inexact Newton method by developing a nonmonotone line search technique. We have also proved the global convergence of the proposed method under some conditions. Numerical experiments on a set of standard test problems are reported. They have shown that the proposed algorithm is promising.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ahookhosh, M., Ghaderi, S.: On efficiency of nonmonotone Armijo-type line searches. arXiv preprint arXiv:1408.2675, (2014)

  2. Amini, K., Ahookhosh, M., Nosratipour, H.: An inexact line search approach using modified nonmonotone strategy for unconstrained optimization. Numer. Algorithms 66(1), 49–78 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  3. Dai, Y.H., Zhang, H.C.: Adaptive two-point stepsize gradient algorithm. Numer. Algorithms 27(4), 377–385 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  4. Dai, Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112(2), 315–330 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  5. Dembo, R.S., Eisenstat, S.C., Steihaug, T.: Inexact Newton method. SIAM J. Numer. Anal. 19(2), 400–408 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dembo, R.S., Steihaug, T.: Truncated-Newton algorithms for large-scale unconstrained optimization. Math. Program. 26(2), 190–212 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  7. Deng, N.Y., Xiao, Y., Zhou, F.J.: Nonmonotonic trust-region algorithm. J. Optim. Theory Appl. 76(2), 259–285 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  8. Deng, N.Y., Wang, Z.Z.: Theoretical efficiency of an inexact Newton method. J. Optim. Theory Appl. 105(1), 97–112 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gao, H., Tong, X.J., Dai, Y.H.: Barzilai-Borwein-like methods for the extreme eigenvalue problem. J. Ind. Manag. Optim. 11(3), 999–1019 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  10. Golub, G.H., Van Loan, C.F.: Matrix Computations. JHU Press, Baltimore (2012)

    MATH  Google Scholar 

  11. Gould, N.I.M., Orban, D., Toint, P.L.: CUTEr and SifDec: A constrained and unconstrained testing environment, revisited. ACM T. Math. Software(TOMS). 29(4), 373–394 (2003)

  12. Grippo, L., Lamparillo, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23(4), 707–716 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  13. Grippo, L., Lamparillo, F., Lucidi, S.: Newton-type algorithms with nonmonotone line search for large-scale unconstrained optimization. Springer, Berlin (1988)

    Book  Google Scholar 

  14. Grippo, L., Lamparillo, F., Lucidi, S.: A truncated Newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. 60(3), 401–419 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  15. Grippo, L., Lamparillo, F., Lucidi, S.: A class of nonmonotone stabilization methods in unconstrained optimization. Numer. Math. 59(1), 779–805 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  16. Jiang, B., Dai, Y.H.: Feasible Barzilai-Borwein-like methods for extreme symmetric eigenvalue problems. Optim. Method Softw. 28(4), 756–784 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Liu, G., Han, J., Sun, D.: Global convergece of the BFGS algorithm with nonmonotone linesearch. Optimization 34(2), 147–159 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  18. Lu, Z.S., Xiao, L.: Randomized block coordinate non-monotone gradient method for a class of nonlinear programming. (2013). arXiv:1306.5918

  19. Saad, Y.: Iterative methods for sparse linear systems. SIAM (2003)

  20. Toint, P.L.: Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints. Math. Program. 77(3), 69–94 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  21. Toint, P.L.: Towards an efficient sparsity exploiting Newton method for minimization. Sparse matrices and their uses. Academic Press, London (1981)

    MATH  Google Scholar 

  22. Zhang, H.B., Deng, N.Y.: An improved inexact Newton method. J. Glob. Optim. 39(2), 221–234 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  23. Zhu, D.: Nonmonotone backtracking inexact quasi-Newton algorithms for solving smooth nonlinear equations. Appl. Math. Comput. 161(3), 875–895 (2005)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Prof. Yu-Hong Dai from Chinese Academy of Sciences for discussion on this paper and careful checking on an early version of this manuscript, and they would like to thank Piaoyang Qi for his help on the numerical experiments. The authors also thank the anonymous referees very for their patient and valuable comments, which improved the quality of this paper greatly. This work was supported by National Natural Science Foundation of China (Grant No. 61179033), by Collaborative Innovation Center on Beijing Society-building and Social Governance and by China Postdoctoral Science Foundation (No. 2014M561081)

Author information

Authors and Affiliations

  1. College of Applied Science, Beijing University of Technology, Beijing, China

    Huan Gao & Hai-Bin Zhang

  2. State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China

    Zhi-Bao Li

  3. Computer Science and Software Engineering, Xi’an Jiaotong-Liverpool University, SIP, Suzhou, China

    Emmanuel Tadjouddine

Authors
  1. Huan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai-Bin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhi-Bao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emmanuel Tadjouddine
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huan Gao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, H., Zhang, HB., Li, ZB. et al. A nonmonotone inexact Newton method for unconstrained optimization. Optim Lett 11, 947–965 (2017). https://doi.org/10.1007/s11590-015-0976-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-015-0976-2

Keywords

Search

Navigation