Skip to main content
SpringerLink
Log in

A nonmonotone line search method for noisy minimization

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

A nonmonotone line search method for optimization in noisy environment is proposed. The method is defined for arbitrary search directions and uses only the noisy function values. Convergence of the proposed method is established under a set of standard assumptions. The computational issues are considered and the presented numerical results affirm that nonmonotone strategies are worth considering. Four different line search rules with three different directions are compared numerically. The influence of nonmonotonicity is discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Andradottir, S.: A scaled stochastic approximation algorithm. Manag. Sci. 42(4), 475–498 (1996)

    Article  MATH  Google Scholar 

  2. Barzilai, J., Borwein, J.M.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  3. Billups, S.C., Larson, J., Graf, P.: Derivative-free optimization of expensive functions with computational error using weighted regression. SIAM J. Optim. 23(1), 27–53 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  4. Birgin, E.G., Krejić, N., Martínez, J.M.: Globally convergent inexact quasi-Newton methods for solving nonlinear systems. Numer. Algorithms 32(2–4), 249–260 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  5. Cheng, W., Li, D.H.: A derivative-free nonmonotone line searh and its application to the spectral residual method. IMA J. Numer. Anal. 29, 814–825 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cruz, W.L., Martinez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Math. Comput. 75(255), 1429–1448 (2006)

    Article  MATH  Google Scholar 

  7. Dennis, J.E. Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, Philadelphia (1996)

  8. Diniz-Ehrhardt, M.A., Martínez, J.M., Raydan, M.: A derivative free nonmonotone line-search technique for unconstrained optimization. J. Comput. Appl. Math. 219(2), 383–397 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  9. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Progr. Ser. A 91, 201–213 (2002)

    Article  MATH  Google Scholar 

  10. Fu, M.C.: Gradient estimation. In: Henderson, S.G., Nelson, B.L. (eds.) Handbook in OR & MS, vol. 13, pp. 575–616. Elsevier, North Holland (2006)

  11. Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  12. Kelley, C.T.: Iterative Methods for Optimization. SIAM, Philadelphia (1999)

    Book  MATH  Google Scholar 

  13. Krejić, N., Krklec Jerinkić, N.: Nonmonotone line search methods with variable sample size. Numer. Algorithms (2014). doi:10.1007/s11075-014-9869-1

  14. Krejić, N., Luz̆anin, J., Stojkovska, I.: A gradient method for unconstrained optimization in noisy environment. Appl. Numer. Math. 70, 1–21 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  15. Krejić, N., Rapajić, S.: Gllobaly convergent Jacobian smoothing inexact Newton methods for NCP. Comput. Optim. Appl. 41(2), 243–261 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  16. Li, D.H., Fukushima, M.: A derivative-free line search and global convergence of Broyden-like method for nonlinear equations. Optim. Methods Softw. 13, 181–201 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  17. Lucidi, S., Sciandrone, M.: A derivative-free algorithm for bound constrained optimization. Comput. Optim. Appl. 21, 119–142 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  18. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)

    Article  MATH  Google Scholar 

  19. Nikolovski, F., Stojkovska, I.: New derivative-free nonmonotone line search methods for unconstrained minimization. In: Proceedings of the Fifth International Scientific Conference—FMNS2013, Mathematics and Informatics, vol. 1, pp. 47–53. South-West University “Neofit Rilski”, Blagoevgrad, Bulgaria (2013)

  20. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  21. Powell, M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Large Scale Nonlinear Optimization, Nonconvex Optimization and Its Applications, pp. 255–297. Springer, US (2006)

  22. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  23. Spall, J.C.: Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. Wiley, Hoboken (2003)

    Book  Google Scholar 

  24. Xu, Z., Dai, Y.-H.: New stochastic approximation algorithms with adaptive step size. Optim. Lett. 6(8), 1831–1846 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  25. Yu, Z., Pu, D.: A new nonmonotone line search technique for unconstrained optimization. J. Comput. Appl. Math. 219, 134–144 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  26. Zhang, B., Zhu, Z., Li, S.: A modified spectral conjugate gradient projection algorithm for total variation image restoration. Appl. Math. Lett. 27, 26–35 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  27. Zhang, H., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14(4), 1043–1056 (2004)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

We are grateful to the anonymous referee whose comments helped us to improve the paper.

Author information

Authors and Affiliations

  1. Department of Mathematics and Informatics, Faculty of Science, University of Novi Sad, Trg Dositeja Obradovića 4, 21000, Novi Sad, Serbia

    Nataša Krejić & Zorana Lužanin

  2. Department of Mathematics, Faculty of Natural Sciences and Mathematics, Ss. Cyril and Methodius University, Arhimedova 3, 1000, Skopje, Macedonia

    Filip Nikolovski & Irena Stojkovska

Authors
  1. Nataša Krejić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zorana Lužanin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Filip Nikolovski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Irena Stojkovska
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irena Stojkovska.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krejić, N., Lužanin, Z., Nikolovski, F. et al. A nonmonotone line search method for noisy minimization. Optim Lett 9, 1371–1391 (2015). https://doi.org/10.1007/s11590-015-0848-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-015-0848-9

Keywords

Mathematics Subject Classification

Search

Navigation