Skip to main content
SpringerLink
Log in

A Modified Spectral Conjugate Gradient Method with Global Convergence

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, a modified version of the spectral conjugate gradient algorithm suggested by Jian, Chen, Jiang, Zeng and Yin is proposed. It is proved that the new method is globally convergent for general nonlinear functions, under some standard assumptions. Based on the modified secant condition and quasi-Newton directions, some new spectral parameters are introduced. It is shown that the search direction satisfies the sufficient descent property independent of the line search. Numerical experiments indicate a promising behavior of the new algorithm, especially for large-scale problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  2. Fletcher, R., Reeves, C.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  3. Polak, E., Ribiére, G.: Note sur la convergence des mthodes de directions conjugèes. Rev. Fr. Inf. Rech. Oper. 16, 35–43 (1969)

    MATH  Google Scholar 

  4. Polyak, B.T.: The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 9, 94–112 (1969)

    Article  MATH  Google Scholar 

  5. Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  6. Fletcher, R.: Practical Methods of Optimization. Unconstrained Optimization, vol. 1. Wiley, New York (1987)

    MATH  Google Scholar 

  7. Dai, Y.H., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  8. Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  9. Dai, Y., Kou, C.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23(1), 296–320 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  10. Yu, G.H., Guan, L.T., Chen, W.F.: Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization. Optim. Methods. Softw. 23(2), 275–293 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  11. Jian, J., Chen, Q., Jiang, X., Zeng, Y., Yin, J.: A new spectral conjugate gradient method for large-scale unconstrained optimization. Optim. Methods. Softw. 32(3), 503–515 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  12. Amini, K., Faramarzi, P., Pirfalah, N.: A modified Hestenes–Stiefel conjugate gradient method with an optimal property. Optim. Methods. Softw. (2018). https://doi.org/10.1080/10556788.2018.1457150

    MATH  Google Scholar 

  13. Dong, X.L., Han, D., Dai, Zh, Li, L., Zhu, J.: An accelerated three-term conjugate gradient method with sufficient descent condition and conjugacy condition. J. Optim. Theory Appl. (2018). https://doi.org/10.1007/s10957-018-1377-3

    MathSciNet  MATH  Google Scholar 

  14. Liu, H., Liu, Z.: An efficient Barzilai–Borwein conjugate gradient method for unconstrained optimization. J. Optim. Theory Appl. (2018). https://doi.org/10.1007/s10957-018-1393-3

    MATH  Google Scholar 

  15. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  16. Raydan, M.: The Barzilain and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  17. Birgin, E.G., Martínez, J.M.: A spectral conjugate gradient method for unconstrained optimization. Appl. Math. Optim. 43, 117–128 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  18. Perry, A.: A modified conjugate gradient algorithm. Oper. Res. 26, 1073–1078 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  19. Andrei, N.: A scaled BFGS preconditioned conjugate gradient algorithm for unconstrained optimization. Appl. Math. Lett. 20, 645–650 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. Andrei, N.: Another hybrid conjugate gradient algorithm for unconstrained optimization. Numer. Algorithms. 47, 143–156 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  21. Andrei, N.: Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization. Eur. J. Oper. Res. 204(3), 410–420 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  22. Andrei, N.: New accelerated conjugate gradient algorithms as a modification of Dai-Yuan’s computational scheme for unconstrained optimization. J. Comput. Appl. Math. 234, 3397–3410 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  23. Babaie-Kafaki, S., Mahdavi-Amiri, N.: Two modifed hybrid conjugate gradient methods based on a hybrid secant equation. Math. Model. Anal. 18(1), 32–52 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  24. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2000)

    MATH  Google Scholar 

  25. Hager, W.W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2(1), 335–358 (2006)

    MathSciNet  MATH  Google Scholar 

  26. Zoutendijk, G.: Nonlinear programming, computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland Publishing Company, Amsterdam (1970)

    Google Scholar 

  27. Powell, M.J.D.: Non-convex minimization calculations and the conjugate gradient method. In: Griffiths, D.F. (ed.) Numerical Analysis, Lecture Notes in Mathematics 1066, pp. 122–141. Springer, Berlin (1984)

    Google Scholar 

  28. Al-Baali, M.: Descent property and global convergence of the Fletcher–Reeves method with inexact line search. IMA J. Numer. Anal. 5(1), 121–124 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  29. Fatemi, M.: An optimal parameter for Dai–Liao family of conjugate gradient methods. J. Optim. Theory Appl. 169, 587–605 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  30. Zheng, Y., Zheng, B.: Two new Dai–Liao-type conjugate gradient methods for unconstrained optimization problems. J. Optim. Theory Appl. 175, 502–509 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  31. Andrei, N.: A Dai–Liao conjugate gradient algorithm with clustering of eigenvalues. Numer. Algorithms 77, 1273–1282 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  32. Aminifard, Z., Babaie-Kafaki, S.: An optimal parameter choice for the Dai–Liao family of conjugate gradient methods by avoiding a direction of the maximum magnification by the search direction matrix. 4OR Q. J. Oper. Res. (2018). https://doi.org/10.1007/s10288-018-0387-1

    Google Scholar 

  33. Li, D.H., Fukushima, M.: A modified BFGS method and its global convergence in non-convex minimization. J. Comput. Appl. Math. 129(1–2), 15–35 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  34. Zhou, W., Zhang, L.: A nonlinear conjugate gradient method based on the MBFGS secant condition. Optim. Methods Softw. 21(5), 707–714 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  35. Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2(1), 21–42 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  36. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)

    MathSciNet  MATH  Google Scholar 

  37. Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, PhL: CUTE: Constrained and unconstrained testing environments. ACM Trans. Math. Softw. 21, 123–160 (1995)

    Article  MATH  Google Scholar 

  38. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program 91(2), 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the anonymous referees and editor for suggestions and comments during the preparation of the paper.

Author information

Authors and Affiliations

  1. Mathematics, Faculty of Science, Razi University, Kermanshah, Iran

    Parvaneh Faramarzi & Keyvan Amini

Authors
  1. Parvaneh Faramarzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keyvan Amini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Keyvan Amini.

Additional information

Alexandre Cabot.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Faramarzi, P., Amini, K. A Modified Spectral Conjugate Gradient Method with Global Convergence. J Optim Theory Appl 182, 667–690 (2019). https://doi.org/10.1007/s10957-019-01527-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-019-01527-6

Keywords

Mathematics Subject Classification

Search

Navigation