Skip to main content
Springer Nature Link
Log in

New Hybrid Conjugate Gradient and Broyden–Fletcher–Goldfarb–Shanno Conjugate Gradient Methods

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

Three hybrid methods for solving unconstrained optimization problems are introduced. These methods are defined using proper combinations of the search directions and included parameters in conjugate gradient and quasi-Newton methods. The convergence of proposed methods with the underlying backtracking line search is analyzed for general objective functions and particularly for uniformly convex objective functions. Numerical experiments show the superiority of the proposed methods with respect to some existing methods in view of the Dolan and Moré’s performance profile.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  2. Fletcher, R.: Practical Methods of Optimization. Unconstrained Optimization, vol. 1. Wiley, New York (1987)

    MATH  Google Scholar 

  3. Dai, Y.H., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  4. Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  5. Polak, E., Ribiere, G.: Note sur la convergence des mthodes de directions conjuguées. Rev. Francaise Imformat Recherche Opertionelle 16, 35–43 (1969)

    MATH  Google Scholar 

  6. Polyak, B.T.: The conjugate gradient method in extreme problems. U.S.S.R. Comput. Math. Math. Phys. 9, 94–112 (1969)

    Article  MATH  Google Scholar 

  7. Liu, Y., Storey, C.: Efficient generalized conjugate gradient algorithms, part 1: theory. J. Optim. Theory Appl. 69(1), 129–137 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  8. Yang, X., Luo, Z., Dai, X.: A global convergence of LS-CD hybrid conjugate gradient method. In: Advances in Numerical Analysis 2013. Hindawi Publishing Corporation, Article ID 517452, 5 pp. (2013)

  9. Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. Zheng, Y., Zheng, B.: Two new Dai–Liao-type conjugate gradient methods for unconstrained optimization problems. J. Optim. Theory Appl. 175, 502–509 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  11. Andrei, N.: An acceleration of gradient descent algorithm with backtracking for unconstrained optimization. Numer. Algorithms 42, 63–73 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  12. Stanimirović, P.S., Miladinović, M.B.: Accelerated gradient descent methods with line search. Numer. Algorithms 54, 503–520 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Touati-Ahmed, D., Storey, C.: Efficient hybrid conjugate gradient techniques. J. Optim. Theory Appl. 64(2), 379–397 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  14. Zhang, L., Zhou, W.: Two descent hybrid conjugate gradient methods for optimization. J. Comput. Appl. Math. 216, 251–264 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2(1), 21–42 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  16. Dai, Y.H., Yuan, Y.: An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 33–47 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  17. Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  18. Zhang, L., Zhou, W.J., Li, D.H.: A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 26, 629–640 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhang, L., Zhou, W.J., Li, D.H.: Global convergence of a modified Fletcher–Reeves conjugate method with Armijo-type line search. Numer. Math. 104, 561–572 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  20. Zhang, L.: Nonlinear Conjugate Gradient Methods for Optimization Problems. Ph.D. Thesis, College of Mathematics and Econometrics, Hunan University, Changsha, China (2006)

  21. Ibrahim, M.A.H., Mamat, M., Leong, W.J.: The hybrid BFGS-CG method in solving unconstrained optimization problems. In: Abstract and Applied Analysis 2014. Hindawi Publishing Corporation. Article ID 507102, 6 pp. (2014)

  22. Zoutendijk, G.: Nonlinear programming, computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland, Amsterdam (1970)

    Google Scholar 

  23. Cheng, W.: A two-term PRP-based descent method. Numer. Funct. Anal. Optim. 28(11–12), 1217–1230 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Luo, Y.Z., Tang, G.J., Zhou, L.N.: Hybrid approach for solving systems of nonlinear equations using chaos optimization and quasi-Newton method. Appl. Soft Comput. 8(2), 1068–1073 (2008)

    Article  Google Scholar 

  25. Han, L., Neumann, M.: Combining quasi-Newton and Cauchy directions. Int. J. Appl. Math. 12(2), 167–191 (2003)

    MathSciNet  MATH  Google Scholar 

  26. Baluch, B., Salleh, Z., Alhawarat, A., Roslan, U.A.M.: A new modified three-term conjugate gradient method with sufficient descent property and its global convergence. J. Math. 2017, Article ID 2715854, 12 pp. (2017)

  27. Khanaiah, Z., Hmod, G.: Novel hybrid algorithm in solving unconstrained optimizations problems. Int. J. Novel Res. Phys. Chem. Math. 4(3), 36–42 (2017)

    Google Scholar 

  28. Osman, W.F.H.W., Ibrahim, M.A.H., Mamat, M.: Hybrid DFP-CG method for solving unconstrained optimization problems. J. Phys. Conf. Ser. 890(2017), 012033 (2017)

    Article  Google Scholar 

  29. Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26(3), 727–739 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  30. Andrei, N.: An Unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)

    MathSciNet  MATH  Google Scholar 

  31. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The first author gratefully acknowledge support from the Research Project 174013 of the Serbian Ministry of Science.

Author information

Authors and Affiliations

  1. Faculty of Sciences and Mathematics, University of Niš, Višegradska 33, 18 000, Nis, Serbia

    Predrag S. Stanimirović & Ivona Brajević

  2. Technical Faculty in Bor, University of Belgrade, Vojske Jugoslavije 12, 19 210, Bor, Serbia

    Branislav Ivanov

  3. Faculty of Technology in Leskovac, University of Niš, Bulevar Oslobodjenja 124, 16 000, Leskovac, Serbia

    Snežana Djordjević

Authors
  1. Predrag S. Stanimirović
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Branislav Ivanov
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Snežana Djordjević
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ivona Brajević
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Predrag S. Stanimirović.

Additional information

Communicated by Ilio Galligani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stanimirović, P.S., Ivanov, B., Djordjević, S. et al. New Hybrid Conjugate Gradient and Broyden–Fletcher–Goldfarb–Shanno Conjugate Gradient Methods. J Optim Theory Appl 178, 860–884 (2018). https://doi.org/10.1007/s10957-018-1324-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-018-1324-3

Keywords

Mathematics Subject Classification