Skip to main content
SpringerLink
Log in

A new family of hybrid three-term conjugate gradient methods with applications in image restoration

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper, based on the hybrid conjugate gradient method and the convex combination technique, a new family of hybrid three-term conjugate gradient methods are proposed for solving unconstrained optimization. The conjugate parameter in the search direction is a hybrid of Dai-Yuan conjugate parameter and any one. The search direction then is the sum of the negative gradient direction and a convex combination in relation to the last search direction and the gradient at the previous iteration. Without choosing any specific conjugate parameters, we show that the search direction generated by the family always possesses the descent property independent of line search technique, and that it is globally convergent under usual assumptions and the weak Wolfe line search. To verify the effectiveness of the presented family, we further design a specific conjugate parameter, and perform medium-large-scale numerical experiments for smooth unconstrained optimization and image restoration problems. The numerical results show the encouraging efficiency and applicability of the proposed methods even compared with the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. All codes are available at https://github.com/jhyin-optim/FHTTCGMs_with_applications

References

  1. Hestenes, M.R., Stiefel, E.: Method of conjugate gradient for solving linear equations. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)

    Article  MATH  Google Scholar 

  2. Fletcher, R., Reeves, C.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  3. Polak, E., Ribière, G.: Note surla convergence de directions conjugèes. Rev. Fr. Informat Rech. Operationelle 3e Anneè 16(3), 35–43 (1969)

    MATH  Google Scholar 

  4. Polyak, B.T.: The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 9, 94–112 (1969)

    Article  MATH  Google Scholar 

  5. Fletcher, R.: Unconstrained Optimization Practical Methods of Optimization, vol. 1. Wiley, New York (1987)

    MATH  Google Scholar 

  6. Liu, Y., Storey, C.: Efficient generalized conjugate gradient algorithms, part 1: theory. J. Optim. Theory Appl. 69(1), 129–137 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  7. Dai, Y.H., Yuan, Y.X.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  9. Hager, W.W., Zhang, H.C.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  10. Hager, W.W., Zhang, H.C.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2(1), 35–58 (2006)

    MathSciNet  MATH  Google Scholar 

  11. Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23(1), 296–320 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  12. Andrei, N.: Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization. Eur. J. Oper. Res. 204(3), 410–420 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Birgin, E.G., Martínez, J.M.: A spectral conjugate gradient method for unconstrained optimization. Appl. Math. Optim. 43(2), 117–128 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  14. Li, M.: A three term Polak-Ribière-Polyak conjugate gradient method close to the memoryless BFGS quasi-Newton method. J. Ind. Manag. Optim. 16(1), 245–260 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  15. Babaie-Kafaki, S.: A modified three-term conjugate gradient method with sufficient descent property. Appl. Math.: J. Chin. Univ. (Ser. B) 30(03), 263–272 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  16. Faramarzi, P., Amini, K.: A scaled three-term conjugate gradient method for large-scale unconstrained optimization problem. Calcolo 56(4), 1–15 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  17. Liu, J.K., Du, S.Q., Chen, Y.Y.: A sufficient descent nonlinear conjugate gradient method for solving M-tensor equations. J. Comput. Appl. Math. 371, 112709 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  18. Ziadi, R., Ellaia, R., Bencherif-Madani, A.: Global optimization through a stochastic perturbation of the Polak-Ribière conjugate gradient method. J. Comput. Appl. Math. 317, 672–684 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhu, X.J.: A Riemannian conjugate gradient method for optimization on the Stiefel manifold. Comput. Optim. Appl. 67(1), 73–110 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  20. Liu, J.K., Du, X.L.: A gradient projection method for the sparse signal reconstruction in compressive sensing. Appl. Anal. 97(12), 2122–2131 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  21. Chen, X.J., Zhou, W.J.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3(4), 765–790 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  22. Yin, J.H., Jian, J.B., Jiang, X.Z., et al.: A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numer. Algorithms 88(1), 389–418 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  23. Xiao, Y., Wang, Q., Hu, Q.: Non-smooth equations based method for ℓ1-norm problems with applications to compressed sensing. Nonlinear Anal. Theory Methods Appl. 74(11), 3570–3577 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  24. Xiao, Y., Zhu, H.: A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405(1), 310–319 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  25. Yuan, G.L., Li, T.T., Hu, W.J.: A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Appl. Numer. Math. 147, 129–141 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  26. Yin, J., Jian, J., Jiang, X.: A generalized hybrid CGPM-based algorithm for solving large-scale convex constrained equations with applications to image restoration. J. Comput. Appl. Math. 391, 113423 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  27. Liu, Y.F., Zhu, Z.B., Zhang, B.X.: Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing. J. Appl. Math. Comput. https://doi.org/10.1007/s12190-021-01589-8 (2021)

  28. Shi, Z.J., Guo, J.: A new family of conjugate gradient methods. J. Comput. Appl. Math. 224(1), 444–457 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  29. Beale, E.M.: A Derivation of Conjugate Gradients. Numerical Methods for Nonlinear Optimization, pp. 39–43 (1972)

  30. Zhang, L., Zhou, W.J., Li, D.H.: A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 26(4), 629–640 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  31. Kou, C.X., Dai, Y.H.: A modified self-scaling memoryless Broyden-Fletcher-Goldfarb-Shanno method for unconstrained optimization. J. Optim. Theory Appl. 165 (1), 209–224 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  32. Narushima, Y., Yabe, H., Ford, J.A.: A three-term conjugate gradient method with sufficient descent property for unconstrained optimization. SIAM J. Optim. 21(1), 212–230 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  33. Liu, J.K., Xu, J.Ł., Zhang, L.Q.: Partially symmetrical derivative-free Liu-Storey projection method for convex constrained equations. Int. J. Comput. Math. 96(9), 1787–1798 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  34. Liu, J.K., Zhao, Y.X., Wu, X.L.: Some three-term conjugate gradient methods with the new direction structure. Appl. Numer. Math. 150, 433–443 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  35. Dai, Y.H., Yuan, Y.X.: An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 33–47 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  36. Zoutendijk, G.: Nonlinear programming, computational methods. Integer and Nonlinear Programming, pp. 37–86 (1970)

  37. Gould, N.I.M., Orban, D., Toint P L.: CUTEr and SifDec: A constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. (TOMS) 29(4), 373–394 (2003)

    Article  MATH  Google Scholar 

  38. Moré, J J, Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. (TOMS) 7(1), 17–41 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  39. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)

    MathSciNet  MATH  Google Scholar 

  40. Sellami, B., Laskri, Y., Benzine, R.: A new two-parameter family of nonlinear conjugate gradient methods. Optimization 64(4), 993–1009 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  41. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  42. Chan, R.H., Ho, C.W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005)

    Article  Google Scholar 

  43. Cai, J.F., Chan, R., Morini, B.: Minimization of an Edge-Preserving Regularization Functional by Conjugate Gradient type Methods. Image Processing Based on Partial Differential Equations, pp 109–122. Springer, Berlin, Heidelberg (2007)

    Google Scholar 

  44. Yu, G.H., Huang, J.H., Zhou, Y.: A descent spectral conjugate gradient method for impulse noise removal. Appl. Math. Lett. 23(5), 555–560 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  45. Cao, J.Y., Wu, J.Z.: A conjugate gradient algorithm and its applications in image restoration. Appl. Numer. Math. 152, 243–252 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  46. Aminifard, Z., Babaie-Kafaki, S.: Dai-Liao extensions of a descent hybrid nonlinear conjugate gradient method with application in signal processing. Numer. Algoritm. https://doi.org/10.1007/s11075-021-01157-y (2021)

  47. Hwang, H., Haddad, R.A.: Adaptive median filters: New algorithms and results. IEEE Trans. Image Process. 4(4), 499–502 (1995)

    Article  Google Scholar 

  48. Bovik, A.: Handbook of Image and Video Processing. Academic Press, San Diego (2000)

    MATH  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (Grant No. 11771383), the Natural Science Foundation of Guangxi Province (No. 2020GXNSFDA238017), Research Project of Guangxi University for Nationalities (Grant No. 2018KJQD02) and Innovation Project of Guangxi Graduate Education (gxun-chxp201909).

Author information

Authors and Affiliations

  1. College of Mathematics and Physics, Guangxi University for Nationalities, Nanning, 530006, China

    Xianzhen Jiang, Wei Liao, Jianghua Yin & Jinbao Jian

Authors
  1. Xianzhen Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianghua Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinbao Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinbao Jian.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, X., Liao, W., Yin, J. et al. A new family of hybrid three-term conjugate gradient methods with applications in image restoration. Numer Algor 91, 161–191 (2022). https://doi.org/10.1007/s11075-022-01258-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-022-01258-2

Keywords

Mathematics Subject Classification (2010)

Search

Navigation