Skip to main content
Springer Nature Link
Log in

Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing

  • Original Research
  • Published:
Journal of Applied Mathematics and Computing Aims and scope Submit manuscript

Abstract

In this paper, we present two new three-term conjugate gradient methods which can generate sufficient descent directions for the large-scale optimization problems. Note that this property is independent of the line search used. We prove that these three-term conjugate gradient methods are global convergence under the Wolfe line search. Numerical experiments and comparisons demonstrate that the proposed algorithms are efficient approaches for test functions. Moreover, we use the proposed methods to solve the \(\ell _1-\alpha \ell _2\) regularization problem of sparse signal decoding in compressed sensing, and the results show that our methods have certain advantages over the existing solvers on such problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  2. Polak, E., Ribière, G.: Note sur la convergence de méthodes de directions conjugées. Rev. Française Inf. Rech. Oper. 3, 35–43 (1969)

  3. Polyak, B.T.: The conjugate gradient method in extreme problems. U.S.S.R. Comput. Math. and Math. Phys. 9, 94–112 (1969)

  4. Hestenes, M.R., Stiefel, E.: Method of conjugate gradient for solving linear equations. J. Res. Natl. Bureau Stand. 49, 409–436 (1952)

    Article  MATH  Google Scholar 

  5. Fletcher, R.: Practical Method of Optimization, Vol I: Unconstrained Optimization. Wiley, NewYork 1987

  6. Dai, Y.H., Yuan, Y.: A three-parameter family of nonlinear conjugate gradient methods. Math. Comput. 70, 1155–1167 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  7. Liu, Y., Storey, C.: Efficient generalized conjugate gradient algorithms, part I: theory. J. Optim. Theory Appl. 69, 129–137 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  8. Sun, W.M., Zhu, Z.B., Zhao, R.W.: A modified LS conjugate gradient algorithm for solving Symm integral equation. J. Guilin Univ. Electron. Technol. 38, 158–161 (2018)

    Google Scholar 

  9. Su, J.P., Zhu, Z.B.: An improved CD conjugate gradient method for application to black body radiation inversion problem. J. Guilin Univ. Electron. Technol. 38, 496–499 (2018)

    Google Scholar 

  10. Lu, J., Yuan, G., Wang, Z.: A modified Dai-Liao conjugate gradient method for solving unconstrained optimization and image restoration problems. J. Appl. Math. Comput. (2021). https://doi.org/10.1007/s12190-021-01548-3

  11. Yuan, G., Li, T., Hu, W.: A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Appl. Numer. Math. 147, 129–141 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  12. Yuan, G., Lu, J., Wang, Z.: The PRP conjugate gradient algorithm with a modified WWP line search and its application in the image restoration problems. Appl. Numer. Math. 152, 1–11 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  13. Yuan, G., Lu, J., Wang, Z.: The modified PRP conjugate gradient algorithm under a non-descent line search and its application in the Muskingum model and image restoration problems. Soft Comput. 25, 5867–5879 (2021)

    Article  Google Scholar 

  14. Pan, L., Chen, X., Yeo, S.P.: A compressive-sensing-based phaseless imaging method for point-like dielectric objects. IEEE Trans. Antennas Propag. 60, 5472–5475 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  15. Zhang, L., Zhou, W., Li, D.: A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 26, 629–640 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  16. Zhang, L., Zhou, W., Li, D.: Global convergence of a modified Fletcher-Reeves conjugate gradient method with Armijo-type line search. Numer. Math. 104, 561–572 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhang, L., Zhou, W., Li, D.: Some descent three-term conjugate gradient methods and their global convergence. Optim. Methods Softw. 22, 697–711 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  18. Yin, L., Chen, X.: Global convergence of two kinds of three-term conjugate gradient methods without line search. Asia-Pac. J. Oper. Res. 30, 1–10 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43, 87–101 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  20. Andrei, N.: Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization. Bull. Malays. Math. Sci. Soc. 34, 319–330 (2011)

    MathSciNet  MATH  Google Scholar 

  21. Zhou, W., Zhang, L.: A nonlinear conjugate gradient method based on the MBFGS secant condition. Optim. Methods Softw. 21, 707–714 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  22. Li, D.H., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. 129, 15–35 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  23. Babaie-Kafaki, S., Ghanbari, R.: Two modified three-term conjugate gradient methods with sufficient descent property. Optim Lett. 8, 2285–2297 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  24. Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1, 586–597 (2007)

    Article  Google Scholar 

  25. Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hager, W.W., Zhang, H.: Algorithm 851: CG\_DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32, 113–137 (2006)

    Article  MATH  Google Scholar 

  27. Nguyen, C.T., Saheya, B., Chang, Y.L., Chen, J.S.: Unified smoothing functions for absolute value equation associated with second-order cone. Appl. Numer. Math. 135, 206–227 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  28. Sun, W., Yuan, Y.X.: Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006)

    MATH  Google Scholar 

  29. Du, D.Z., Pardalos, P.M., Wu, W.: Mathematical Theory of Optimization. Kluwer, Dordrecht (2001)

    Book  MATH  Google Scholar 

  30. Andrei, N.: Another hybrid conjugate gradient algorithm for unconstrained optimization. Numer. Algorithms 47, 143–156 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  31. Sugiki, K., Narushima, Y., Yabe, H.: Globally convergent three-term conjugate gradient methods that use secant conditions and generate descent search directions for unconstrained optimization. J. Optim. Theory Appl. 153, 733–757 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  32. Zhu, Z., Ma, J., Zhang, B.: A new conjugate gradient hard thresholding pursuit algorithm for sparse signal recovery. Comput. Appl. Math. 39, 1–20 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  33. Gould, N.I., Orban, D., Toint, P.L.: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)

    Article  MATH  Google Scholar 

  34. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wu, C., Wang, J., Alcantara, J.H., Chen, J.S.: Smoothing strategy along with conjugate gradient algorithm for signal reconstruction. J. Sci. Comput. 87, 1–18 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  36. Yin, P., Lou, Y., He, Q., Xin, J.: Minimization of \(\ell _{1-2}\) for compressed sensing. SIAM J. Sci. Comput. 37, A536–A563 (2015)

    Article  MathSciNet  Google Scholar 

  37. Lu, W., Li, L., Cai, A., Zhang, H., Wang, L., Yan, B.: A weighted difference of L1 and L2 on the gradient minimization based on alternating direction method for circular computed tomography. J. X-Ray Sci. Technol. 25, 813–829 (2017)

    Article  Google Scholar 

  38. Wu, C., Zhan, J., Lu, Y., Chen, J.S.: Signal reconstruction by conjugate gradient algorithm based on smoothing \(\ell _1\)-norm. Calcolo 56, 1–26 (2019)

    Article  Google Scholar 

  39. Zhu, H., Xiao, Y., Wu, S.Y.: Large sparse signal recovery by conjugate gradient algorithm based on smoothing technique. Comput. Math. Appl. 66, 24–32 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  40. Yang, X., Luo, Z., Dai, X.: A global convergence of LS-CD hybrid conjugate gradient method. Adv. Numer. Anal. 2013, 1–5 (2013)

    MathSciNet  MATH  Google Scholar 

  41. Koh, K., Kim, S.J., Boyd, S.: l1\(\_\)ls: A Matlab Solver for Large-Scale \(\ell _1\)-Regularized Least Squares Problems. (2007). http://www.stanford.edu/~boyd/l1_ls/

  42. Figueiredo, M.A.T., Nowak, R.D.: A bound optimization approach to wavelet-based image deconvolution. In: IEEE International Conference on Image Processing (ICIP), Genoa Italy (2005)

  43. Becker, S., Bobin, J., Candès, E.J.: NESTA: a fast and accurate first-order method for sparse recovery. SIAM J. Imaging Sci. 4, 1–39 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Engineering and Automation, Guangxi Key Laboratory of Automatic Detecting Technology and Instruments, Guilin University of Electronic Technology, Guilin, 541004, China

    Yufeng Liu & Benxin Zhang

  2. School of Mathematics and Computing Science, Guangxi Colleges and Universities Key Laboratory of Data Analysis and Computation, Guilin University of Electronic Technology, Guilin, 541004, China

    Zhibin Zhu

Authors
  1. Yufeng Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhibin Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Benxin Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Zhibin Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the National Natural Science Foundation of China (61967004, 11901137, 11961011), Guangxi Natural Science Foundation (2018GXNSFBA281023), China Postdoctoral Science Foundation (2020M682959), Guangxi Key Laboratory of Cryptography and Information Security (GCIS201927), Guangxi Key Laboratory of Automatic Detecting Technology and Instruments (YQ20113) , and Innovation Project of Guangxi Graduate Education (2021YCXS118)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Zhu, Z. & Zhang, B. Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing. J. Appl. Math. Comput. 68, 1787–1816 (2022). https://doi.org/10.1007/s12190-021-01589-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12190-021-01589-8

Keywords