Skip to main content
Log in

Convergence of iterative hard-thresholding algorithm with continuation

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

In this paper, we study the convergence of iterative hard-thresholding algorithm with continuation for solving the \(\ell ^0\)-regularized minimization. A decreasing continuation strategy is used for the regularization parameter. By using the Kurdyka–Łojasiewicz property, we prove that the algorithm globally converges to a critical point of a known objective function. Numerical results are also presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  2. Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 5(12), 4203–4215 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  3. Blumensath, T., Davies, M.E.: Iterative thresholding for sparse approximations. J. Fourier Anal. Appl. 14(5–6), 629–654 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  4. Blumensath, T., Yaghoobi, M., Davies, M.E.: Iterative hard thresholding and \(\ell _0\) regularisation. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2007. ICASSP 2007, vol. 3, pp. III-877–III-880. IEEE (2007)

  5. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137(1–2), 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  6. Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 123–231 (2013)

    Google Scholar 

  7. Frankel, P., Garrigos, G., Peypouquet, J.: Splitting methods with variable metric for KL functions and general convergence rates. J. Optim. Theory Appl. 165(3), 874–900 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  8. Hale, E.T., Yin, W., Zhang, Y.: Fixed-point continuation for \(\ell _1\)-minimization: methodology and convergence. SIAM J. Optim. 19(3), 1107–1130 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  9. Hale, E.T., Yin, W., Zhang, Y.: A fixed-point continuation method for \(\ell _1\)-regularized minimization with applications to compressed sensing. CAAM TR07-07, vol. 43, p. 44, Rice University (2007)

  10. Goldfarb, D., Ma, S.: Convergence of fixed-point continuation algorithms for matrix rank minimization. Found. Comput. Math. 11(2), 183–210 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Yin, W., Osher, S., Goldfarb, D., et al.: Bregman iterative algorithms for \(\ell _1\)-minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1(1), 143–168 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  12. Toh, K.C., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac. J. Optim. 6(615–640), 15 (2010)

    MathSciNet  MATH  Google Scholar 

  13. Jiao, Y., Jin, B., Lu, X.: A primal dual active set with continuation algorithm for the L0-regularized optimization problem. Appl. Comput. Harmon. Anal. 39(3), 400–426 (2015)

  14. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I: Basic Theory. Springer Science & Business Media, Berlin (2006)

    Google Scholar 

  15. Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer Science & Business Media, Berlin (2009)

    MATH  Google Scholar 

  16. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (2015)

    MATH  Google Scholar 

  17. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  18. Lewis, A.S., Sendov, H.S.: Nonsmooth analysis of singular values, part I: theory. Set Valued Anal. 13, 213–241 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  19. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116(1–2), 5–16 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors are indebted to the editors and anonymous referees for their useful suggestions. We are grateful for the support from the National Natural Science Foundation of Hunan Province, China (13JJ2001), and the Science Project of National University of Defense Technology (JC120201), and National Science Foundation of China (No. 61402495).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tao Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, T., Cheng, L. Convergence of iterative hard-thresholding algorithm with continuation. Optim Lett 11, 801–815 (2017). https://doi.org/10.1007/s11590-016-1062-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-016-1062-0

Keywords

Navigation