Skip to main content
SpringerLink
Log in

A convergence analysis for iterative sparsification projection with soft-thresholding

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The recently proposed iterative sparsification projection (ISP), a fast and robust sparse signal recovery algorithm framework, can be classified as smooth-ISP and nonsmooth-ISP. However, no convergence analysis has been established for the nonsmooth-ISP in the previous works. Motivated by this absence, the present paper provides a convergence analysis for ISP with soft-thresholding (ISP-soft) which is an instance of the nonsmooth-ISP. In our analysis, the composite operator of soft-thresholding and proximal projection is viewed as a fixed point mapping, whose nonexpansiveness plays a key role. Specifically, our convergence analysis for the sequence generated by ISP-soft can be summarized as follows: 1) For each inner loop, we prove that the sequence has a unique accumulation point which is a fixed point, and show that it is a Cauchy sequence; 2) for the last inner loop, we prove that the accumulation point of the sequence is a critical point of the objective function if the final value of the threshold satisfies a condition, and show that the corresponding objective values are monotonically nonincreasing. A numerical experiment is given to validate some of our results and intuitively present the convergence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Generally, under the case where \(J({\mathbf {x}})=\Vert {\mathbf {x}}\Vert _{1}\), problem (2) is equivalently reformulated as an unconstrained regularized optimization, termed basis pursuit denoising (BPDN).

  2. The smooth version of (2) means that \(J(\cdot )\) is replaced by its smooth approximation.

  3. The ISP-\(\ell _{0}\), ISP-\(\ell _{1}\), ISP-hard and ISP-soft are named by [16]. The smooth-ISP and nonsmooth-ISP are named by [17].

  4. In Algorithm 1, \({\mathbf {A}}^{\dagger }\) denotes the Moore–Penrose pseudo-inverse of \({\mathbf {A}}\), and other initial values for \({\mathbf {x}}\) can also be used.

  5. We did multiple trials in our experiment, but only present the results of one trial, since in each trial we observed similar results.

References

  1. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  2. Ranjan, S., Vidyasagar, M.: Tight performance bounds for compressed sensing with conventional and group sparsity. IEEE Tran. Signal Process. 67(11), 2854–2867 (2019)

    Article  MathSciNet  Google Scholar 

  3. Elad, M.: Sparse and Redundant Representations-From Theory to Applications in Signal and Image Processing. Springer, New York (2010)

    Book  Google Scholar 

  4. Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)

    Article  MathSciNet  Google Scholar 

  5. Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers, pp 40–44 (1993)

  6. Wang, J.: Support recovery with orthogonal matching pursuit in the presence of noise. IEEE Trans. Signal Process. 63(21), 5868–5877 (2015)

    Article  MathSciNet  Google Scholar 

  7. Needell, D., Tropp, J.A.: CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–321 (2009)

    Article  MathSciNet  Google Scholar 

  8. Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)

    Article  MathSciNet  Google Scholar 

  9. Cartis, C., Thompson, A.: A new and improved quantitative recovery analysis for iterative hard thresholding algorithms in compressed sensing. IEEE Trans. Inf. Theory 61(4), 2019–2042 (2015)

    Article  MathSciNet  Google Scholar 

  10. Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. J. Issued Courant Inst. Math. Sci. 57(11), 1413–1457 (2004)

    Article  MathSciNet  Google Scholar 

  11. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  12. Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18(11), 2419–2434 (2009)

    Article  MathSciNet  Google Scholar 

  13. Kim, D., Fessler, J.A.: Another look at the fast iterative shrinkage/thresholding algorithm (FISTA). SIAM J. Optim. 28(1), 223–250 (2018)

    Article  MathSciNet  Google Scholar 

  14. Zibetti, M.V.W., Helou, E.S., Regatte, R.R., Herman, G.T.: Monotone FISTA with variable acceleration for compressed sensing magnetic resonance imaging. IEEE Trans. Comput. Imag. 5(1), 109–119 (2019)

    Article  Google Scholar 

  15. Florea, M.I., Vorobyov, S.A.: A generalized accelerated composite gradient method: Uniting Nesterov’s fast gradient method and FISTA. IEEE Trans. Signal Process. 68, 3033–3048 (2020)

  16. Sadeghi, M., Babaie-Zadeh, M.: Iterative sparsification-projection: Fast and robust sparse signal approximation. IEEE Trans. Signal Process. 64(21), 5536–5548 (2016)

  17. Ghayem, F., Sadeghi, M., Babaie-Zadeh, M., Chatterjee, S., Skoglund, M., Jutten, C.: Sparse signal recovery using iterative proximal projection. IEEE Trans. Signal Process. 66(4), 879–894 (2018)

    Article  MathSciNet  Google Scholar 

  18. Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer (2011)

  19. Parikh, N., Boyd, S.: Proximal algorithms. In: Foundations and Trends® in Optimization 1(3), 123–231 (2014)

  20. Mordukhovich, B.S.: Variational analysis and generalized differentiation I: Basic theory. Vol. 330 of Grundlehren der mathematischen Wissenschaften. Berlin, Germany: Springer (2006)

  21. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1972)

    Google Scholar 

  22. Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)

    Article  MathSciNet  Google Scholar 

  23. Hale, E.T., Yin, W., Zhang, Y.: Fixed-point continuation for \(\ell _{1}\)-minimization: methodology and convergence. SIAM J. Optim. 19(3), 1107–1130 (2008)

    Article  MathSciNet  Google Scholar 

  24. Calamai, P.H., Moré, J.J.: Projected gradient methods for linearly constrained problems. Math. Prog. 39(1), 93–116 (1987)

    Article  MathSciNet  Google Scholar 

  25. Granas, A., Dugundji, J.: Fixed Point Theory. Springer, New York (2013)

    MATH  Google Scholar 

  26. Sohrab, H.H.: Basic Real Analysis. Springer, Boston (2014)

    MATH  Google Scholar 

  27. Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imag. Sci. 7(2), 1388–1419 (2014)

    Article  MathSciNet  Google Scholar 

  28. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. In: Foundations and Trends® in Machine learning 3(1), 1–122 (2011)

  29. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Cham (2017)

    Book  Google Scholar 

Download references

Funding

This work was supported in part by the Natural Science Foundation of Guangdong Province under Grants 2019A1515010861, in part by Guangzhou Technical Project under Grant 201902020008, and in part by NSFC under Grant 61471174.

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, South China University of Technology, Guangzhou, 510640, China

    Tao Zhu

Authors
  1. Tao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, T. A convergence analysis for iterative sparsification projection with soft-thresholding. SIViP 15, 1705–1712 (2021). https://doi.org/10.1007/s11760-021-01910-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01910-9

Keywords

Search

Navigation