Skip to main content
SpringerLink
Log in

FISTA-CSNet: a deep compressed sensing network by unrolling iterative optimization algorithm

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In order to fast sample an image and accurately reconstruct the image from a small amount of sampled data, we design a novel deep network for optimization-based algorithm mapping to efficiently tackle the problem of image compressed sensing (CS). The new deep network structure, dubbed FISTA-CSNet, unrolls the fast iterative shrinkage-thresholding algorithm (FISTA) into two modules: sampling matrix module and reconstruction network module. The two modules are optimized jointly and the parameters in the matrix and network are discriminatively learned by end-to-end training. The sampling matrix is adaptively learned from the training images, which can better utilize the image texture information for CS reconstruction. The reconstruction network module is subdivided into two parts. The first part casts the optimization-based algorithm into deep network form and the second part uses a set of convolutional filters and nonlinear activation function to reduce the blocking artifacts introduced by block CS. In view of the unavailability of the reconstruction network at different sampling ratios, the ratio-adaptive sampling matrix and the reconstruction network are proposed to realize the multi-sampling ratio reuse version of FISTA-CSNet, dubbed FISTA-CSNet*, so that the system can operate on a range of sampling ratios. Extensive experiments show that the proposed FISTA-CSNets outperform previous state-of-the-art CS methods in term of PSNR, SSIM, FSIM and visual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  2. Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  3. Wright, S.J., Wright, R.D.S.J., Nowak, R.D., Figueiredo, M.A.T.: Sparse reconstruction by separable approximation. In: 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3373–3376 (2008)

  4. Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)

    Article  Google Scholar 

  5. Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53(12), 4655–4666 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  6. Needell, D., Tropp, J.A.: CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–321 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bannour Lahaw, Z., Seddik, H.: A new greedy sparse recovery algorithm for fast solving sparse representation. Vis. Comput. (2021). https://doi.org/10.1007/s00371-021-02121-6

    Article  Google Scholar 

  8. Haupt, J., Nowak, R.: Signal reconstruction from noisy random projections. IEEE Trans. Inf. Theory 52(9), 4036–4048 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A.: ReconNet: non-iterative reconstruction of images from compressively sensed measurements. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 449–458 (2016)

  10. Yao, H., Dai, F., Zhang, D., Ma, Y., Zhang, S., Zhang, Y., Tian, Q.: DR2-Net: deep residual reconstruction network for image compressive sensing. Neurocomputing 359, 483–493 (2019)

    Article  Google Scholar 

  11. Zhou, S., He, Y., Liu, Y., Li, C., Zhang, J.: Multi-channel deep networks for block-based image compressive sensing. IEEE Trans. Multimedia 23, 2627–2640 (2020)

    Article  Google Scholar 

  12. Shi, W., Jiang, F., Liu, S., Zhao, D.: Image compressed sensing using convolutional neural network. IEEE Trans. Image Process. 29, 375–388 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhang, J., Ghanem, B.: ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1828–1837 (2018)

  14. Yang, Y., Sun, J., Li, H., Xu, Z.: ADMM-CSNet: a deep learning approach for image compressive sensing. IEEE Trans. Pattern Anal. Mach. Intell. 42(3), 521–538 (2020)

    Article  Google Scholar 

  15. Daubechies, I., Defrise, M., Mol, C.D.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. 57(11), 1413–1457 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  16. Li, L., Pan, J., Lai, W.S., Gao, C., Sang, N., Yang, M.H.: Blind image deblurring via deep discriminative priors. Int. J. Comput. Vis. 127(8), 1025–1043 (2019)

    Article  Google Scholar 

  17. Dong, J., Roth, S., Schiele, B.: DWDN: deep wiener deconvolution network for non-blind image deblurring. IEEE Trans. Pattern Anal. Mach. Intell. https://doi.org/10.1109/TPAMI.2021.3138787

  18. Beck, A., Teboulle, M.: A fast iterative shrinkage thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  19. Li, C., Yin, W., Zhang, Y.: TVAL3: TV minimization by augmented Lagrangian and alternating direction algorithm 2009 (2013). https://www.caam.rice.edu/~optimization/L1/TV-AL3/

  20. Li, C., Yin, W., Jiang, H., Zhang, Y.: An efficient augmented Lagrangian method with applications to total variation minimization. Comput. Optim. Appl. 56(3), 507–530 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Mun, S., Fowler, J.E.: Block compressed sensing of images using directional transforms. In: 2010 Data Compression Conference, pp. 547–547 (2010)

  22. He, L., Carin, L.: Exploiting structure in wavelet-based Bayesian compressive sensing. IEEE Trans. Signal Process. 57(9), 3488–3497 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hernández-Bautista, I., Carrasco-Ochoa, J.A., Martínez-Trinidad, J.F.: Automatic filter coefficient calculation in lifting scheme wavelet transform for lossless image compression. Vis. Comput. 37, 957–972 (2021). https://doi.org/10.1007/s00371-020-01846-0

    Article  Google Scholar 

  24. Kim, Y., Nadar, M.S., Bilgin, A.: Compressed sensing using a Gaussian Scale Mixtures model in wavelet domain. In: 2010 IEEE International Conference on Image Processing, pp. 3365–3368 (2010)

  25. Zhao, C., Ma, S., Zhang, J., Xiong, R., Gao, W.: Video compressive sensing reconstruction via reweighted residual sparsity. IEEE Trans. Circuits Syst. Video Technol. 27(6), 1182–1195 (2017)

    Article  Google Scholar 

  26. Zhang, J., Zhao, D., Zhao, C., Xiong, R., Ma, S., Gao, W.: Image compressive sensing recovery via collaborative sparsity. IEEE J. Emerging Sel. Top. Circuits Syst. 2(3), 380–391 (2012)

  27. Zhang, J., Zhao, D., Gao, W.: Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23(8), 3336–3351 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  28. Zhao, C., Zhang, J., Ma, S., Gao, W.: Nonconvex Lp nuclear norm based ADMM framework for compressed sensing. In: 2016 Data Compression Conference (DCC), pp. 161–170 (2016)

  29. Qu, X., Hou, Y., Lam, F., Guo, D., Zhong, J., Chen, Z.: Magnetic resonance image reconstruction from under sampled measurements using a patch-based nonlocal operator. Med. Image Anal. 18(6), 843–856 (2014)

    Article  Google Scholar 

  30. Wang, S., Liu, J., Peng, X., Dong, P., Liu, Q., Liang, D.: Two-layer tight frame sparsifying model for compressed sensing magnetic resonance imaging. Biomed Res. Int. 2016, 1–7 (2016)

    Google Scholar 

  31. Pfister, L., Bresler, Y.: Learning sparsifying filter banks. Int. Soc. Opt. Photonics 9597, 959703.1-959703.10 (2015)

    Google Scholar 

  32. Zha, Z., Liu, X., Zhang, X., et al.: Compressed sensing image reconstruction via adaptive sparse nonlocal regularization. Vis. Comput. 34, 117–137 (2018)

    Article  Google Scholar 

  33. Ravishankar, S., Bresler, Y.: MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Med. Imaging 30(5), 1028–1041 (2011)

    Article  Google Scholar 

  34. Zhan, Z., Cai, J., Guo, D., Liu, Y., Chen, Z., Qu, X.: Fast multiclass dictionaries learning with geometrical directions in MRI reconstruction. IEEE Trans. Biomed. Eng. 63(9), 1850–1861 (2016)

    Article  Google Scholar 

  35. Montazeri, A., Shamsi, M., Dianat, R.: MLK-SVD, the new approach in deep dictionary learning. Vis. Comput. 37, 707–715 (2021)

    Article  Google Scholar 

  36. Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  37. Mousavi, A., Patel, A.B., Baraniuk, R.G.: A deep learning approach to structured signal recovery. In: 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1336–1343 (2015)

  38. Iliadis, M., Spinoulas, L., Katsaggelos, A.K.: Deep fully connected networks for video compressive sensing. Digital Signal Process. 72, 9–18 (2018)

    Article  Google Scholar 

  39. Mousavi, A., Baraniuk, R.G.: Learning to invert: signal recovery via deep convolutional networks. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2272–2276 (2017)

  40. Dhengre, N., Sinha, S.K.: sparse autoencoder-based accelerated reconstruction of magnetic resonance imaging. Vis. Comput. 38, 837–847 (2022). https://doi.org/10.1007/s00371-020-02054-6

    Article  Google Scholar 

  41. Gregor, K., LeCun, Y.: Learning fast approximations of sparse coding. In: International Conference on Machine Learning, pp. 399–406 (2010)

  42. Yang, Y., Sun, J., Li, H., Xu, Z.: Deep ADMM-Net for compressive sensing MRI. In: Proceedings of the30th International Conference on Neural Information Processing Systems, pp. 10–18 (2016)

  43. Sullivan, G.J., Ohm, J., Han, W., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  44. Wallace, G.K.: The JPEG still picture compression standard. IEEE Trans. Consum. Electron. 38(1), xviii–xxxiv (1992)

    Article  Google Scholar 

  45. Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  46. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  47. Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  48. Shi, W., Jiang, F., Liu, S., Zhao, D.: Scalable convolutional neural network for image compressed sensing. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), pp. 12282–12291

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 61501334.

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430072, China

    Liqi Xin & Dingwen Wang

  2. School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, 430072, China

    Wenxuan Shi

Authors
  1. Liqi Xin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dingwen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenxuan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenxuan Shi.

Ethics declarations

Conflict of interest

The authors declared that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xin, L., Wang, D. & Shi, W. FISTA-CSNet: a deep compressed sensing network by unrolling iterative optimization algorithm. Vis Comput 39, 4177–4193 (2023). https://doi.org/10.1007/s00371-022-02583-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02583-2

Keywords

Search

Navigation