Skip to main content
SpringerLink
Log in

A hybrid sampling and gradient attention network for compressed image sensing

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Block compressed sensing (BCS), which is widely used in compressed image sensing (CIS), brings the advantages of lower complexity for sampling and reconstruction. But it will result in redundant information sampling in the smooth block and insufficient information sampling in the texture block, which makes the reconstruction quality of image details poor. To address this problem, we propose a novel Hybrid Sampling and Gradient Attention Network for CIS, dubbed HSGANet. In HSGANet, a new linear sampling strategy, called hybrid BCS (HBCS) sampling is proposed to realize BCS sampling with balanced information entropy for sampling blocks. Specifically, HBCS is composed of the proposed block permutation sampling (BPS) and BCS sampling, where the BPS is used to increase the proportion of texture block information in the image measurement, which is realized by BCS sampling following image block permutation. Furthermore, a selection algorithm is developed to achieve optimal image information balanced permutation. To match HBCS, we design the initial reconstruction fusion sub-network and the deep reconstruction sub-network which is constructed by cascading GA-Blocks with gradient attention sub-network. In each phase, the gradient attention sub-network can achieve pixel-level adaptive fusion of the gradient map obtained by minimizing measurement errors of BPS and BCS. Extensive experimental results show that our HSGANet has a great improvement in reconstruction accuracy than the state-of-the-art methods with a comparable running speed and model complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Candes, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005). https://doi.org/10.1109/TIT.2005.858979

    Article  MathSciNet  MATH  Google Scholar 

  2. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). https://doi.org/10.1109/TIT.2006.871582

    Article  MathSciNet  MATH  Google Scholar 

  3. Duarte, M.F., Davenport, M.A., Takhar, D., Laska, J.N., Sun, T., Kelly, K.F., Baraniuk, R.G.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25(2), 83–91 (2008). https://doi.org/10.1109/MSP.2007.914730

    Article  Google Scholar 

  4. Liu, Y., Yuan, X., Suo, J., Brady, D.J., Dai, Q.: Rank minimization for snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 2990–3006 (2019). https://doi.org/10.1109/TPAMI.2018.2873587

    Article  Google Scholar 

  5. Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn. Reson. Med. Off. J. Int. Soc. Magn. Reson. Med. 58(6), 1182–1195 (2007)

    Article  Google Scholar 

  6. Gan, L.: Block compressed sensing of natural images. In: 2007 15th International Conference on Digital Signal Processing, pp. 403–406 (2007). https://doi.org/10.1109/ICDSP.2007.4288604

  7. Chen, C., Tramel, E.W., Fowler, J.E.: Compressed-sensing recovery of images and video using multihypothesis predictions. In: 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR), pp. 1193–1198 (2011). https://doi.org/10.1109/ACSSC.2011.6190204

  8. Zhang, J., Zhao, D., Gao, W.: Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23(8), 3336–3351 (2014). https://doi.org/10.1109/TIP.2014.2323127

    Article  MathSciNet  MATH  Google Scholar 

  9. Zha, Z., Liu, X., Zhang, X., Chen, Y., Tang, L., Bai, Y., Wang, Q., Shang, Z.: Compressed sensing image reconstruction via adaptive sparse nonlocal regularization. Vis. Comput. 34(1), 117–137 (2018)

    Article  Google Scholar 

  10. Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A.: Reconnet: Non-iterative reconstruction of images from compressively sensed measurements. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 449–458 (2016). https://doi.org/10.1109/CVPR.2016.55

  11. Yao, H., Dai, F., Zhang, S., Zhang, Y., Tian, Q., Xu, C.: Dr2-net: Deep residual reconstruction network for image compressive sensing. Neurocomputing 359, 483–493 (2019)

    Article  Google Scholar 

  12. Mousavi, A., Baraniuk, R.G.: Learning to invert: signal recovery via deep convolutional networks. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2272–2276 (2017). https://doi.org/10.1109/ICASSP.2017.7952561

  13. Lohit, S., Kulkarni, K., Kerviche, R., Turaga, P., Ashok, A.: Convolutional neural networks for noniterative reconstruction of compressively sensed images. IEEE Trans. Comput. Imaging 4(3), 326–340 (2018). https://doi.org/10.1109/TCI.2018.2846413

    Article  Google Scholar 

  14. Shi, W., Jiang, F., Zhang, S., Zhao, D.: Deep networks for compressed image sensing. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 877–882 (2017). https://doi.org/10.1109/ICME.2017.8019428

  15. Shi, W., Jiang, F., Liu, S., Zhao, D.: Scalable convolutional neural network for image compressed sensing. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12282–12291 (2019). https://doi.org/10.1109/CVPR.2019.01257

  16. Canh, T.N., Jeon, B.: Difference of convolution for deep compressive sensing. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 2105–2109 (2019). https://doi.org/10.1109/ICIP.2019.8803165

  17. Shi, W., Jiang, F., Liu, S., Zhao, D.: Image compressed sensing using convolutional neural network. IEEE Trans. Image Process. 29, 375–388 (2020). https://doi.org/10.1109/TIP.2019.2928136

    Article  MathSciNet  MATH  Google Scholar 

  18. Canh, T.N., Jeon, B.: Multi-scale deep compressive imaging. IEEE Trans. Comput. Imaging 7, 86–97 (2021). https://doi.org/10.1109/TCI.2020.3034433

    Article  Google Scholar 

  19. Tian, J., Yuan, W., Tu, Y.: Image compressed sensing using multi-scale residual generative adversarial network. Vis. Comput. 1–10 (2021)

  20. Zhang, J., Ghanem, B.: Ista-net: Interpretable optimization-inspired deep network for image compressive sensing. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1828–1837 (2018). https://doi.org/10.1109/CVPR.2018.00196

  21. Zhang, J., Zhao, C., Gao, W.: Optimization-inspired compact deep compressive sensing. IEEE J. Sel. Top. Signal Process. 14(4), 765–774 (2020). https://doi.org/10.1109/JSTSP.2020.2977507

    Article  Google Scholar 

  22. Pei, H., Yang, C., Cao, Y.: Deep smoothed projected landweber network for block-based image compressive sensing. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 2870–2874 (2020). https://doi.org/10.1109/ICIP40778.2020.9191043

  23. Zhang, Z., Liu, Y., Liu, J., Wen, F., Zhu, C.: Amp-net: Denoising-based deep unfolding for compressive image sensing. IEEE Trans. Image Process. 30, 1487–1500 (2021). https://doi.org/10.1109/TIP.2020.3044472

    Article  MathSciNet  Google Scholar 

  24. Su, Y., Lian, Q.: ipiano-net: Nonconvex optimization inspired multi-scale reconstruction network for compressed sensing. Signal Processing: Image Communication 89, 115989 (2020)

    Google Scholar 

  25. You, D., Xie, J., Zhang, J.: Ista-net++: Flexible deep unfolding network for compressive sensing. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2021). https://doi.org/10.1109/ICME51207.2021.9428249

  26. You, D., Zhang, J., Xie, J., Chen, B., Ma, S.: Coast: Controllable arbitrary-sampling network for compressive sensing. IEEE Trans. Image Process. 30, 6066–6080 (2021). https://doi.org/10.1109/TIP.2021.3091834

    Article  MathSciNet  Google Scholar 

  27. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015). https://doi.org/10.1109/CVPR.2015.7298594

  28. Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  29. Metzler, C.A., Maleki, A., Baraniuk, R.G.: From denoising to compressed sensing. IEEE Trans. Inf. Theory 62(9), 5117–5144 (2016). https://doi.org/10.1109/TIT.2016.2556683

    Article  MathSciNet  MATH  Google Scholar 

  30. Li, C., Yin, W., Jiang, H., Zhang, Y.: An efficient augmented Lagrangian method with applications to total variation minimization. Comput. Optim. Appl. 56(3), 507–530 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. Du, J., Xie, X., Wang, C., Shi, G., Xu, X., Wang, Y.: Fully convolutional measurement network for compressive sensing image reconstruction. Neurocomputing 328, 105–112 (2019)

    Article  Google Scholar 

  32. Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

  33. Mardani, M., Sun, Q., Vasawanala, S., Papyan, V., Monajemi, H., Pauly, J., Donoho, D.: Neural proximal gradient descent for compressive imaging. arXiv preprint arXiv:1806.03963 (2018)

  34. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

  35. Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011). https://doi.org/10.1109/TPAMI.2010.161

    Article  Google Scholar 

  36. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  37. Huang, J.-B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5197–5206 (2015). https://doi.org/10.1109/CVPR.2015.7299156

  38. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–4232 (2001). https://doi.org/10.1109/ICCV.2001.937655

  39. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730 (2010). Springer

  40. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the Key Program of Natural Science Foundation of Guangdong Province (2017A030311028) and the Natural Science Foundation of Guangdong Province (2019A1515011949)

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, South China University of Technology, Guangzhou, 510641, China

    Xin Yang, Chunling Yang & Wenjun Chen

Authors
  1. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunling Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenjun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunling Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Yang, C. & Chen, W. A hybrid sampling and gradient attention network for compressed image sensing. Vis Comput 39, 4213–4226 (2023). https://doi.org/10.1007/s00371-022-02585-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02585-0

Keywords

Search

Navigation