Skip to main content
SpringerLink
Log in

Recovery of image and video based on compressive sensing via tensor approximation and Spatio-temporal correlation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, block-based compressive sensing (BCS) has been extensively studied because it can reduce computational complexity and data storage by dividing the image into smaller patches, but the performance of the reconstruction algorithm is not satisfactory. In this paper, a new reconstruction model for image and video is proposed. The model makes full use of spatio-temporal correlation and utilizes low-rank tensor approximation to improve the quality of the reconstructed image and video. For image recovery, the proposed model obtains a low-rank approximation of a tensor formed by non-local similar patches, and improves the reconstruction quality from a spatial perspective by combining non-local similarity and low-rank property. For video recovery, the reconstruction process is divided into two phases. In the first phase, each frame of the video sequence is regarded as an independent image to be reconstructed by taking advantage of spatial property. The second phase performs tensor approximation through searching similar patches within frames near the target frame, to achieve reconstruction by putting the spatio-temporal correlation into full play. The resulting model is solved by an efficient Alternating Direction Method of Multipliers (ADMM) algorithm. A series of experiments show that the quality of the proposed model is comparable to the current state-of-the-art recovery methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Asif MS, Romberg J (2013) Fast and accurate algorithms for re-weighted ℓ1-norm minimization. IEEE Trans Signal Process 61(23):5905–5916

    Article  MathSciNet  Google Scholar 

  2. Baraniuk RG, Cevher V, Duarte MF, et al (2010) Model-based compressive sensing. IEEE Trans Inf Theory 56(4)1982–2001. https://doi.org/10.1109/TIT.2010.2040894

  3. Bruckstein AM, Donoho DL, Elad M (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51(1):34–81

    Article  MathSciNet  Google Scholar 

  4. Candès EJ, Wakin MB, Boyd SP (2007) Enhancing sparsity by reweighted L1 minimization. J Fourier Anal Appl 14(5):877–905. https://doi.org/10.1007/s00041-008-9045-x

  5. Candès EJ, Wakin MB, Boyd SP (2007) Enhancing sparsity by reweighted L1 minimization. J Fourier Anal Appl 14(5):877–905

    MATH  Google Scholar 

  6. Chen C, Tramel EW, Fowler JE (2012) Compressed-sensing recovery of images and video using multihypothesis predictions, in: IEEE 46th Signals, systems and computers, pp. 1193–1198.

  7. Dong W, Shi G, Li X (2014) Compressive sensing via nonlocal low-rank regularization. IEEE Trans Image Process 23(8):3618–3632

    Article  MathSciNet  Google Scholar 

  8. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Article  MathSciNet  Google Scholar 

  9. Eftekhari A, Wakin MB (2015) New analysis of manifold Embeddings and signal recovery from compressive measurements. Appl Comput Harmon Anal 39(1):67–109. https://doi.org/10.1016/j.acha.2014.08.005

  10. Emmanuel J. Candès, Wakin M B , Boyd S P, "Enhancing Sparsity by Reweighted L1 Minimization," Journal of Fourier Analysis &Applications, vol. 14, no. 5, pp. 877-905, 2007.

  11. Fowler JE, Mun S, Tramel EW (2012) Block-based compressed sensing of images and video. Found Trends Signal Process 4(4):297–416

    Article  Google Scholar 

  12. Gan L (2007) Block compressed sensing of natural images, in: IEEE Proceedings of the 15th International Conference on Digital Signal Processing, pp. 403–406.

  13. Hegde C, Baraniuk RG (2012) Signal recovery on incoherent manifolds. IEEE Trans. Inf. Theory 58(12):7204–7214

    Article  MathSciNet  Google Scholar 

  14. Hegde C, Wakin M, Baraniuk R (2008) Random projections for manifold learning, in: Advances in neural information processing systems, pp. 641–648.

  15. Illiasdis M, Spinoulas L, Katsaggelos AK (2018) Deep fully-connected networks for video compressive sensing, Elsevier Digital Signal Processing, 72

  16. Kindermann S, Osher S, Jones PW (2005) Deblurring and denoising of images by nonlocal functionals. Siam Journal on Multiscale Modeling & Simulation 4(4):1091–1115. https://doi.org/10.1137/050622249

  17. Li C, Yin W, Jiang H, Zhang Y (2013) An efficient augmented lagrangian method with applications to total variation minimization. Comput Optim Appl 56(3):507–530

    Article  MathSciNet  Google Scholar 

  18. Lingala SG, Hu Y, DiBella E, Jacob M (2011) Accelerated dynamic MRI exploiting sparsity and low-rank structure: k-t SLR. IEEE TransMed Imag 30(5):1042–1054

    Article  Google Scholar 

  19. Lu H, Li S, Liu Q et al. (2018) MF-LRTC: Multi-filters guided low-rank tensor coding for image restoration, NEUROCOMPUTING.

  20. Mairal J, Elad M, Sapiro G (2007) Sparse representation for color image restoration. IEEE Trans Image Process 17(1):53–69

    Article  MathSciNet  Google Scholar 

  21. Mun S, Fowler JE (2010) Block Compressed Sensing of Images Using Directional Transforms, in: IEEE 17th International Conference on Image Processing, p. 547.

  22. Mun S, Fowler JE (2011) Residual reconstruction for block-based compressed sensing of video, in: Proc. IEEE Data Compress. Conf. (DCC), pp. 183–192.

  23. Palangi H, Ward R, Deng L (2016) Distributed compressive sensing: a deep learning approach. IEEE Trans Signal Process 64(17):4504–4518. https://doi.org/10.1109/TSP.2016.2557301

  24. Shi W, Jiang F, Liu S et al. (2019) Scalable convolutional neural network for image compressed sensing. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  25. Siddamal KV, Bhat SP, Saroja VS (2015) A survey on compressive sensing, in: IEEE 2nd International Conference on Electronics and Communication Systems, pp. 639–643.

  26. Tramel EW, Fowler JE (2011) Video compressed sensing with multihypothesis, in: Proc. IEEE Data Compress. Conf. (DCC), pp. 193–202.

  27. Ulyanov D, Vedaldi A, Lempitsky V (2018) Deep image prior, IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  28. Van Veen D, Jalal A, Soltanolkotabi M, et al. (2018) Compressed sensing with deep image prior and learned regularization

  29. Xu J, Ma J, Zhang D, Zhang Y, Lin S (2012) Improved total variation minimization method for compressive sensing by intra-prediction. Signal Process 92(11):2614–2623

    Article  Google Scholar 

  30. Zhang J, Ghanem B (2018) ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing. IEEE Conference on Computer Vision and Pattern Recognition:1828–1837. https://doi.org/10.1109/CVPR.2018.00196

  31. Zhang X, Burger M, Bresson X (2010) Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J Imag Sci 3(3):253–276

    Article  MathSciNet  Google Scholar 

  32. Zhang J, Zhao D, Zhao C (2012) Image compressive sensing recovery via collaborative sparsity. IEEE J Emerging Sel Top Circuits Syst 2(3):380–391

    Article  MathSciNet  Google Scholar 

  33. Zhang J, Zhao C, Zhao D et al (2014) Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization. Signal Process 103:114–126. https://doi.org/10.1016/j.sigpro.2013.09.025

  34. Zhang J, Zhao C, Gao W (2020) Optimization-inspired compact deep compressive sensing. IEEE Journal of Selected Topics in Signal Processing

  35. Zhao C, Ma S, Gao W (2014) Video compressive sensing via structured Laplacian modelling, in: Proc. IEEE Vis. Commun. Image Process. Conf. pp. 402–405.

  36. Zhao C, Ma S, Zhang J, Xiong R (2016) Video compressive sensing reconstruction via reweighted residual sparsity. IEEE Trans Circuits Syst Video Technol 27(6):1–1

    Google Scholar 

Download references

Funding

This work was supported by National Natural Science Foundation (No.61501069) and by Technology Innovation and Application Development Project of Chongqing (cstc2019jscx-msxmX0167).

Author information

Authors and Affiliations

  1. School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, China

    Yuanhong Zhong, Jing Zhang & Xinyu Cheng

  2. Chongqing Automotive Collaborative Innovation Center, Chongqing University, Chongqing, China

    Zhaokun Zhou & Guan Huang

  3. Tibet Branch of State Power Investment Corporation, Chongqing, China

    Qiang Li

Authors
  1. Yuanhong Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaokun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinyu Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanhong Zhong.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, Y., Zhang, J., Zhou, Z. et al. Recovery of image and video based on compressive sensing via tensor approximation and Spatio-temporal correlation. Multimed Tools Appl 80, 7433–7450 (2021). https://doi.org/10.1007/s11042-020-09907-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09907-1

Keywords

Search

Navigation