Skip to main content
SpringerLink
Log in

Various Plug-and-Play Algorithms with Diverse Total Variation Methods for Video Snapshot Compressive Imaging

  • Conference paper
  • First Online:
Book cover Artificial Intelligence (CICAI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13069))

Included in the following conference series:

  • 2056 Accesses

Abstract

Sampling high-dimensional images is challenging due to limited availability of sensors; scanning is usually necessary in these cases. To mitigate this challenge, snapshot compressive imaging (SCI) was proposed to capture the high-dimensional (usually 3D) images using a 2D sensor (detector). Via novel optical design, the measurement captured by the sensor is an encoded image of multiple frames of the 3D desired signal. Following this, reconstruction algorithms are employed to retrieve the high-dimensional data. In this paper, we consider different plug-and-play (PnP) algorithms for SCI reconstruction, where various denoisers can be used into diverse solvers, such as ISTA, FISTA, TwIST, ADMM and GAP. Regarding the denoisers, though various algorithms have been proposed, the total variation (TV) based method is still the most efficient one due to a good trade-off between computational time and performance. This paper aims to answer the question of which TV penalty (anisotropic TV, isotropic TV and vectorized TV) works best for video SCI reconstruction? Various TV denoising and solvers are developed and tested for video SCI reconstruction on both simulation and real datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18(11), 2419–2434 (2009)

    Article  MathSciNet  Google Scholar 

  2. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  3. Bioucas-Dias, J., Figueiredo, M.: A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16(12), 2992–3004 (2007)

    Article  MathSciNet  Google Scholar 

  4. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  5. Bresson, X., Chan, T.F.: Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Probl. Imaging 2, 455 (2008)

    Article  MathSciNet  Google Scholar 

  6. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004)

    MathSciNet  MATH  Google Scholar 

  7. Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Rangarajan, A., Vemuri, B., Yuille, A.L. (eds.) EMMCVPR 2005. LNCS, vol. 3757, pp. 136–152. Springer, Heidelberg (2005). https://doi.org/10.1007/11585978_10

    Chapter  Google Scholar 

  8. Chan, S.H., Wang, X., Elgendy, O.A.: Plug-and-play ADMM for image restoration: fixed-point convergence and applications. IEEE Trans. Comput. Imaging 3, 84–98 (2017)

    Article  MathSciNet  Google Scholar 

  9. Cheng, Z., et al.: Memory-efficient network for large-scale video compressive sensing. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021

    Google Scholar 

  10. Cheng, Z., et al.: BIRNAT: bidirectional recurrent neural networks with adversarial training for video snapshot compressive imaging. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12369, pp. 258–275. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58586-0_16

    Chapter  Google Scholar 

  11. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  12. Jalali, S., Yuan, X.: Snapshot compressed sensing: performance bounds and algorithms. IEEE Trans. Inf. Theory 65(12), 8005–8024 (2019)

    Article  MathSciNet  Google Scholar 

  13. Liao, X., Li, H., Carin, L.: Generalized alternating projection for weighted-\(\ell _{2,1}\) minimization with applications to model-based compressive sensing. SIAM J. Imaging Sci. 7(2), 797–823 (2014)

    Article  MathSciNet  Google Scholar 

  14. Liu, Y., Yuan, X., Suo, J., Brady, D., Dai, Q.: Rank minimization for snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 2990–3006 (2019)

    Article  Google Scholar 

  15. Llull, P., et al.: Coded aperture compressive temporal imaging. Opt. Express 21(9), 10526–10545 (2013)

    Article  Google Scholar 

  16. Ma, J., Liu, X., Shou, Z., Yuan, X.: Deep tensor ADMM-Net for snapshot compressive imaging. In: IEEE/CVF Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  17. Ma, X., Yuan, X., Fu, C., Arce, G.R.: Led-based compressive spectral-temporal imaging. Opt. Express 29(7), 10698–10715 (2021)

    Article  Google Scholar 

  18. Meng, Z., Jalali, S., Yuan, X.: Gap-net for snapshot compressive imaging. arXiv:2012:08364 (2020)

  19. Meng, Z., Ma, J., Yuan, X.: End-to-end low cost compressive spectral imaging with spatial-spectral self-attention. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12368, pp. 187–204. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58592-1_12

    Chapter  Google Scholar 

  20. Meng, Z., Qiao, M., Ma, J., Yu, Z., Xu, K., Yuan, X.: Snapshot multispectral endomicroscopy. Opt. Lett. 45(14), 3897–3900 (2020)

    Article  Google Scholar 

  21. Miao, X., Yuan, X., Pu, Y., Athitsos, V.: \(\lambda \)-Net: reconstruct hyperspectral images from a snapshot measurement. In: IEEE/CVF Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  22. Qiao, M., Liu, X., Yuan, X.: Snapshot spatial-temporal compressive imaging. Opt. Lett. 45(7), 1659–1662 (2020)

    Article  Google Scholar 

  23. Qiao, M., Liu, X., Yuan, X.: Snapshot temporal compressive microscopy using an iterative algorithm with untrained neural networks. Opt. Lett. 46(8), 1888–1891 (2021)

    Article  Google Scholar 

  24. Qiao, M., Meng, Z., Ma, J., Yuan, X.: Deep learning for video compressive sensing. APL Photonics 5(3), 030801 (2020)

    Article  Google Scholar 

  25. Selesnick, I., Bayram, I.: Total variation filtering. Connexions (2009)

    Google Scholar 

  26. Sun, Y., Yuan, X., Pang, S.: Compressive high-speed stereo imaging. Opt. Express 25(15), 18182–18190 (2017)

    Article  Google Scholar 

  27. Sun, Y., Yuan, X., Pang, S.: High-speed compressive range imaging based on active illumination. Opt. Express 24(20), 22836–22846 (2016)

    Article  Google Scholar 

  28. Wagadarikar, A., Pitsianis, N., Sun, X., Brady, D.: Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Express 17(8), 6368–6388 (2009)

    Article  Google Scholar 

  29. Wang, Z., Zhang, H., Cheng, Z., Chen, B., Yuan, X.: MetaSCI: scalable and adaptive reconstruction for video compressive sensing. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021

    Google Scholar 

  30. Yang, J., et al.: Compressive sensing by learning a Gaussian mixture model from measurements. IEEE Trans. Image Process. 24(1), 106–119 (2015)

    Article  MathSciNet  Google Scholar 

  31. Yang, J., et al.: Video compressive sensing using Gaussian mixture models. IEEE Trans. Image Process. 23(11), 4863–4878 (2014)

    Article  MathSciNet  Google Scholar 

  32. Yuan, X.: Generalized alternating projection based total variation minimization for compressive sensing. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2539–2543, September 2016

    Google Scholar 

  33. Yuan, X., Brady, D.J., Katsaggelos, A.K.: Snapshot compressive imaging: theory, algorithms, and applications. IEEE Sig. Process. Mag. 38(2), 65–88 (2021)

    Article  Google Scholar 

  34. Yuan, X., et al.: Low-cost compressive sensing for color video and depth. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

    Google Scholar 

  35. Yuan, X., Tsai, T.H., Zhu, R., Llull, P., Brady, D.J., Carin, L.: Compressive hyperspectral imaging with side information. IEEE J. Sel. Top. Sig. Process. 9(6), 964–976 (2015)

    Article  Google Scholar 

  36. Yuan, X., Liu, Y., Suo, J., Dai, Q.: Plug-and-Play algorithms for large-scale snapshot compressive imaging. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    Google Scholar 

  37. Yuan, X., Yang Liu, J.S., Durand, F., Dai, Q.: Plug-and-Play algorithms for video snapshot compressive imaging. arXiv: 2101.04822, January 2021

  38. Zhu, M., Wright, S.J., Chan, T.F.: Duality-based algorithms for total-variation-regularized image restoration. Comput. Optim. Appl. 47(3), 377–400 (2010). https://doi.org/10.1007/s10589-008-9225-2

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Westlake University, Hangzhou, 310024, Zhejiang, China

    Xin Yuan

Authors
  1. Xin Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Yuan .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Lu Fang

  2. Duke University, Durham, NC, USA

    Yiran Chen

  3. Shanghai Jiao Tong University, Shanghai, China

    Guangtao Zhai

  4. University of British Columbia, Vancouver, BC, Canada

    Jane Wang

  5. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Ruiping Wang

  6. Xidian University, Xi’an, China

    Weisheng Dong

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yuan, X. (2021). Various Plug-and-Play Algorithms with Diverse Total Variation Methods for Video Snapshot Compressive Imaging. In: Fang, L., Chen, Y., Zhai, G., Wang, J., Wang, R., Dong, W. (eds) Artificial Intelligence. CICAI 2021. Lecture Notes in Computer Science(), vol 13069. Springer, Cham. https://doi.org/10.1007/978-3-030-93046-2_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-93046-2_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-93045-5

  • Online ISBN: 978-3-030-93046-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation