Skip to main content
SpringerLink
Log in

Compressed sensing using generative models based on fisher information

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In compressed sensing applications, self-learning generative models have attracted increasing attention because they provide guarantees that are similar to those of standard compressed sensing without employing sparsity. However, improving the performances of a generative model is challenging. In this paper, we improve the recovery performances of generative models (generative adversarial networks) by making use of prior knowledge about the support of the vector of the original signal in the relevant domain. We demonstrate the advantage of using a parametric model with the Fisher distance metric for the exploitation of a distribution over the support when constraints on the distribution have been specified. We combine the generative model with the Fisher distance to study the recovery of sparse signals that satisfy a distribution for the purpose of improving the recovery performance of the model when there are some constraints on the distribution. Finally, we present the results of extensive experiments conducted on simulated signals and imaging signals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Abdur RM, Simon D (2014) Energy-efficient sensing in wireless sensor networks using compressed sensing. Sensors 14(2):2822–2859

    Article  Google Scholar 

  2. Wang DX, Mackie TR, Tome WA (2009) Proton computed tomography reconstruction using compressed sensing and prior image constrained compressed sensing. Med Phys 36(6):2443

    Article  Google Scholar 

  3. Bo K, Gengxin Z, Wei Z, Dongming B, Zhidong X (2016) Data persistence in planetary surface network using raptor codes and probabilistic broadcasting Int. J Distrib Sens Netw 12(8):51–65

  4. Candes EJ, Wakin MB (2008) An introduction to compressive sampling. IEEE Signal Proc Mag 25(2):21–30

    Article  Google Scholar 

  5. Muthukrishnan S (2003) Data streams: algorithms and applications. Found Trends Theor Comput Sci 1(2):413–413

    MathSciNet  Google Scholar 

  6. Candes EJ, Tao T (2005) Decoding by linear programming. IEEE Trans Inf Theory 51(12):4203–4215

    Article  MathSciNet  Google Scholar 

  7. Blumensath T, Davies ME (2009) Iterative hard thresholding for compressed sensing. Appl Comput Harmon Anal 27(3):265–274

    Article  MathSciNet  Google Scholar 

  8. Baron D, Sarvotham S, Baraniuk RG (2010) Bayesian compressive sensing via belief propagation. IEEE Signal Proc 58(3):269–280

    Article  MathSciNet  Google Scholar 

  9. Chen SS, Donoho DL, Saunders MA (1998) Atomic decomposition by basis pursuit. SIAM J Sci Comput 20(1):33–61

    Article  MathSciNet  Google Scholar 

  10. Gilbert A, Guha S, Indyk P, Muthukrishnan S, Strauss M (2000) Near-optimal sparse Fourier representations via sampling.In: Proceedings of the 34th annual ACM symposium on theory of computing, pp 152−161

  11. Cai TT, Wang L (2011) Orthogonal matching pursuit for sparse signal recovery with noise. IEEE Trans Inf Theory 57(7):4680–4688

    Article  MathSciNet  Google Scholar 

  12. Li S, Fang L (2011) Signal denoising with random refined orthogonal matching pursuit. IEEE Instrum Meas Soc 23:26–34

    Article  Google Scholar 

  13. Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43(1):129–159

    Article  MathSciNet  Google Scholar 

  14. Bazerque JA, Giannakis GB (2013) Nonparametric basis pursuit via sparse kernel-based learning: a unifying view with advances in blind methods. IEEE Signal Process Mag 30(4):112–125

    Article  Google Scholar 

  15. Baron D, Sarvotham S, Baraniuk RG (2010) Bayesian compressive sensing via belief propagation. IEEE Trans Signal Process 58(1):269–280

    Article  MathSciNet  Google Scholar 

  16. He L, Carin L (2009) Exploiting structure in wavelet-based bayesian compressive sensing. IEEE Trans Signal Process 57(9):3488–3497

    Article  MathSciNet  Google Scholar 

  17. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  18. Palangi H, Ward R, Deng Li (2017) Convolutional deep stacking networks for distributed compressive sensing. Signal Process 131:181–189

    Article  Google Scholar 

  19. Wang M, Xiao CB, Ning ZH, Li T, Gong B (2019) Neural networks for compressed sensing based on information geometry. Circuits Syst Signal Process 38(2):569–589

    Article  MathSciNet  Google Scholar 

  20. Merhej D et al (2011) Embedding prior knowledge within compressed sensing by neural networks. IEEE Trans Neural Netw 22(10):1638–1649

    Article  Google Scholar 

  21. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst vol 2, pp 2672–2680

  22. Bora A, Jalal A, Price E, Dimakis AG (2018) Compressed sensing using generative models. In: Proceedings of the 34th international conference on machine learning, PMLR, vol 70, pp 537–546

  23. Mardani M, Gong E, Cheng JY, Vasanawala S, Xing L, Pauly JM (2017) Deep generative adversarial networks for compressed sensing (GANCS) automates MRI. In: 31st Conference on neural information processing systems (NIPS 2017), Long Beach, CA, USA

  24. Quan TM, Nguyen-Duc T, Jeong W-K (2018) Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging 37(6):1488–1497

    Article  Google Scholar 

  25. Yang G, Yu S, Dong H, Slabaugh G, Dragotti PL, Ye X, Liu F, Arridge S, Keegan J, Guo Y, Firmin D (2018) DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging 37(6):1310–1321

    Article  Google Scholar 

  26. Hao L, Dawei X, Xingwu Y, Xinyu Z (2019) Compressed sensing method for IGBT high-speed switching time on-line monitoring. IEEE Trans Ind Electron 66(4):3185–3195

    Article  Google Scholar 

  27. Razzaque MA, Simon D (2014) Energy-efficient sensing in wireless sensor networks using compressed sensing. Sensors 14:2822–2859

    Article  Google Scholar 

  28. Hong-An L, Zhan-Li L, Zhuo-Ming D (2017) A reconstruction method of compressed sensing 3D medical models based on the weighted 0-norm. J Med Imaging Health Inform 7(2):416–420

    Article  Google Scholar 

  29. Majumdar A, Ward RK (2010) Compressive color imaging with group-sparsity on analysis prior. In: Proc. 17th IEEE Int. Conf. Image Process. (ICIP), Sep. 26–29, pp 1337–1340

  30. Yu L et al (2011) Bayesian compressive sensing for clustered sparse signals. In: Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 22–27, pp 3948–3951

  31. Vera E et al (2009) Bayesian compressive sensing of wavelet coefficients using multiscale Laplacian priors. In: Proc. IEEE/Signal Process. 15th Workshop Statist. Signal Process. (SSP), Aug. 31–Sep. 3, pp 229–232

  32. Amari S (1985) Differential-geometrical methods in statistics. Lect Notes Stat 28(5):168–178

    MathSciNet  MATH  Google Scholar 

  33. Duong TV, Phung DQ, Bui HH, Venkatesh S (2006) Human behavior recognition with generic exponential family duration modeling in the hidden semi-Markov model. In: International conference on pattern recognition, pp 202–208

  34. Ravanbakhsh S, Poczos B, Schneider J (2016) Stochastic neural networks with monotonic activation functions. In: Appearing in Proceedings of the 19th international conference on artificial intelligence and statistics (AISTATS) 2016, Cadiz, Spain. JMLR: W&CP vol 41

  35. Rao CR (1945) Information and accuracy attainable in the estimation of statistical parameters. Bull Calcutta Math Soc 37:81–91

    MathSciNet  MATH  Google Scholar 

  36. Costa SIR, Santos SA, Strapasson JE (2014) Fisher information distance: a geometrical reading. Discrete Appl Math S0166218X14004211

  37. Arwini K, Dodson CTJ (2008) Information geometry: near randomness and near independence. Lecture Notes in Mathematics, Springer

  38. Hazewinkel M (ed) (2001) Jensen inequality. In: Encyclopedia of mathematics. Springer Science + Business Media B.V./Kluwer Academic Publishers, ISBN 978-1-55608-010-4

  39. Candès EJ, Romberg JK, Tao T (2006) Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 59(8):1207–1223

    Article  MathSciNet  Google Scholar 

  40. Slavche P, Venceslav K (2015) Asymptotic capacity lower bound for an OFDM system with lasso compressed sensing channel estimation for Bernoulli-Gaussian channel. IEEE Commun Soc 19(3):379–382

    Google Scholar 

  41. Candes EJ, Tao T (2006) Near-optimal signal recovery from random projections: universal encoding strategies. IEEE Trans Inf Theory 52(12):5406–5425

    Article  MathSciNet  Google Scholar 

  42. LeCun M, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  43. Liu Z, Luo P, Wang X et al (2015) Deep learning face attributes in the wild. In: Proceedings of the IEEE international conference on computer vision, pp 3730–3738

Download references

Acknowledgements

This research was funded by the Beijing Science and Technology Planning Program of China (Z171100004717001), Beijing Natural Science Foundation (4172002), and Natural Science Foundation of China (61701009).

Author information

Authors and Affiliations

  1. Beijing Information Science and Technology University, Beijing, China

    Meng Wang

  2. Beijing University of Technology, Beijing, China

    Jing Yu, Zhen-Hu Ning & Chuang-Bai Xiao

Authors
  1. Meng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen-Hu Ning
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chuang-Bai Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhen-Hu Ning.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, M., Yu, J., Ning, ZH. et al. Compressed sensing using generative models based on fisher information. Int. J. Mach. Learn. & Cyber. 12, 2747–2759 (2021). https://doi.org/10.1007/s13042-021-01337-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01337-1

Keywords

Search

Navigation