Skip to main content

Joint Optimization of Hadamard Sensing and Reconstruction in Compressed Sensing Fluorescence Microscopy

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 (MICCAI 2021)

Abstract

Compressed sensing fluorescence microscopy (CS-FM) proposes a scheme whereby less measurements are collected during sensing and reconstruction is performed to recover the image. Much work has gone into optimizing the sensing and reconstruction portions separately. We propose a method of jointly optimizing both sensing and reconstruction end-to-end under a total measurement constraint, enabling learning of the optimal sensing scheme concurrently with the parameters of a neural network-based reconstruction network. We train our model on a rich dataset of confocal, two-photon, and wide-field microscopy images comprising of a variety of biological samples. We show that our method outperforms several baseline sensing schemes and a regularized regression reconstruction algorithm. Our code is publicly-available at https://github.com/alanqrwang/csfm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Note we are not considering the effects of a point-spread function here.

  2. 2.

    Note we are defining one measurement to be the computation of one Hadamard coefficient without averaging, despite the fact that this is a two-step process in the physical imaging setup.

  3. 3.

    We used publicly-available code from: https://github.com/alanqrwang/HQSNet.

  4. 4.

    Negative values of b were replaced with 0 following the paper’s experiments [33].

References

  1. Bahadir, C.D., Wang, A.Q., Dalca, A.V., Sabuncu, M.R.: Deep-learning-based optimization of the under-sampling pattern in MRI. IEEE Trans. Comput. Imaging 6, 1139–1152 (2020)

    Article  Google Scholar 

  2. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  4. Chakrabarti, A.: Learning sensor multiplexing design through back-propagation. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 2016, pp. 3089–3097. Curran Associates Inc., Red Hook (2016)

    Google Scholar 

  5. Diamond, S., Sitzmann, V., Heide, F., Wetzstein, G.: Unrolled optimization with deep priors. CoRR abs/1705.08041 (2017)

    Google Scholar 

  6. Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.: Practical poissonian-gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Process. 17(10), 1737–1754 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  7. Gibson, G.M., Johnson, S.D., Padgett, M.J.: Single-pixel imaging 12 years on: a review. Opt. Express 28(19), 28190–28208 (2020)

    Article  Google Scholar 

  8. Hopt, A., Neher, E.: Highly nonlinear photodamage in two-photon fluorescence microscopy. Biophys. J . 80(4), 2029–2036 (2001)

    Article  Google Scholar 

  9. Jang, E., Gu, S., Poole, B.: Categorical reparametrization with gumble-softmax. In: Proceedings of the International Conference on Learning Representations 2017. OpenReviews.net, April 2017

    Google Scholar 

  10. Kellman, M., Bostan, E., Chen, M., Waller, L.: Data-driven design for Fourier ptychographic microscopy. In: 2019 IEEE International Conference on Computational Photography (ICCP), pp. 1–8 (2019)

    Google Scholar 

  11. Lee, S., Negishi, M., Urakubo, H., Kasai, H., Ishii, S.: Mu-net: multi-scale U-net for two-photon microscopy image denoising and restoration. Neural Netw. 125, 92–103 (2020)

    Article  Google Scholar 

  12. Lichtman, J., Conchello, J.: Fluorescence microscopy. Nat. Methods 2, 910–919 (2005)

    Article  Google Scholar 

  13. Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: a continuous relaxation of discrete random variables. In: International Conference on Learning Representations (2017)

    Google Scholar 

  14. Magidson, V., Khodjakov, A.: Circumventing photodamage in live-cell microscopy. In: Sluder, G., Wolf, D.E. (eds.) Digital Microscopy, Methods in Cell Biology, vol. 114, pp. 545–560. Academic Press (2013)

    Google Scholar 

  15. Makitalo, M., Foi, A.: Optimal inversion of the generalized Anscombe transformation for Poisson-Gaussian noise. IEEE Trans. Image Process. 22(1), 91–103 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  16. Parot, V.J., et al.: Compressed Hadamard microscopy for high-speed optically sectioned neuronal activity recordings. J. Phys. D Appl. Phys. 52(14), 144001 (2019)

    Article  Google Scholar 

  17. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  18. Sitzmann, V., et al.: End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging. ACM Trans. Graph. 37(4), 1–13 (2018)

    Article  Google Scholar 

  19. Streeter, L., Burling-Claridge, G.R., Cree, M.J., Künnemeyer, R.: Optical full Hadamard matrix multiplexing and noise effects. Appl. Opt. 48(11), 2078–2085 (2009)

    Article  Google Scholar 

  20. Studer, V., Bobin, J., Chahid, M., Mousavi, H.S., Candes, E., Dahan, M.: Compressive fluorescence microscopy for biological and hyperspectral imaging. Proc. Natl. Acad. Sci. 109(26), E1679–E1687 (2012)

    Article  Google Scholar 

  21. Sun, H., Dalca, A.V., Bouman, K.L.: Learning a probabilistic strategy for computational imaging sensor selection. In: 2020 IEEE International Conference on Computational Photography (ICCP), pp. 1–12 (2020)

    Google Scholar 

  22. Sun, M.J., Meng, L.T., Edgar, M.: A Russian Dolls ordering of the Hadamard basis for compressive single-pixel imaging. Sci. Rep. 7 (2017). Article number: 3464

    Google Scholar 

  23. Wang, A.Q., Dalca, A.V., Sabuncu, M.R.: Neural network-based reconstruction in compressed sensing MRI without fully-sampled training data. In: Deeba, F., Johnson, P., Würfl, T., Ye, J.C. (eds.) MLMIR 2020. LNCS, vol. 12450, pp. 27–37. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61598-7_3

    Chapter  Google Scholar 

  24. Wang, H., Rivenson, Y., Jin, Y.: Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods 16, 103–110 (2019)

    Article  Google Scholar 

  25. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  26. Weigert, M., Schmidt, U., Boothe, T.: Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15(12), 1090–1097 (2018)

    Article  Google Scholar 

  27. Wijesinghe, P., Escobet-Montalbán, A., Chen, M., Munro, P.R.T., Dholakia, K.: Optimal compressive multiphoton imaging at depth using single-pixel detection. Opt. Lett. 44(20), 4981 (2019)

    Article  Google Scholar 

  28. Xue, Y., Bigras, G., Hugh, J., Ray, N.: Training convolutional neural networks and compressed sensing end-to-end for microscopy cell detection. IEEE Trans. Med. Imaging 38(11), 2632–2641 (2019)

    Article  Google Scholar 

  29. Yang, Y., Sun, J., Li, H., Xu, Z.: Deep ADMM-Net for compressive sensing MRI. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016)

    Google Scholar 

  30. Yao, R., Ochoa, M., Yan, P.: Net-FLICS: fast quantitative wide-field fluorescence lifetime imaging with compressed sensing - a deep learning approach. Light Sci. Appl. 8 (2019). Article number: 26

    Google Scholar 

  31. Yu, X., Yang, F., Gao, B., Ran, J., Huang, X.: Deep compressive single pixel imaging by reordering Hadamard basis: a comparative study. IEEE Access 8, 55773–55784 (2020)

    Article  Google Scholar 

  32. Zhang, J., et al.: Extending LOUPE for K-space under-sampling pattern optimization in multi-coil MRI. In: Deeba, F., Johnson, P., Würfl, T., Ye, J.C. (eds.) MLMIR 2020. LNCS, vol. 12450, pp. 91–101. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61598-7_9

    Chapter  Google Scholar 

  33. Zhang, Y., et al.: A Poisson-Gaussian denoising dataset with real fluorescence microscopy images. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11702–11710 (2019)

    Google Scholar 

Download references

Acknowledgments

This work was, in part, supported by NIH R01 grants (R01LM012719 and R01AG053949, to MRS), the NSF NeuroNex grant (1707312, to MRS), an NSF CAREER grant (1748377, to MRS), and an NSF NeuroNex Hub grant (DBI-1707312, to CX).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alan Q. Wang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, A.Q., LaViolette, A.K., Moon, L., Xu, C., Sabuncu, M.R. (2021). Joint Optimization of Hadamard Sensing and Reconstruction in Compressed Sensing Fluorescence Microscopy. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12906. Springer, Cham. https://doi.org/10.1007/978-3-030-87231-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87231-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87230-4

  • Online ISBN: 978-3-030-87231-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics