Abstract
A single-pixel camera is a spatial-multiplexing device that reconstructs an image from a sequence of projections of the scene onto some patterns. This architecture is used, for example, to assist neurosurgery with hyperspectral imaging. However, capturing dynamic scenes is very challenging: as the different projections measure different frames of the scene, standard reconstruction approaches suffer from strong motion artifacts. This paper presents a general framework to reconstruct a moving scene with two main contributions. First, we extend the field of view of the camera beyond that defined by the spatial light modulator, which dramatically reduces the model mismatch. Second, we propose to build the dynamic system matrix without warping the patterns, effectively dismissing discretization errors. Numerical experiments show that both our contributions are necessary for an artifact-free reconstruction. The influence of a reduced measured set, robustness to noise and to motion errors were also evaluated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Beneti Martins, G., Mahieu-Williame, L., Baudier, T., Ducros, N.: OpenSpyrit: an ecosystem for open single-pixel hyperspectral imaging. Opt. Express 31(10), 15599 (2023). https://doi.org/10.1364/OE.483937, https://opg.optica.org/abstract.cfm?URI=oe-31-10-15599
Bi, S., Zeng, X., Tang, X., Qin, S., Lai, K.W.C.: Compressive video recovery using block match multi-frame motion estimation based on single pixel cameras. Sensors 16(3), 318 (2016)
Bravo, J.J., Olson, J.D., Davis, S.C., Roberts, D.W., Paulsen, K.D., Kanick, S.C.: Hyperspectral data processing improves PpIX contrast during fluorescence guided surgery of human brain tumors. Sci. Rep. 7(1), 9455 (2017)
Duarte, M.F., et al.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25(2), 83–91 (2008)
Edgar, M.P., Gibson, G.M., Padgett, M.J.: Principles and prospects for single-pixel imaging. Nat. Photon. 13(1), 13–20 (2019)
Gibson, G.M., Johnson, S.D., Padgett, M.J.: Single-pixel imaging 12 years on: a review. Opt. Express 28(19), 28190–28208 (2020)
Hahn, B.N.: Efficient algorithms for linear dynamic inverse problems with known motion. Inverse Prob. 30(3), 035008 (2014)
Jiao, J., et al.: Direct parametric reconstruction with joint motion estimation/correction for dynamic brain PET data. IEEE Trans. Med. Imaging 36(1), 203–213 (2016)
Jiao, S., Sun, M., Gao, Y., Lei, T., Xie, Z., Yuan, X.: Motion estimation and quality enhancement for a single image in dynamic single-pixel imaging. Opt. Express 27(9), 12841 (2019).https://doi.org/10.1364/OE.27.012841, https://opg.optica.org/abstract.cfm?URI=oe-27-9-12841
Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)
Maitre, T., Bretin, E., Mahieu-Williame, L., Sdika, M., Ducros, N.: Hybrid single-pixel camera for dynamic hyperspectral imaging. In: 2024 IEEE 21st International Symposium on Biomedical Imaging (ISBI). IEEE (2024)
Mur, A.L., Leclerc, P., Peyrin, F., Ducros, N.: Single-pixel image reconstruction from experimental data using neural networks. Opt. Express 29(11), 17097–17110 (2021)
Mur, A.L., Peyrin, F., Ducros, N.: Recurrent neural networks for compressive video reconstruction. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1651–1654. IEEE (2020)
Mur, A.L., Peyrin, F., Ducros, N.: Deep expectation-maximization for single-pixel image reconstruction with signal-dependent noise. IEEE Trans. Comput. Imaging 8, 759–769 (2022)
Rit, S., Sarrut, D., Desbat, L.: Comparison of analytic and algebraic methods for motion-compensated cone-beam CT reconstruction of the thorax. IEEE Trans. Med. Imaging 28(10), 1513–1525 (2009)
Sankaranarayanan, A.C., Xu, L., Studer, C., Li, Y., Kelly, K.F., Baraniuk, R.G.: Video compressive sensing for spatial multiplexing cameras using motion-flow models. SIAM J. Imaging Sci. 8(3), 1489–1518 (2015)
Scannell, C.M., Villa, A.D., Lee, J., Breeuwer, M., Chiribiri, A.: Robust non-rigid motion compensation of free-breathing myocardial perfusion MRI data. IEEE Trans. Med. Imaging 38(8), 1812–1820 (2019)
Sdika, M., Alston, L., Rousseau, D., Guyotat, J., Mahieu-Williame, L., Montcel, B.: Repetitive motion compensation for real time intraoperative video processing. Med. Image Anal. 53, 1–10 (2019)
Xu, Y., Lu, L., Saragadam, V., Kelly, K.F.: A compressive hyperspectral video imaging system using a single-pixel detector. Nat. Commun. 15(1), 1456 (2024)
Acknowledgments
This study was funded by the French National Research Agency (ANR), under Grant ANR-22-CE19-0030-01 (ULHYB Project) and performed within the framework of the LABEX PRIMES (ANR-11-LABX-0063) of Université de Lyon, within the program “Investissements d’Avenir” operated by the French National Research Agency (ANR).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Maitre, T., Bretin, E., Phan, R., Ducros, N., Sdika, M. (2024). Dynamic Single-Pixel Imaging on an Extended Field of View Without Warping the Patterns. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15007. Springer, Cham. https://doi.org/10.1007/978-3-031-72104-5_27
Download citation
DOI: https://doi.org/10.1007/978-3-031-72104-5_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72103-8
Online ISBN: 978-3-031-72104-5
eBook Packages: Computer ScienceComputer Science (R0)