Abstract
Unrolled neural networks have enabled state-of-the-art reconstruction performance and fast inference times for the accelerated magnetic resonance imaging (MRI) reconstruction task. However, these approaches depend on fully-sampled scans as ground truth data which is either costly or not possible to acquire in many clinical medical imaging applications; hence, reducing dependence on data is desirable. In this work, we propose modeling the proximal operators of unrolled neural networks with scale-equivariant convolutional neural networks in order to improve the data-efficiency and robustness to drifts in scale of the images that might stem from the variability of patient anatomies or change in field-of-view across different MRI scanners. Our approach demonstrates strong improvements over the state-of-the-art unrolled neural networks under the same memory constraints both with and without data augmentations on both in-distribution and out-of-distribution scaled images without significantly increasing the train or inference time.
B. Gunel and A. Sahiner—Equal Contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aggarwal, H.K., Mani, M.P., Jacob, M.: MoDL: model-based deep learning architecture for inverse problems. IEEE Trans. Med. Imaging 38(2), 394–405 (2018)
Bekkers, E.J., Lafarge, M.W., Veta, M., Eppenhof, K.A.J., Pluim, J.P.W., Duits, R.: Roto-translation covariant convolutional networks for medical image analysis. ArXiv abs/1804.03393 (2018)
Celledoni, E., Ehrhardt, M.J., Etmann, C., Owren, B., Schonlieb, C.B., Sherry, F.: Equivariant neural networks for inverse problems. Inverse Probl. 37, 085006 (2021)
Chen, D., Tachella, J., Davies, M.E.: Equivariant imaging: learning beyond the range space. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4379–4388 (2021)
Cohen, T., Geiger, M., Köhler, J., Welling, M.: Spherical CNNs. ArXiv abs/1801.10130 (2018)
Cohen, T., Welling, M.: Group equivariant convolutional networks. In: International Conference on Machine Learning, pp. 2990–2999. PMLR (2016)
Darestani, M.Z., Heckel, R.: Accelerated MRI with un-trained neural networks. IEEE Trans. Comput. Imaging 7, 724–733 (2021)
Desai, A.D., et al.: VORTEX: Physics-driven data augmentations for consistency training for robust accelerated MRI reconstruction. In: MIDL (2022)
Desai, A.D., et al.: Noise2Recon: a semi-supervised framework for joint MRI reconstruction and denoising. arXiv preprint arXiv:2110.00075 (2021)
Fabian, Z., Heckel, R., Soltanolkotabi, M.: Data augmentation for deep learning based accelerated MRI reconstruction with limited data. In: International Conference on Machine Learning, pp. 3057–3067. PMLR (2021)
Gunel, B., Mardani, M., Chaudhari, A., Vasanawala, S., Pauly, J.: Weakly supervised MR image reconstruction using untrained neural networks. In: Proceedings of International Society of Magnetic Resonance in Medicine (ISMRM) (2021)
Hadamard, J.: Sur les problèmes aux dérivées partielles et leur signification physique. Princeton Univ. Bull. 49–52 (1902)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Knoll, F., et al.: Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge. Magn. Reson. Med. 84(6), 3054–3070 (2020)
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Resona. Med.: Off. J. Int. Soc. Magn. Reson. Med. 58(6), 1182–1195 (2007)
Lustig, M., Donoho, D.L., Santos, J.M., Pauly, J.M.: Compressed sensing MRI. IEEE Signal Process. Mag. 25(2), 72–82 (2008)
Mardani, M., et al.: Neural proximal gradient descent for compressive imaging. Adv. Neural Inf. Process. Syst. 31 (2018)
Müller, P., Golkov, V., Tomassini, V., Cremers, D.: Rotation-equivariant deep learning for diffusion MRI. ArXiv abs/2102.06942 (2021)
Ong, F., Amin, S., Vasanawala, S., Lustig, M.: Mridata.org: an open archive for sharing MRI raw data. In: Proceedings of International Society for Magnetic Resonance in Medicine, vol. 26, p. 1 (2018)
Ong, F., Lustig, M.: SigPy: a python package for high performance iterative reconstruction. In: Proceedings of the ISMRM 27th Annual Meeting, Montreal, Quebec, Canada, vol. 4819 (2019)
Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 127–239 (2014)
Pruessmann, K.P., Weiger, M., Scheidegger, M.B., Boesiger, P.: Sense: sensitivity encoding for fast MRI. Magn. Reson. Med.: Off. J. Int. Soc. Magn. Reson. Med. 42(5), 952–962 (1999)
Sahiner, A., Mardani, M., Ozturkler, B., Pilanci, M., Pauly, J.: Convex regularization behind neural reconstruction. arXiv preprint arXiv:2012.05169 (2020)
Sandino, C.M., Cheng, J.Y., Chen, F., Mardani, M., Pauly, J.M., Vasanawala, S.S.: Compressed sensing: from research to clinical practice with deep neural networks: shortening scan times for magnetic resonance imaging. IEEE Signal Process. Mag. 37(1), 117–127 (2020)
Sosnovik, I., Moskalev, A., Smeulders, A.: DISCO: accurate discrete scale convolutions. arXiv preprint arXiv:2106.02733 (2021)
Sosnovik, I., Szmaja, M., Smeulders, A.: Scale-equivariant steerable networks. arXiv preprint arXiv:1910.11093 (2019)
Sriram, A., et al.: End-to-end variational networks for accelerated MRI reconstruction. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 64–73. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_7
Sun, J., Li, H., Xu, Z., et al.: Deep ADMM-net for compressive sensing MRI. Adv. Neural Inf. Process. Syst. 29 (2016)
Vasanawala, S.S., et al.: Practical parallel imaging compressed sensing MRI: summary of two years of experience in accelerating body MRI of pediatric patients. In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1039–1043 (2011)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Weiler, M., Geiger, M., Welling, M., Boomsma, W., Cohen, T.: 3D steerable CNNs: learning rotationally equivariant features in volumetric data. In: NeurIPS (2018)
Winkels, M., Cohen, T.: 3D G-CNNs for pulmonary nodule detection. ArXiv abs/1804.04656 (2018)
Worrall, D.E., Welling, M.: Deep scale-spaces: equivariance over scale. ArXiv abs/1905.11697 (2019)
Ying, L., Sheng, J.: Joint image reconstruction and sensitivity estimation in SENSE (JSENSE). Magn. Reson. Med.: Off. J. Int. Soc. Magn. Reson. Med. 57(6), 1196–1202 (2007)
Acknowledgements
Beliz Gunel, Arda Sahiner, Shreyas Vasanawala, and John Pauly were supported by NIH R01EB009690 and NIH U01-EB029427-01. Mert Pilanci was partially supported by the National Science Foundation under grants IIS-1838179, ECCS- 2037304, DMS-2134248, and the Army Research Office. Arjun Desai and Akshay Chaudhari were supported by grants R01 AR077604, R01 EB002524, K24 AR062068, and P41 EB015891 from the NIH; the Precision Health and Integrated Diagnostics Seed Grant from Stanford University; National Science Foundation (DGE 1656518, CCF1763315, CCF1563078); DOD – National Science and Engineering Graduate Fellowship (ARO); Stanford Artificial Intelligence in Medicine and Imaging GCP grant; Stanford Human-Centered Artificial Intelligence GCP grant; Microsoft Azure through Stanford Data Science’s Cloud Resources Program; GE Healthcare and Philips.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gunel, B. et al. (2022). Scale-Equivariant Unrolled Neural Networks for Data-Efficient Accelerated MRI Reconstruction. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13436. Springer, Cham. https://doi.org/10.1007/978-3-031-16446-0_70
Download citation
DOI: https://doi.org/10.1007/978-3-031-16446-0_70
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16445-3
Online ISBN: 978-3-031-16446-0
eBook Packages: Computer ScienceComputer Science (R0)
