Skip to main content
Springer Nature Link
Log in

Rethinking Deep Unrolled Model for Accelerated MRI Reconstruction

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15133))

Included in the following conference series:

  • 914 Accesses

Abstract

Magnetic Resonance Imaging (MRI) is a widely used imaging modality for clinical diagnostics and the planning of surgical interventions. Accelerated MRI seeks to mitigate the inherent limitation of long scanning time by reducing the amount of raw k-space data required for image reconstruction. Recently, the deep unrolled model (DUM) has demonstrated significant effectiveness and improved interpretability for MRI reconstruction, by truncating and unrolling the conventional iterative reconstruction algorithms with deep neural networks. However, the potential of DUM for MRI reconstruction has not been fully exploited. In this paper, we first enhance the gradient and information flow within and across iteration stages of DUM, then we highlight the importance of using various adjacent information for accurate and memory-efficient sensitivity map estimation and improved multi-coil MRI reconstruction. Extensive experiments on several public MRI reconstruction datasets show that our method outperforms existing MRI reconstruction methods by a large margin. The code is available at https://github.com/hellopipu/PromptMR-plus.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    It is usually a part of the acquired central k-space data.

References

  1. Adler, J., Öktem, O.: Learned primal-dual reconstruction. IEEE Trans. Med. Imaging 37(6), 1322–1332 (2018)

    Article  Google Scholar 

  2. Aggarwal, H.K., Mani, M.P., Jacob, M.: MoDL: model-based deep learning architecture for inverse problems. IEEE Trans. Med. Imaging 38(2), 394–405 (2018)

    Article  Google Scholar 

  3. Arnold, T.C., Freeman, C.W., Litt, B., Stein, J.M.: Low-field MRI: clinical promise and challenges. J. Magn. Reson. Imaging 57(1), 25–44 (2023)

    Article  Google Scholar 

  4. Arvinte, M., Vishwanath, S., Tewfik, A.H., Tamir, J.I.: Deep J-Sense: accelerated MRI reconstruction via unrolled alternating optimization. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 350–360. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_34

    Chapter  Google Scholar 

  5. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  6. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1), 1–122 (2011)

    Google Scholar 

  7. Candès, E.J., et al.: Compressive sampling. In: Proceedings of the International Congress of Mathematicians, vol. 3, pp. 1433–1452, Madrid, Spain (2006)

    Google Scholar 

  8. Deshmane, A., Gulani, V., Griswold, M.A., Seiberlich, N.: Parallel MR imaging. J. Magn. Reson. Imaging 36(1), 55–72 (2012)

    Article  Google Scholar 

  9. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(7), 2121–2159 (2011)

    Google Scholar 

  10. El-Rewaidy, H., et al.: Multi-domain convolutional neural network (MD-CNN) for radial reconstruction of dynamic cardiac MRI. Magn. Reson. Med. 85(3), 1195–1208 (2021)

    Article  Google Scholar 

  11. Eo, T., Jun, Y., Kim, T., Jang, J., Lee, H.J., Hwang, D.: KIKI-Net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn. Reson. Med. 80(5), 2188–2201 (2018)

    Article  Google Scholar 

  12. Fabian, Z., Tinaz, B., Soltanolkotabi, M.: HUMUS-Net: hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction. Adv. Neural. Inf. Process. Syst. 35, 25306–25319 (2022)

    Google Scholar 

  13. Gregor, K., LeCun, Y.: Learning fast approximations of sparse coding. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, pp. 399–406 (2010)

    Google Scholar 

  14. Griswold, M.A., et al.: Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn. Reson. Med. Official J. Int. Soc. Magn. Reson. Med. 47(6), 1202–1210 (2002)

    Article  Google Scholar 

  15. Hammernik, K., et al.: Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79(6), 3055–3071 (2018)

    Article  Google Scholar 

  16. Hosseini, S.A.H., Yaman, B., Moeller, S., Hong, M., Akçakaya, M.: Dense recurrent neural networks for accelerated MRI: history-cognizant unrolling of optimization algorithms. IEEE J. Sel. Topics Signal Process. 14(6), 1280–1291 (2020)

    Article  Google Scholar 

  17. Jun, Y., Shin, H., Eo, T., Hwang, D.: Joint deep model-based MR image and coil sensitivity reconstruction network (Joint-ICNet) for fast MRI. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5270–5279 (2021)

    Google Scholar 

  18. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  19. Lauterbur, P.C.: Image formation by induced local interactions: examples employing nuclear magnetic resonance. Nature 242(5394), 190–191 (1973)

    Google Scholar 

  20. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  21. Liu, X., Pang, Y., Jin, R., Liu, Y., Wang, Z.: Dual-domain reconstruction network with V-Net and K-Net for fast MRI. Magn. Reson. Med. 88(6), 2694–2708 (2022)

    Article  Google Scholar 

  22. Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. Official J. Int. Soc. Magn. Reson. Med. 58(6), 1182–1195 (2007)

    Article  Google Scholar 

  23. Monga, V., Li, Y., Eldar, Y.C.: Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Process. Mag. 38(2), 18–44 (2021)

    Article  Google Scholar 

  24. Munoz, C., Fotaki, A., Botnar, R.M., Prieto, C.: Latest advances in image acceleration: all dimensions are fair game. J. Magn. Reson. Imaging 57(2), 387–402 (2023)

    Article  Google Scholar 

  25. Nayak, K.S., Lim, Y., Campbell-Washburn, A.E., Steeden, J.: Real-time magnetic resonance imaging. J. Magn. Reson. Imaging 55(1), 81–99 (2022)

    Article  Google Scholar 

  26. Ongie, G., Jalal, A., Metzler, C.A., Baraniuk, R.G., Dimakis, A.G., Willett, R.: Deep learning techniques for inverse problems in imaging. IEEE J. Sel. Areas Inf. Theory 1(1), 39–56 (2020)

    Article  Google Scholar 

  27. Ottesen, J.A., Caan, M.W., Groote, I.R., Bjørnerud, A.: A densely interconnected network for deep learning accelerated MRI. Magn. Reson. Mater. Phys., Biol. Med. 36(1), 65–77 (2023)

    Article  Google Scholar 

  28. Pezzotti, N., et al.: An adaptive intelligence algorithm for undersampled knee MRI reconstruction. IEEE Access 8, 204825–204838 (2020)

    Article  Google Scholar 

  29. Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)

    Article  Google Scholar 

  30. Pruessmann, K.P., Weiger, M., Scheidegger, M.B., Boesiger, P.: SENSE: sensitivity encoding for fast MRI. Magn. Reson. Med. Official J. Int. Soc. Magn. Reson. Med. 42(5), 952–962 (1999)

    Article  Google Scholar 

  31. Putzky, P., et al.: i-RIM applied to the fastMRI challenge. arXiv preprint arXiv:1910.08952 (2019)

  32. Putzky, P., Welling, M.: Recurrent inference machines for solving inverse problems. arXiv preprint arXiv:1706.04008 (2017)

  33. Putzky, P., Welling, M.: Invert to learn to invert. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  34. Ramzi, Z., Chaithya, G., Starck, J.L., Ciuciu, P.: NC-PDNet: a density-compensated unrolled network for 2D and 3D non-cartesian MRI reconstruction. IEEE Trans. Med. Imaging 41(7), 1625–1638 (2022)

    Article  Google Scholar 

  35. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds.) Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015, Proceedings, Part III 18, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

  36. Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2017)

    Article  Google Scholar 

  37. Singh, D., Monga, A., de Moura, H.L., Zhang, X., Zibetti, M.V., Regatte, R.R.: Emerging trends in fast MRI using deep-learning reconstruction on undersampled k-Space data: a systematic review. Bioengineering 10(9), 1012 (2023)

    Article  Google Scholar 

  38. Song, J., Chen, B., Zhang, J.: Memory-augmented deep unfolding network for compressive sensing. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4249–4258 (2021)

    Google Scholar 

  39. Souza, R., et al.: An open, multi-vendor, multi-field-strength brain MR dataset and analysis of publicly available skull stripping methods agreement. Neuroimage 170, 482–494 (2018)

    Article  Google Scholar 

  40. Sriram, A., et al.: End-to-end variational networks for accelerated MRI reconstruction. In: Martel, A.L., et al. (eds.) Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, 4–8 October 2020, Proceedings, Part II 23, pp. 64–73. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_7

  41. Sun, J., Li, H., Xu, Z., et al.: Deep ADMM-Net for compressive sensing MRI. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

    Google Scholar 

  42. Tieleman, T.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26 (2012)

    Google Scholar 

  43. Uecker, M., et al.: ESPIRiT-an eigenvalue approach to autocalibrating parallel MRI: where sense meets grappa. Magn. Reson. Med. 71(3), 990–1001 (2014)

    Article  Google Scholar 

  44. Wang, C., et al.: CMRxRecon: an open cardiac MRI dataset for the competition of accelerated image reconstruction. arXiv preprint arXiv:2309.10836 (2023)

  45. Xin, B., Ye, M., Axel, L., Metaxas, D.N.: Fill the K-Space and refine the image: prompting for dynamic and multi-contrast MRI reconstruction. In: Camara, O., et al. (eds.) Statistical Atlases and Computational Models of the Heart. Regular and CMRxRecon Challenge Papers, STACOM 2023. LNCS, vol. 14507, pp. 261–273. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-52448-6_25

  46. Yiasemis, G., Sonke, J.J., Sánchez, C., Teuwen, J.: Recurrent variational network: a deep learning inverse problem solver applied to the task of accelerated MRI reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 732–741 (2022)

    Google Scholar 

  47. Ying, L., Sheng, J.: Joint image reconstruction and sensitivity estimation in SENSE (JSENSE). Magn. Reson. Med. Official J. Int. Soc. Magn. Reson. Med. 57(6), 1196–1202 (2007)

    Article  Google Scholar 

  48. You, D., Xie, J., Zhang, J.: ISTA-Net++: flexible deep unfolding network for compressive sensing. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2021)

    Google Scholar 

  49. Zbontar, J., et al.: FastMRI: an open dataset and benchmarks for accelerated MRI. arXiv preprint arXiv:1811.08839 (2018)

  50. Zeiler, M.D.: AdaDelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012)

  51. Zhang, J., Chen, B., Xiong, R., Zhang, Y.: Physics-inspired compressive sensing: beyond deep unrolling. IEEE Signal Process. Mag. 40(1), 58–72 (2023)

    Article  Google Scholar 

  52. Zhang, J., Ghanem, B.: ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1828–1837 (2018)

    Google Scholar 

  53. Zhang, J., Zhang, Z., Xie, J., Zhang, Y.: High-throughput deep unfolding network for compressive sensing MRI. IEEE J. Sel. Topics Signal Process. 16(4), 750–761 (2022)

    Article  Google Scholar 

  54. Zhang, Z., Liu, Y., Liu, J., Wen, F., Zhu, C.: AMP-Net: denoising-based deep unfolding for compressive image sensing. IEEE Trans. Image Process. 30, 1487–1500 (2020)

    Article  MathSciNet  Google Scholar 

  55. Zhao, R., et al.: FastMRI+, clinical pathology annotations for knee and brain fully sampled magnetic resonance imaging data. Sci. Data 9(1), 152 (2022)

    Article  Google Scholar 

  56. Zhou, B., Zhou, S.K.: DuDoRNet: learning a dual-domain recurrent network for fast MRI reconstruction with deep T1 prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4273–4282 (2020)

    Google Scholar 

Download references

Acknowledgments

This research has been partially funded by research grants to D. Metaxas through NSF: 2310966, 2235405, 2212301, 2003874, and FA9550-23-1-0417 and NIH 2R01HL127661.

Author information

Authors and Affiliations

  1. Rutgers University, New Brunswick, NJ, 08901, USA

    Bingyu Xin, Meng Ye & Dimitris N. Metaxas

  2. New York University Langone Medical Center, New York, NY, 10016, USA

    Leon Axel

Authors
  1. Bingyu Xin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leon Axel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dimitris N. Metaxas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bingyu Xin .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Hessen, Germany

    Stefan Roth

  4. Princeton University, Palo Alto, CA, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2040 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xin, B., Ye, M., Axel, L., Metaxas, D.N. (2025). Rethinking Deep Unrolled Model for Accelerated MRI Reconstruction. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15133. Springer, Cham. https://doi.org/10.1007/978-3-031-73226-3_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-73226-3_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-73225-6

  • Online ISBN: 978-3-031-73226-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics