Skip to main content
Springer Nature Link
Log in

Cine Cardiac MRI Reconstruction Using a Convolutional Recurrent Network with Refinement

  • Conference paper
  • First Online:
Statistical Atlases and Computational Models of the Heart. Regular and CMRxRecon Challenge Papers (STACOM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14507))

  • 766 Accesses

Abstract

Cine Magnetic Resonance Imaging (MRI) allows for understanding of the heart’s function and condition in a non-invasive manner. Undersampling of the k-space is employed to reduce the scan duration, thus increasing patient comfort and reducing the risk of motion artefacts, at the cost of reduced image quality. In this challenge paper, we investigate the use of a convolutional recurrent neural network (CRNN) architecture to exploit temporal correlations in supervised cine cardiac MRI reconstruction. This is combined with a single-image super-resolution refinement module to improve single coil reconstruction by 4.4% in structural similarity and 3.9% in normalised mean square error compared to a plain CRNN implementation. We deploy a high-pass filter to our \(\ell _1\) loss to allow greater emphasis on high-frequency details which are missing in the original data. The proposed model demonstrates considerable enhancements compared to the baseline case and holds promising potential for further improving cardiac MRI reconstruction.

Y. Xue, Y. Du, G. Carloni, E. Pachetti, C. Jordan—These authors contributed equally.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/f78bono/deep-cine-cardiac-mri.

References

  1. Aggarwal, H.K., Mani, M.P., Jacob, M.: MoDL: model-based deep learning architecture for inverse problems. IEEE Trans. Med. Imaging 38(2), 394–405 (2019). https://doi.org/10.1109/TMI.2018.2865356

    Article  Google Scholar 

  2. Antun, V., Renna, F., Poon, C., Adcock, B., Hansen, A.C.: On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc. Natl. Acad. Sci. 117(48), 30088–30095 (2020)

    Article  Google Scholar 

  3. Bilecen, B.B., Ayazoglu, M.: Bicubic++: slim, slimmer, slimmest - designing an industry-grade super-resolution network (2023). https://arxiv.org/abs/2305.02126

  4. Hammernik, K., et al.: Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79(6), 3055–3071 (2018). https://doi.org/10.1002/mrm.26977

    Article  Google Scholar 

  5. Han, X., Liu, Y., Lin, Y., Chen, K., Zhang, W., Liu, Q.: MDAMF: reconstruction of cardiac cine MRI under free-breathing using motion-guided deformable alignment and multi-resolution fusion (2023). https://arxiv.org/abs/2303.04968

  6. Han, Y., Yoo, J., Kim, H.H., Shin, H.J., Sung, K., Ye, J.C.: Deep learning with domain adaptation for accelerated projection-reconstruction MR. Magn. Reson. Med. 80(3), 1189–1205 (2018). https://doi.org/10.1002/mrm.27106

    Article  Google Scholar 

  7. Hyun, C.M., Kim, H.P., Lee, S.M., Lee, S., Seo, J.K.: Deep learning for undersampled MRI reconstruction. Phys. Med. Biol. 63(13), 135007 (2018). https://doi.org/10.1088/1361-6560/aac71a

  8. Jalal, A., Arvinte, M., Daras, G., Price, E., Dimakis, A.G., Tamir, J.: Robust compressed sensing MRI with deep generative priors. In: Advances in Neural Information Processing Systems, vol. 34, pp. 14938–14954 (2021)

    Google Scholar 

  9. Kofler, A., Haltmeier, M., Schaeffter, T., Kolbitsch, C.: An end-to-end-trainable iterative network architecture for accelerated radial multi-coil 2D cine MR image reconstruction. Med. Phys. 48(5), 2412–2425 (2021). https://doi.org/10.1002/mp.14809

    Article  Google Scholar 

  10. Küstner, T., et al.: CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions. Sci. Rep. 10, 13710 (2020). https://doi.org/10.1038/s41598-020-70551-8

    Article  Google Scholar 

  11. Lee, D., Yoo, J., Tak, S., Ye, J.C.: Deep residual learning for accelerated MRI using magnitude and phase networks. IEEE Trans. Biomed. Eng. 65(9), 1985–1995 (2018). https://doi.org/10.1109/TBME.2018.2821699

    Article  Google Scholar 

  12. Lyu, J., et al.: Region-focused multi-view transformer-based generative adversarial network for cardiac cine MRI reconstruction. Med. Image Anal. 85, 102760 (2023). https://doi.org/10.1016/j.media.2023.102760

    Article  Google Scholar 

  13. Patel, D., Sastry, P.S.: Memorization in deep neural networks: does the loss function matter? In: Karlapalem, K., et al. (eds.) PAKDD 2021. LNCS (LNAI), vol. 12713, pp. 131–142. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-75765-6_11

    Chapter  Google Scholar 

  14. Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V., Rueckert, D.: Convolutional recurrent neural networks for dynamic MR Image reconstruction. IEEE Trans. Med. Imaging 38(1), 280–290 (2019). https://doi.org/10.1109/TMI.2018.2863670

    Article  Google Scholar 

  15. Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR Image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2018). https://doi.org/10.1109/TMI.2017.2760978

    Article  Google Scholar 

  16. Terpstra, M.L., Maspero, M., Sbrizzi, A., van den Berg, C.A.: \(\perp \)-loss: a symmetric loss function for magnetic resonance imaging reconstruction and image registration with deep learning. Med. Image Anal. 80, 102509 (2022). https://doi.org/10.1016/j.media.2022.102509

    Article  Google Scholar 

  17. Tezcan, K.C., Baumgartner, C.F., Luechinger, R., Pruessmann, K.P., Konukoglu, E.: MR Image reconstruction using deep density priors. IEEE Trans. Med. Imaging 38(7), 1633–1642 (2019). https://doi.org/10.1109/TMI.2018.2887072

    Article  Google Scholar 

  18. Vornehm, M., Wetzl, J., Giese, D., Ahmad, R., Knoll, F.: Spatiotemporal variational neural network for reconstruction of highly accelerated cardiac cine MRI. Eur. Heart J. - Cardiovasc. Imaging 23(Supplement 2), 34–35 (2022). https://doi.org/10.1093/ehjci/jeac141.018

  19. Wang, C., et al.: Recommendation for cardiac magnetic resonance imaging-based phenotypic study: imaging part. Phenomics 1, 151–170 (2021). https://doi.org/10.1007/s43657-021-00018-x

    Article  Google Scholar 

  20. Wang, C., et al.: CMR\(\times \)Recon: an open cardiac MRI dataset for the competition of accelerated image reconstruction (2023)

    Google Scholar 

  21. Wang, S., et al.: Accelerating magnetic resonance imaging via deep learning. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 514–517 (2016). https://doi.org/10.1109/ISBI.2016.7493320

  22. Yang, G., et al.: DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans. Med. Imaging 37(6), 1310–1321 (2018). https://doi.org/10.1109/TMI.2017.2785879

    Article  Google Scholar 

  23. Yang, J., Küstner, T., Hu, P., Liò, P., Qi, H.: End-to-end deep learning of non-rigid groupwise registration and reconstruction of dynamic MRI. Front. Cardiovasc. Med. 9, 880186 (2022). https://doi.org/10.3389/fcvm.2022.880186

  24. Zhang, T., Pauly, J.M., Vasanawala, S.S., Lustig, M.: Coil compression for accelerated imaging with Cartesian sampling. Magn. Reson. Med. 69(2), 571–582 (2013). https://doi.org/10.1002/mrm.24267

    Article  Google Scholar 

  25. Zhang, Y., Hu, Y.: Dynamic cardiac MRI reconstruction using combined tensor nuclear norm and Casorati matrix nuclear norm regularizations. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–4 (2022). https://doi.org/10.1109/ISBI52829.2022.9761409

Download references

Acknowledgements

This work was supported in part by National Institutes of Health (NIH) grant 7R01HL148788-03. C. Jordan, Y. Du and Y. Xue thank additional financial support from the School of Engineering, the University of Edinburgh. Sotirios A. Tsaftaris also acknowledges the support of Canon Medical and the Royal Academy of Engineering and the Research Chairs and Senior Research Fellowships scheme (grant RCSRF1819\(\backslash \)8\(\backslash \)25). The authors would like to thank Dr. Chen and K. Vilouras for inspirational discussions and assistance.

Author information

Authors and Affiliations

  1. Institute for Imaging, Data and Communications, University of Edinburgh, Edinburgh, UK

    Yuyang Xue, Yuning Du & Sotirios A. Tsaftaris

  2. Institute of Information Science and Technologies “Alessandro Faedo”, National Research Council of Italy (CNR), Pisa, Italy

    Gianluca Carloni & Eva Pachetti

  3. Department of Information Engineering, University of Pisa, Pisa, Italy

    Gianluca Carloni & Eva Pachetti

  4. Institute for Infrastructure and Environment, University of Edinburgh, Edinburgh, UK

    Connor Jordan

  5. The Alan Turing Institute, London, UK

    Sotirios A. Tsaftaris

Authors
  1. Yuyang Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuning Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gianluca Carloni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eva Pachetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Connor Jordan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sotirios A. Tsaftaris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Connor Jordan .

Editor information

Editors and Affiliations

  1. Universitat Pompeu Fabra, Barcelona, Spain

    Oscar Camara

  2. King's College London, London, UK

    Esther Puyol-Antón

  3. Inria, Sophia Antipolis, France

    Maxime Sermesant

  4. King's College London, London, UK

    Avan Suinesiaputra

  5. Technische Universiteit Delft, Delft, The Netherlands

    Qian Tao

  6. Fudan University, Shanghai, China

    Chengyan Wang

  7. King's College London, London, UK

    Alistair Young

CMRxRecon Summary Information

CMRxRecon Summary Information

  • Task: Cine. Data used: Single-channel. Docker submitted: Yes.

    Final model: CRNN backbone with SISR module (2.3M parameters).

    Unrolling: Yes. Domain: Complex and amplitude. k-space fidelity: Yes.

    Pretraining: No. Augmentation/standardisation: No.

    Trained on: 3\(\times \)GPU (40GB vRAM). Loss function: \(\perp \)-SSIM-\(\ell _1\)

    Training time: 23h (33 epochs). Inference time: 1 h 45 min (120 subjects).

    Test results: PSNR = 35.582, SSIM = 0.946, NMSE = 0.0374

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xue, Y., Du, Y., Carloni, G., Pachetti, E., Jordan, C., Tsaftaris, S.A. (2024). Cine Cardiac MRI Reconstruction Using a Convolutional Recurrent Network with Refinement. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxRecon Challenge Papers. STACOM 2023. Lecture Notes in Computer Science, vol 14507. Springer, Cham. https://doi.org/10.1007/978-3-031-52448-6_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-52448-6_40

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-52447-9

  • Online ISBN: 978-3-031-52448-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics