Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Machine Learning for Medical Image Reconstruction

MLMIR 2021: Machine Learning for Medical Image Reconstruction pp 14–24Cite as

Efficient Image Registration Network for Non-Rigid Cardiac Motion Estimation

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12964))

Abstract

Cardiac motion estimation plays an essential role in motion-compensated cardiac Magnetic Resonance (MR) image reconstruction. In this work, we propose a robust and lightweight self-supervised deep learning registration framework, termed MRAFT, to estimate non-rigid cardiac motion. The proposed framework combines an efficient architecture with a novel degradation-restoration (DR) loss term, and an enhancement mask derived from a pre-trained segmentation network. This framework enables the prediction of both small and large cardiac motion more precisely, and allows us to handle through-plane motion in a 2D registration setting via the DR loss. The quantitative and qualitative experiments on a retrospective cohort of 42 in-house acquired 2D cardiac CINE MRIs indicate that the proposed method outperforms the competing approaches substantially, with more than 25% reduction in residual photometric error, and up to 100\(\times \) faster inference speed compared to conventional methods.

T. Küstner and K. Hammernik—Equal contribution.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Schmidt, M., et al.: Novel highly accelerated real-time CINE-MRI featuring compressed sensing with k-t regularization in comparison to TSENSE segmented and real-time Cine imaging. J. Cardiovasc. Magn. Reson. 15, P36 (2013)

    Article  Google Scholar 

  2. Hansen, M.S., Sorensen, T.S., Arai, A.E., Kellman, P.: Retrospective reconstruction of high temporal resolution cine images from real-time MRI using iterative motion correction. Magn. Reson. Med. 68(3), 741–750 (2012)

    Article  Google Scholar 

  3. Feng, L., et al.: 5D whole-heart sparse MRI. Magn. Reson. Med. 79(2), 826–838 (2018)

    Article  Google Scholar 

  4. Coppo, S., et al.: Free-running 4D whole-heart self-navigated golden angle MRI: initial results. Magn. Reson. Med. 74(5), 1306–16 (2015)

    Article  Google Scholar 

  5. Usman M., Ruijsink B., Nazir, et al. Free breathing whole-heart 3D CINE MRI with self-gated Cartesian trajectory. Magn Reson Imaging, 38:129–137, 2017

    Google Scholar 

  6. Küstner, T., Bustin, A., et al.: Fully self-gated free-running 3D Cartesian cardiac CINE with isotropic whole-heart coverage in less than 2 min. NMR Biomed. 34(1), e4409 (2021)

    Google Scholar 

  7. Liu, F., Li, D., Jin, X., Qiu, W., Xia, Q., Sun, B.: Dynamic cardiac MRI reconstruction using motion aligned locally low rank tensor (MALLRT). Magn. Reson. Imaging 66, 104–115 (2020)

    Article  Google Scholar 

  8. Mohsin, Y.Q., Poddar, S., Jacob, M.: Free-breathing and ungated cardiac MRI using iterative SToRM (i-SToRM). IEEE Trans. Med. Imaging 38(10), 2303–2313 (2019)

    Article  Google Scholar 

  9. Lingala, S.G., Hu, Y., DiBella, E., Jacob, M.: Accelerated dynamic MRI exploiting sparsity and low-rank structure: k-t SLR. IEEE Trans. Med. Imaging 30(5), 1042–1054 (2011)

    Article  Google Scholar 

  10. Mohsin, Y.Q., Lingala, S.G., DiBella, E., Jacob, M.: Accelerated dynamic MRI using patch regularization for implicit motion compensation. Magn. Reson. Med. 77(3), 1238–1248 (2017)

    Article  Google Scholar 

  11. Küstner, T., Bustin, A., Jaubert, O., Hajhosseiny, R., Masci, P.G., Neji, R., Botnar, R., Prieto, C.: Isotropic 3D Cartesian single breath-hold CINE MRI with multi-bin patch-based low-rank reconstruction. Magn. Reson. Med. 84(4), 2018–2033 (2020)

    Article  Google Scholar 

  12. Batchelor, P.G., Atkinson, D., Irarrazaval, P., Hill, D.L.G., Hajnal J., Larkman. D.: Matrix description of general motion correction applied to multishot images. Magn. Reson. Med. 54(5), 1273–1280 (2005)

    Google Scholar 

  13. Odille, F., Vuissoz, P.A., Marie, P.Y., et al.: Generalized reconstruction by inversion of coupled systems (GRICS) applied to free-breathing MRI. Magn. Reson. Med. 60(1), 146–157 (2008)

    Article  Google Scholar 

  14. Bustin, A., et al.: 3D whole-heart isotropic sub-millimeter resolution coronary magnetic resonance angiography with non-rigid motion-compensated PROST. J. Cardiovasc. Magn. Reson. 22, 24 (2020)

    Article  Google Scholar 

  15. Thirion, J.P.: Image matching as a diffusion process: an analogy with Maxwell’s demons. Med. Image Anal. 2(3), 243–260 (1998)

    Article  Google Scholar 

  16. Rueckert, D., Sonoda, L.I., Hayes, C., et al.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999)

    Article  Google Scholar 

  17. Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. Neuroimage 45(1), 61–72 (2009)

    Article  Google Scholar 

  18. Modat, M., Ridgway, G.R., Taylor, Z.A., et al.: Fast free-form deformation using graphics processing units. Comput. Meth. Prog. Bio. 98(3), 278–284 (2010)

    Article  Google Scholar 

  19. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)

    Article  Google Scholar 

  20. Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1), 185–203 (1981)

    Article  Google Scholar 

  21. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: European Conference on Computer Vision (ECCV), pp. 25–36 (2004)

    Google Scholar 

  22. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional Networks for Biomedical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

  23. Dosovitskiy, A., Fischer, P., Ilg, E., et al.: Flownet: learning optical flow with convolutional networks. In: IEEE International Conference on Computer Vision (ICCV), pp. 2758–2766 (2015)

    Google Scholar 

  24. Qin, C., et al.: Joint learning of motion estimation and segmentation for cardiac MR image sequences. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, pp. 472–480 (2018)

    Google Scholar 

  25. Morales, M., Izquierdo-Garcia, D., Aganj, I., Kalpathy-Cramer, J., Rosen, B., Catana, C.: Implementation and validation of a three-dimensional cardiac motion estimation network. Radiol. Artif. Intell. 1(4):e180080 (2019)

    Google Scholar 

  26. Zheng, Q., Delingette, H., Ayache, N.: Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow. Med. Image Anal. 56, 80–95 (2019)

    Article  Google Scholar 

  27. Qi, H., et al.: Non-rigid respiratory motion estimation of whole-heart coronary MR images using unsupervised deep learning. IEEE Trans. Med. Imaging 41(1), 444–454 (2021)

    Article  Google Scholar 

  28. Sun, D., Yang, X., Liu, M., Kautz J. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8934–8943 (2018)

    Google Scholar 

  29. Liu, P., King, I., Lyu, M.R., Xu, J.: DDflow: learning optical flow with unlabeled data distillation. In: The AAAI Conference on Artificial Intelligence, vol. 33, no. 1 (2019)

    Google Scholar 

  30. Yu, H., Chen, X., Shi, H., Chen, T., Huang, T.S., Sun, S.: Motion pyramid networks for accurate and efficient cardiac motion estimation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 436–446. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_42

    Chapter  Google Scholar 

  31. Jonschkowski, R., Stone, A., Barron, J.T., Gordon, A., Konolige, K., Angelova, A.: What matters in unsupervised optical flow. arXiv preprint arXiv:2006.04902 (2020)

  32. Teed, Z., Deng, J.: RAFT: recurrent all-pairs field transforms for optical flow. In: European Conference on Computer Vision (ECCV), pp. 402–419 (2020)

    Google Scholar 

  33. Bai, W., et al.: Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J. Cardiovasc. Magn. Reson. 20, 65 (2018)

    Google Scholar 

  34. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

  35. Sun, D., Roth, S., Black, M.J.: A quantitative analysis of current practices in optical flow estimation and the principles behind them. Int. J. Comput. Vis. 106(2), 115–137 (2014)

    Article  Google Scholar 

  36. Zhou, W., Bovik, A.C., Sheikh, H.R., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  37. Lopez-Perez, A., Sebastian, R., Ferrero, J.M.: Three-dimensional cardiac computational modelling: methods, features and applications. Biomed. Eng. 14(1), 35 (2015)

    Google Scholar 

  38. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the European Research Council (Grant Agreement no. 884622).

Author information

Authors and Affiliations

  1. Lab for AI in Medicine, Technical University of Munich, Munich, Germany

    Jiazhen Pan, Daniel Rueckert & Kerstin Hammernik

  2. Medical Image and Data Analysis (MIDAS.lab), University of Tübingen, Tübingen, Germany

    Thomas Küstner

  3. Department of Computing, Imperial College London, London, UK

    Daniel Rueckert & Kerstin Hammernik

Authors
  1. Jiazhen Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Rueckert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Küstner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kerstin Hammernik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiazhen Pan .

Editor information

Editors and Affiliations

  1. Sunnybrook Health Science Centre, Toronto, ON, Canada

    Nandinee Haq

  2. NYU Grossman School of Medicine, New York City, NY, USA

    Patricia Johnson

  3. Pattern Recognition Lab at FAU, Erlangen, Germany

    Andreas Maier

  4. Siemens Healthineers, Erlangen, Germany

    Tobias Würfl

  5. Ulsan National Institute of Science and Technology, Ulsan, Korea (Republic of)

    Jaejun Yoo

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 105 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pan, J., Rueckert, D., Küstner, T., Hammernik, K. (2021). Efficient Image Registration Network for Non-Rigid Cardiac Motion Estimation. In: Haq, N., Johnson, P., Maier, A., Würfl, T., Yoo, J. (eds) Machine Learning for Medical Image Reconstruction. MLMIR 2021. Lecture Notes in Computer Science(), vol 12964. Springer, Cham. https://doi.org/10.1007/978-3-030-88552-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88552-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88551-9

  • Online ISBN: 978-3-030-88552-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation