Skip to main content
Springer Nature Link
Log in

Coil-Agnostic Attention-Based Network for Parallel MRI Reconstruction

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13846))

Included in the following conference series:

Abstract

Magnetic resonance imaging (MRI) is widely used in clinical diagnosis. However, as a slow imaging modality, the long scan time hinders its development in time-critical applications. The acquisition process can be accelerated by types of under-sampling strategies in k-space and reconstructing images from a few measurements. To reconstruct the image, many parallel imaging methods use the coil sensitivity maps to fold multiple coil images with model-based or deep learning-based estimation methods. However, they can potentially suffer from the inaccuracy of sensitivity estimation. In this work, we propose a novel coil-agnostic attention-based framework for multi-coil MRI reconstruction which completely avoids the sensitivity estimation and performs data consistency (DC) via a sensitivity-agnostic data aggregation consistency block (DACB). Experiments were performed on the FastMRI knee dataset and show that the proposed DACB and attention module-integrated framework outperforms other deep learning-based algorithms in terms of image quality and reconstruction accuracy. Ablation studies also indicate the superiority of DACB over conventional DC methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aggarwal, H., Mani, M., Jacob, M.: MoDL: model-based deep learning architecture for inverse problems. IEEE Trans. Med. Imaging 38(2), 394–405 (2019). https://doi.org/10.1109/TMI.2018.2865356

    Article  Google Scholar 

  2. Ahmad, R., et al.: Plug-and-play methods for magnetic resonance imaging: using denoisers for image recovery. IEEE Signal Process. Mag. 37(1), 105–116 (2020). https://doi.org/10.1109/MSP.2019.2949470

    Article  Google Scholar 

  3. Anuroop, S., Jure, Z., Tullie, M., Lawrence, Z., Aaron, D., K.S., D.: GrappaNet: combining parallel imaging with deep learning for multi-coil MRI reconstruction. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14303–14310 (2020). https://doi.org/10.1109/CVPR42600.2020.01432

  4. Arvinte, M., Vishwanath, S., Tewfik, A., Tamir, J.: Deep J-Sense: accelerated MRI reconstruction via unrolled alternating optimization (2021)

    Google Scholar 

  5. Bako, S., et al.: Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graph. (TOG) 36(4), 97 (2017)

    Google Scholar 

  6. Bhadra, S., Zhou, W., Anastasio, M.: Medical image reconstruction with image-adaptive priors learned by use of generative adversarial networks. Medical Imag. 2020: Phys. Med. Imag. 11312, 206–213 (2020). https://doi.org/10.1117/12.2549750

  7. Bińkowski, M., Sutherland, D., Arbel, M., Gretton, A.: Demystifying MMD GANs. In: International Conference on Learning Representations (2018)

    Google Scholar 

  8. Duan, J., et al.: VS-Net: variable splitting network for accelerated parallel MRI reconstruction. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 713–722. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_78

    Chapter  Google Scholar 

  9. Fair, M., Gatehouse, P., DiBella, E., Firmin, D.: A review of 3D first-pass, whole-heart, myocardial perfusion cardiovascular magnetic resonance. J. Cardiovasc. Mag. Reson. 17, 68 (2015). https://doi.org/10.1186/s12968-015-0162-9

  10. Goodfellow, I., et al.: Generative adversarial networks. Adv. Neural. Inf. Process. Syst. 27, 2672–2680 (2014)

    Google Scholar 

  11. Griswold, M., et al.: Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn. Reson. Med. 47(6), 1202–1210 (2002). https://doi.org/10.1002/mrm.10171

    Article  Google Scholar 

  12. Hammernik, K., et al.: Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79(6), 3055–3071 (2017)

    Article  Google Scholar 

  13. Hong, M., Yu, Y., Wang, H., Liu, F., Crozier, S.: Compressed sensing MRI with singular value decomposition-based sparsity basis. Phys. Med. Biol. 56(19), 6311–6325 (2021)

    Article  Google Scholar 

  14. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017)

    Google Scholar 

  15. Huang, Q., Yang, D., Wu, P., Qu, H., Yi, J., Metaxas, D.: MRI reconstruction via cascaded channel-wise attention network. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1622–1626 (2019). https://doi.org/10.1109/ISBI.2019.8759423

  16. Jun, Y., Shin, H., Eo, T., Hwang, D.: Joint deep model-based MR image and coil sensitivity reconstruction network (Joint-ICNet) for fast MRI. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5270–5279 (2021)

    Google Scholar 

  17. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations (ICLR) (2015). http://arxiv.org/abs/1412.6980

  18. Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25 (2012). https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

  19. Lee, D., Yoo, J., Tak, S., Ye, J.: Deep residual learning for accelerated MRI using magnitude and phase networks. IEEE Trans. Biomed. Eng. 65(9), 1985–1995 (2018)

    Article  Google Scholar 

  20. Lingala, S., Jacob, M.: Blind compressive sensing dynamic MRI. IEEE Trans. Med. Imaging 32(6), 1132–1145 (2013)

    Article  Google Scholar 

  21. Liu, J., Yaghoobi, M.: Fine-grained MRI reconstruction using attentive selection generative adversarial networks. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1155–1159 (2021)

    Google Scholar 

  22. Liu, R., Zhang, Y., Cheng, S., Luo, Z., Fan, X.: A deep framework assembling principled modules for CS-MRI: unrolling perspective, convergence behaviors, and practical modeling. IEEE Trans. Med. Imaging 39(12), 4150–4163 (2020). https://doi.org/10.1109/TMI.2020.3014193

    Article  Google Scholar 

  23. Liu, Y., Liu, Q., Zhang, M., Yang, Q., Wang, S., Liang, D.: IFR-Net: Iterative feature refinement network for compressed sensing MRI. IEEE Trans. Comput. Imag. 6, 434–446 (2020)

    Google Scholar 

  24. Mao, X., Li, Q., Xie, H., Lau, R., Wang, Z., Smolley, S.: Least squares generative adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2813–2821 (2017)

    Google Scholar 

  25. Mardani, M., et al.: Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans. Med. Imaging 38(1), 167–179 (2019). https://doi.org/10.1109/TMI.2018.2858752

    Article  Google Scholar 

  26. Narnhofer, D., Hammernik, K., Knoll, F., Pock, T.: Inverse GANs for accelerated MRI reconstruction. Wavelets Spars. XVIII 11138, 111381A (2019). https://doi.org/10.1117/12.2527753

  27. Pezzotti, N., et al.: An adaptive intelligence algorithm for undersampled knee MRI reconstruction. IEEE Access 8, 204825–204838 (2020)

    Article  Google Scholar 

  28. Ravishankar, S., Bresler, Y.: MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Med. Imaging 30(5), 1028–1041 (2011)

    Article  Google Scholar 

  29. Sajjadi, M., Schölkopf, B., Hirsch, M.: EnhanceNet: single image super-resolution through automated texture synthesis. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4501–4510 (2017). https://arxiv.org/abs/1612.07919/

  30. Schlemper, J., Caballero, J., Hajnal, J., Price, A., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2018). https://doi.org/10.1109/TMI.2017.2760978

    Article  Google Scholar 

  31. Schlemper, J., et al.: Data consistency networks for (calibration-less) accelerated parallel MR image reconstruction. Abstract from ISMRM 27th Annual Meeting and Exhibition (2019)

    Google Scholar 

  32. Sriram, A., et al.: End-to-end variational networks for accelerated MRI reconstruction. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 64–73 (2020)

    Google Scholar 

  33. Tezcan, K., Baumgartner, C., Luechinger, R., Pruessmann, K., Konukoglu, E.: MR image reconstruction using deep density priors. IEEE Trans. Med. Imaging 38(7), 1633–1642 (2019). https://doi.org/10.1109/TMI.2018.2887072

    Article  Google Scholar 

  34. Uecker, M., et al.: ESPIRiT-an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magn. Reson. Med. 71(3), 990–1001 (2014)

    Article  Google Scholar 

  35. Valsesia, D., Fracastoro, G., Magli, E.: Deep graph-convolutional image denoising. IEEE Trans. Image Process. 29, 8226–8237 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  36. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems 30 (2017). https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

  37. Wang, Y., Tao, X., Qi, X., Shen, X., Jia, J.: Image inpainting via generative multi-column convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 331–340 (2018)

    Google Scholar 

  38. Yang, G., et al.: DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans. Med. Imaging 37(6), 1310–1321 (2018)

    Article  Google Scholar 

  39. Ying, L., Sheng, J.: Joint image reconstruction and sensitivity estimation in SENSE (JSENSE). Magn. Reson. Med. 57(6), 1196–1202 (2007)

    Article  Google Scholar 

  40. Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 63–79. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11021-5_5

    Chapter  Google Scholar 

  41. Yuan, Z., et al.: SARA-GAN: self-attention and relative average discriminator based generative adversarial networks for fast compressed sensing MRI reconstruction. Front. Neuroinform. 14, 1–12 (2020). https://doi.org/10.3389/fninf.2020.611666

    Article  Google Scholar 

  42. Zbontar, J., et al.: FastMRI: an open dataset and benchmarks for accelerated MRI. CoRR abs/1811.08839 (2018)

    Google Scholar 

  43. Zhang, C., Liu, Y., Shang, F., Li, Y., Liu, H.: A novel learned primal-dual network for image compressive sensing. IEEE Access 9, 26041–26050 (2021). https://doi.org/10.1109/ACCESS.2021.3057621

    Article  Google Scholar 

  44. Zhang, J., Ghanem, B.: ISTA-Net: iterative shrinkage-thresholding algorithm inspired deep network for image compressive sensing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1828–1837 (2018)

    Google Scholar 

  45. Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration WTH neural networks. IEEE Trans. Comput. Imag. 3(1), 47–57 (2017)

    Article  Google Scholar 

  46. Zhou, S., Zhang, J., Pan, J., Xie, H., Zuo, W., Ren, J.: Spatio-temporal filter adaptive network for video deblurring. In: Proceedings of the IEEE International Conference on Computer Vision (2019)

    Google Scholar 

  47. Zhu, J., Park, T., Isola, P., Efros, A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017). https://doi.org/10.1109/ICCV.2017.244

Download references

Author information

Authors and Affiliations

  1. IDCOM, School of Engineering, University of Edinburgh, Edinburgh, UK

    Jingshuai Liu & Mehrdad Yaghoobi

  2. Department of Electrical and Electronic Engineering, Imperial College London, London, UK

    Chen Qin

Authors
  1. Jingshuai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehrdad Yaghoobi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jingshuai Liu .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (zip 8951 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, J., Qin, C., Yaghoobi, M. (2023). Coil-Agnostic Attention-Based Network for Parallel MRI Reconstruction. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13846. Springer, Cham. https://doi.org/10.1007/978-3-031-26351-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26351-4_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26350-7

  • Online ISBN: 978-3-031-26351-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics