Skip to main content
Springer Nature Link
Log in

Fourier Transformer for Joint Super-Resolution and Reconstruction of MR Image

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14555))

Included in the following conference series:

  • 1002 Accesses

Abstract

Super resolution (SR) and fast magnetic resonance (MR) imaging play a paramount role in medical diagnosis community. Though significant progress has been achieved, existing methods are mostly trapped at the local feature dependencies. Transformer holds the potential to tackle this issue, yet it is notoriously known with heavy computational burden. As a remedy, we propose a Fourier Transformer Network (FTN), which leverages 2D Fast Fourier Transform (FFT) to harvest the global relationships. Specifically, multiple Fourier Transformer Blocks (FTBs) are stacked as the backbone, comprehensively extracting the sematic representations. Due to the appealing efficiency of FFT, the overall model complexity is linear to tokens, as opposed to the quadratic complexity of previous transformer-based models. Besides, an edge-enhancement branch is incorporated into FTB in a flexible manner, which further sharpens the contour information of the organs and tissues. Extensive experiments on IXI dataset validate the superiority of FTN over most state-of-the-art methods.

Supported by organization x.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)

  2. Carmi, E., Liu, S., Alon, N., Fiat, A., Fiat, D.: Resolution enhancement in MRI. Magn. Reson. Imag. 24(2), 133–154 (2006)

    Article  Google Scholar 

  3. Chen, Y., et al.: Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3435–3444 (2019)

    Google Scholar 

  4. Feng, C.-M., Fu, H., Yuan, S., Xu, Y.: Multi-contrast MRI super-resolution via a multi-stage integration network. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 140–149. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_14

    Chapter  Google Scholar 

  5. Feng, C.M., Yan, Y., Chen, G., Xu, Y., Hu, Y., Shao, L., Fu, H.: Multi-modal transformer for accelerated mr imaging. IEEE Transactions on Medical Imaging (2022)

    Google Scholar 

  6. Feng, C.-M., Yan, Y., Fu, H., Chen, L., Xu, Y.: Task transformer network for joint MRI reconstruction and super-resolution. In: Bruijne de, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 307–317. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_30

    Chapter  Google Scholar 

  7. Feng, C.M., Yan, Y., Yu, K., Xu, Y., Shao, L., Fu, H.: Exploring separable attention for multi-contrast mr image super-resolution. arXiv preprint arXiv:2109.01664 (2021)

  8. Feng, C.M., Yang, Z., Fu, H., Xu, Y., Yang, J., Shao, L.: Donet: dual-octave network for fast MR image reconstruction. IEEE Transactions on Neural Networks and Learning Systems (2021)

    Google Scholar 

  9. Hammernik, K., et al.: Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79(6), 3055–3071 (2018)

    Article  Google Scholar 

  10. Hu, X., Wang, H., Cai, Y., Zhao, X., Zhang, Y.: Pyramid orthogonal attention network based on dual self-similarity for accurate mr image super-resolution. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2021)

    Google Scholar 

  11. Huang, Q., Yang, D., Wu, P., Qu, H., Yi, J., Metaxas, D.: MRI reconstruction via cascaded channel-wise attention network. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1622–1626. IEEE (2019)

    Google Scholar 

  12. Jiang, M., Zhai, F., Kong, J.: A novel deep learning model ddu-net using edge features to enhance brain tumor segmentation on MR images. Artif. Intell. Med. 121, 102180 (2021)

    Article  Google Scholar 

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  14. Li, G., et al.: Transformer-empowered multi-scale contextual matching and aggregation for multi-contrast mri super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20636–20645 (2022)

    Google Scholar 

  15. Li, G., Lyu, J., Wang, C., Dou, Q., Qin, J.: Wavtrans: synergizing wavelet and cross-attention transformer for multi-contrast mri super-resolution. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 463–473. Springer (2022). https://doi.org/10.1007/978-3-031-16446-0_44

  16. Liu, Q., Yang, Q., Cheng, H., Wang, S., Zhang, M., Liang, D.: Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors. Magn. Reson. Med. 83(1), 322–336 (2020)

    Article  Google Scholar 

  17. Lyu, Q., et al.: Multi-contrast super-resolution MRI through a progressive network. IEEE Trans. Med. Imaging 39(9), 2738–2749 (2020)

    Article  Google Scholar 

  18. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  19. Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2017)

    Article  Google Scholar 

  20. Sun, J., Li, H., Xu, Z., et al.: Deep admm-net for compressive sensing MRI. In: Advances in Neural Information Processing Systems 29 (2016)

    Google Scholar 

  21. Wang, H., Hu, X., Zhao, X., Zhang, Y.: Wide weighted attention multi-scale network for accurate MR image super-resolution. IEEE Trans. Circuits Syst. Video Technol. 32(3), 962–975 (2021)

    Article  Google Scholar 

  22. Wang, S., et al.: Deepcomplexmri: exploiting deep residual network for fast parallel MR imaging with complex convolution. Magn. Reson. Imaging 68, 136–147 (2020)

    Article  Google Scholar 

  23. Wang, W., Shen, H., Chen, J., Xing, F.: Mhan: multi-stage hybrid attention network for MRI reconstruction and super-resolution. Computers in Biology and Medicine, p. 107181 (2023)

    Google Scholar 

  24. Zhang, M., Zhang, M., Zhang, F., Chaddad, A., Evans, A.: Robust brain MR image compressive sensing via re-weighted total variation and sparse regression. Magn. Reson. Imag. 85, 271–286 (2022)

    Article  Google Scholar 

  25. Zhao, C., et al.: A deep learning based anti-aliasing self super-resolution algorithm for MRI. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 100–108. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_12

    Chapter  Google Scholar 

  26. Zhao, X., Zhang, Y., Zhang, T., Zou, X.: Channel splitting network for single MR image super-resolution. IEEE Trans. Image Process. 28(11), 5649–5662 (2019)

    Article  MathSciNet  Google Scholar 

  27. Zhou, B., Zhou, S.K.: Dudornet: learning a dual-domain recurrent network for fast mri reconstruction with deep t1 prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4273–4282 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China

    Jiacheng Chen, Fei Wu, Wanliang Wang & Haoxin Sheng

Authors
  1. Jiacheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanliang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haoxin Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanliang Wang .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Stevan Rudinac

  2. Delft University of Technology, Delft, The Netherlands

    Alan Hanjalic

  3. Delft University of Technology, Delft, The Netherlands

    Cynthia Liem

  4. University of Amsterdam, Amsterdam, The Netherlands

    Marcel Worring

  5. Reykjavik University, Reykjavik, Iceland

    Björn Þór Jónsson

  6. Microsoft Research Lab – Asia, Beijing, China

    Bei Liu

  7. The University of Tokyo, Tokyo, Japan

    Yoko Yamakata

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, J., Wu, F., Wang, W., Sheng, H. (2024). Fourier Transformer for Joint Super-Resolution and Reconstruction of MR Image. In: Rudinac, S., et al. MultiMedia Modeling. MMM 2024. Lecture Notes in Computer Science, vol 14555. Springer, Cham. https://doi.org/10.1007/978-3-031-53308-2_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53308-2_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53307-5

  • Online ISBN: 978-3-031-53308-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics