Skip to main content
Springer Nature Link
Log in

Multiscale U-net-based accelerated magnetic resonance imaging reconstruction

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Magnetic resonance imaging (MRI) is one of the prominent imaging modalities in the clinical settings. However, the acquisition time of MRI is long because of sequential acquisition of phase encoded k-space data. To accelerate the MRI acquisition process, model-based compressive sensing methods attempt to reconstruct fully sampled MR image from partially sampled k-space with the help of suitable priors. End to end trainable deep learning-based compressive sensing methods map the partially sampled MR image to fully sampled MR image with the parameters of mapping learned during training. In this work, we propose an end to end trainable U-net-based deep learning method to accelerate MRI acquisition process. To enable more efficient feature extraction, a multi-kernel convolutional feature fusion mechanism is introduced in each encoder and decoder stage of U-net. Multi-kernel convolutional paths extract features at different receptive fields. Moreover, to efficiently use decoder features at different scales in reconstructing the image, a feature fusion mechanism is used which fuse the decoder features at different scales. The performance of the proposed method is compared against several model-based and deep learning-based reconstruction methods. The reconstruction results demonstrate that the proposed method performs better compared to other methods in qualitative and quantitative terms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. Off. J. Int. Soc. Magn. Reson. Med. 58(6), 1182–1195 (2007)

    Article  Google Scholar 

  2. Guerquin-Kern, M., Van De Ville, D., Vonesch, C., Baritaux, J.C., Pruessmann, K.P., Unser, M.: Wavelet-regularized reconstruction for rapid MRI. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 193–196. IEEE (2009)

  3. Qu, X., Guo, D., Ning, B., Hou, Y., Lin, Y., Cai, S., Chen, Z.: Undersampled MRI reconstruction with patch-based directional wavelets. Magn. Reson. Imaging 30(7), 964–977 (2012)

    Article  Google Scholar 

  4. Chen, C., Huang, J.: Exploiting the wavelet structure in compressed sensing MRI. Magn. Reson. Imaging 32(10), 1377–1389 (2014)

    Article  Google Scholar 

  5. Ravishankar, S., Bresler, Y.: MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Med. Imaging 30(5), 1028–1041 (2010)

    Article  Google Scholar 

  6. Eksioglu, E.M.: Decoupled algorithm for MRI reconstruction using nonlocal block matching model: BM3D-MRI. J. Math. Imaging Vis. 56(3), 430–440 (2016)

    Article  MathSciNet  Google Scholar 

  7. Zhan, Z., Cai, J.F., Guo, D., Liu, Y., Chen, Z., Qu, X.: Fast multiclass dictionaries learning with geometrical directions in MRI reconstruction. IEEE Trans. Biomed. Eng. 63(9), 1850–1861 (2015)

    Article  Google Scholar 

  8. Wang, S., Su, Z., Ying, L., Peng, X., Zhu, S., Liang, F., Feng, D., Liang, D.: Accelerating magnetic resonance imaging via deep learning. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 514–517. IEEE (2016)

  9. Lee, D., Yoo, J., Ye, J.C.: Deep residual learning for compressed sensing MRI. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 15–18. IEEE (2017)

  10. Ding, P.L.K., Li, Z., Zhou, Y., Li, B.: Deep residual dense U-net for resolution enhancement in accelerated MRI acquisition. In: Medical Imaging 2019: Image Processing, vol. 10949, p. 109490F. International Society for Optics and Photonics (2019)

  11. Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2017)

    Article  Google Scholar 

  12. Sun, L., Fan, Z., Ding, X., Huang, Y., Paisley, J.: Joint CS-MRI reconstruction and segmentation with a unified deep network. In: International Conference on Information Processing in Medical Imaging, pp. 492–504. Springer (2019)

  13. Huang, Q., Yang, D., Wu, P., Qu, H., Yi, J., Metaxas, D.: MRI reconstruction via cascaded channel-wise attention network. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1622–1626. IEEE (2019)

  14. Xiang, L., Chen, Y., Chang, W., Zhan, Y., Lin, W., Wang, Q., Shen, D.: Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Trans. Biomed. Eng. 66(7), 2105–2114 (2018)

    Article  Google Scholar 

  15. Yang, G., Yu, S., Dong, H., Slabaugh, G., Dragotti, P.L., Ye, X., Liu, F., Arridge, S., Keegan, J., Guo, Y., et al.: Dagan: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans. Med. Imaging 37(6), 1310–1321 (2017)

    Article  Google Scholar 

  16. Quan, T.M., Nguyen-Duc, T., Jeong, W.K.: Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans. Med. Imaging 37(6), 1488–1497 (2018)

    Article  Google Scholar 

  17. Wang, G., Gong, E., Banerjee, S., Pauly, J., Zaharchuk, G.: Accelerated MRI reconstruction with dual-domain generative adversarial network. In: International Workshop on Machine Learning for Medical Image Reconstruction, pp. 47–57. Springer (2019)

  18. Seitzer, M., Yang, G., Schlemper, J., Oktay, O., Würfl, T., Christlein, V., Wong, T., Mohiaddin, R., Firmin, D., Keegan, J., et al.: Adversarial and perceptual refinement for compressed sensing MRI reconstruction. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 232–240. Springer (2018)

  19. Mardani, M., Gong, E., Cheng, J.Y., Vasanawala, S., Zaharchuk, G., Alley, M., Thakur, N., Han, S., Dally, W., Pauly, J.M., et al.: Deep generative adversarial networks for compressed sensing automates MRI (2017). arXiv preprint arXiv:1706.00051

  20. Dar, S.U., Yurt, M., Shahdloo, M., Ildız, M.E., Tınaz, B., Çukur, T.: Prior-guided image reconstruction for accelerated multi-contrast MRI via generative adversarial networks. IEEE J. Sel. Top. Signal Process. 14(6), 1072–1087 (2020)

    Article  Google Scholar 

  21. Yang, Y., Sun, J., Li, H., Xu, Z.: Deep ADMM-Net for compressive sensing MRI. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 10–18 (2016)

  22. Hammernik, K., Klatzer, T., Kobler, E., Recht, M.P., Sodickson, D.K., Pock, T., Knoll, F.: Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79(6), 3055–3071 (2018)

    Article  Google Scholar 

  23. Aggarwal, H.K., Mani, M.P., Jacob, M.: Modl: model-based deep learning architecture for inverse problems. IEEE Trans. Med. Imaging 38(2), 394–405 (2018)

    Article  Google Scholar 

  24. Akçakaya, M., Moeller, S., Weingärtner, S., Uğurbil, K.: Scan-specific robust artificial-neural-networks for k-space interpolation (raki) reconstruction: database-free deep learning for fast imaging. Magn. Reson. Med. 81(1), 439–453 (2019)

    Article  Google Scholar 

  25. Eo, T., Jun, Y., Kim, T., Jang, J., Lee, H.J., Hwang, D.: Kiki-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn. Reson. Med. 80(5), 2188–2201 (2018)

    Article  Google Scholar 

  26. Wang, Z., Jiang, H., Du, H., Xu, J., Qiu, B.: Ikwi-net: a cross-domain convolutional neural network for undersampled magnetic resonance image reconstruction. Magn. Reson. Med. 73, 1–10 (2020)

    Google Scholar 

  27. Dar, S.U.H., Özbey, M., Çatlı, A.B., Çukur, T.: A transfer-learning approach for accelerated MRI using deep neural networks. Magn. Reson. Med. 84(2), 663–685 (2020)

    Article  Google Scholar 

  28. Tezcan, K.C., Baumgartner, C.F., Luechinger, R., Pruessmann, K.P., Konukoglu, E.: MR image reconstruction using deep density priors. IEEE Trans. Med. Imaging 38(7), 1633–1642 (2018)

    Article  Google Scholar 

  29. Korkmaz, Y., Dar, S.U., Yurt, M., Özbey, M., Çukur, T.: Unsupervised MRI reconstruction via zero-shot learned adversarial transformers (2021). arXiv preprint arXiv:2105.08059

  30. Liu, C., Wu, X., Yu, X., Tang, Y., Zhang, J., Zhou, J.: Fusing multi-scale information in convolution network for MR image super-resolution reconstruction. Biomed. Eng. Online 17(1), 1–23 (2018)

    Article  Google Scholar 

  31. Xue, S., Qiu, W., Liu, F., Jin, X.: Wavelet-based residual attention network for image super-resolution. Neurocomputing 382, 116–126 (2020)

    Article  Google Scholar 

  32. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  33. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

  34. Liu, S., Huang, D., et al.: Receptive field block net for accurate and fast object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 385–400 (2018)

  35. Li, J., Fang, F., Mei, K., Zhang, G.: Multi-scale residual network for image super-resolution. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 517–532 (2018)

  36. Cai, Y., Wang, Y.: MA-Unet: An improved version of Unet based on multi-scale and attention mechanism for medical image segmentation (2020). arXiv preprint arXiv:2012.10952

  37. IXI dataset. https://brain-development.org/ixi-dataset/

  38. Souza, R., Lucena, O., Garrafa, J., Gobbi, D., Saluzzi, M., Appenzeller, S., Rittner, L., Frayne, R., Lotufo, R.: An open, multi-vendor, multi-field-strength brain MR dataset and analysis of publicly available skull stripping methods agreement. NeuroImage 170, 482–494 (2018)

    Article  Google Scholar 

  39. Lan, R., Sun, L., Liu, Z., Lu, H., Pang, C., Luo, X.: Madnet: a fast and lightweight network for single-image super resolution. IEEE Trans. Cybern. 51(3), 1443–1453 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

This publication is an outcome of R&D work undertaken in the project under Visvesvaraya PhD Scheme of Ministry of Electronics and Information Technology, Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia).

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Visvesvaraya National Institute of Technology, Nagpur, 440010, India

    Nikhil Dhengre & Saugata Sinha

Authors
  1. Nikhil Dhengre
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Saugata Sinha
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Nikhil Dhengre.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dhengre, N., Sinha, S. Multiscale U-net-based accelerated magnetic resonance imaging reconstruction. SIViP 16, 881–888 (2022). https://doi.org/10.1007/s11760-021-02030-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-02030-0

Keywords