Skip to main content

Advertisement

A multi-domain feature alignment and hierarchical cross feature enhancement network for under-sampled magnetic image reconstruction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Magnetic resonance imaging (MRI) is widely used in clinical diagnosis due to its high resolution and non-invasive scanning capabilities. However, long scanning times limit its development. To reduce acquisition time and obtain high-quality reconstructed images, a novel multi-domain MRI reconstruction network that fully utilizes the image domain, k-space, and wavelet domain is proposed. This network includes a parallel convolutional neural network (CNN) with k-space and wavelet domain branches, as well as a U-shaped image domain network. Following the parallel dual-domain CNN, a dual-domain feature alignment module aligns features from the k-space and wavelet domains into a unified representation space, mitigating artifact impacts. This design enhances the model’s understanding of multi-domain signals and improves generalization. Additionally, in the image domain, a hierarchical cross-feature enhancement module, based on Nested UNet, incorporates two cross-attention modules into different hierarchical skip connections of the Nested U-Net to reduce information propagation loss and enhance feature representation. Deep supervision within the image domain network further boosts the network’s performance and robustness. Extensive experiments on two public MRI datasets, FastMRI and CC359, as well as the private clinical dataset, validate the proposed method. Compared to several state-of-the-art deep learning methods, our approach demonstrates good reconstruction performance in both numerical assessments and visual effects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability and access

The data and code of the study are available from the corresponding author upon reasonable request.

References

  1. Chatterjee S, Breitkopf M, Sarasaen C, Yassin H, Rose G, Nürnberger A, Speck O (2022) Reconresnet: regularised residual learning for mr image reconstruction of undersampled cartesian and radial data. Comput Biol Med 143:105321. https://doi.org/10.1016/j.compbiomed.2022.105321

    Article  Google Scholar 

  2. Patel MR, Klufas RA (1999) Gradient-and spin-echo mr imaging of the brain. Am J Neuroradiol 20(7):1381–1383

    MATH  Google Scholar 

  3. Deshmane A, Gulani V, Griswold MA, Seiberlich N (2012) Parallel mr imaging. J Magn Reson Imaging 36(1):55–72. https://doi.org/10.1002/jmri.23639

    Article  Google Scholar 

  4. Lustig M, Donoho D, Pauly JM (2007) Sparse mri: the application of compressed sensing for rapid mr imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 58(6):1182–1195. https://doi.org/10.1002/mrm.21391

    Article  MATH  Google Scholar 

  5. Hu Y, Li P, Chen H, Zou L, Wang H (2021) High-quality mr fingerprinting reconstruction using structured low-rank matrix completion and subspace projection. IEEE Trans Med Imaging 41(5):1150–1164. https://doi.org/10.1109/TMI.2021.3133329

    Article  MATH  Google Scholar 

  6. Eo T, Shin H, Jun Y, Kim T, Hwang D (2020) Accelerating cartesian mri by domain-transform manifold learning in phase-encoding direction. Med Image Anal 63:101689. https://doi.org/10.1016/j.media.2020.101689

    Article  Google Scholar 

  7. Qu X, Guo D, Ning B, Hou Y, Lin Y, Cai S, Chen Z (2012) Undersampled mri reconstruction with patch-based directional wavelets. Magn Reson Imaging 30(7):964–977. https://doi.org/10.1016/j.mri.2012.02.019

    Article  MATH  Google Scholar 

  8. Han X, Liu Y, Liu G, Lin Y, Liu Q (2023) Loanet: a lightweight network using object attention for extracting buildings and roads from uav aerial remote sensing images. PeerJ Comp Sci 9:1467. https://doi.org/10.7717/peerj-cs.1467

    Article  MATH  Google Scholar 

  9. Liu Y, Han X, Liang T, Dong B, Yuan J, Hu M, Liu Q, Chen J, Li Q, Zhang Y (2023) Edmae: an efficient decoupled masked autoencoder for standard view identification in pediatric echocardiography. Biomed Signal Process Control 86:105280. https://doi.org/10.1016/j.bspc.2023.105280

    Article  Google Scholar 

  10. Liu Y, Huang Q, Han X, Liang T, Zhang Z, Lu X, Dong B, Yuan J, Wang Y, Hu M et al (2024) Atrial septal defect detection in children based on ultrasound video using multiple instances learning. J Imaging Inform Med, 1–11. https://doi.org/10.1007/s10278-024-00987-1

  11. Theerthagiri P, Ruby AU (2023) Seasonal learning based arima algorithm for prediction of brent oil price trends. Multimed Tools Appl 82(16):24485–24504. https://doi.org/10.1007/s11042-023-14819-x

    Article  MATH  Google Scholar 

  12. Wang S, Su Z, Ying L, Peng X, Zhu S, Liang F, Feng D, Liang D (2016) Accelerating magnetic resonance imaging via deep learning. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 514–517. https://doi.org/10.1109/ISBI.2016.7493320

  13. Eo T, Jun Y, Kim T, Jang J, Lee H-J, Hwang D (2018) Kiki-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn Reson Med 80(5):2188–2201. https://doi.org/10.1002/mrm.27201

    Article  MATH  Google Scholar 

  14. Souza R, Frayne R (2019) A hybrid frequency-domain/image-domain deep network for magnetic resonance image reconstruction. In: 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, pp 257–264. https://doi.org/10.1109/SIBGRAPI.2019.00042

  15. Souza R, Lebel RM, Frayne R (2019) A hybrid, dual domain, cascade of convolutional neural networks for magnetic resonance image reconstruction. In: International conference on medical imaging with deep learning. PMLR, pp 437–446

  16. Sun L, Wu Y, Shu B, Ding X, Cai C, Huang Y, Paisley J (2020) A dual-domain deep lattice network for rapid mri reconstruction. Neurocomputing 397:94–107. https://doi.org/10.1016/j.neucom.2020.01.063

    Article  MATH  Google Scholar 

  17. Ran M, Xia W, Huang Y, Lu Z, Bao P, Liu Y, Sun H, Zhou J, Zhang Y (2020) Md-recon-net: a parallel dual-domain convolutional neural network for compressed sensing mri. IEEE Trans Radiat Plasma Med Sci 5(1):120–135. https://doi.org/10.1109/TRPMS.2020.2991877

    Article  Google Scholar 

  18. Wang S, Ke Z, Cheng H, Jia S, Ying L, Zheng H, Liang D (2022) Dimension: dynamic mr imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training. NMR Biomed 35(4):4131. https://doi.org/10.1002/nbm.4131

    Article  MATH  Google Scholar 

  19. Liu Y, Pang Y, Liu X, Liu Y, Nie J (2023) Diik-net: a full-resolution cross-domain deep interaction convolutional neural network for mr image reconstruction. Neurocomputing 517:213–222. https://doi.org/10.1016/j.neucom.2022.09.048

    Article  MATH  Google Scholar 

  20. Wang Z, Jiang H, Du H, Xu J, Qiu B (2020) Ikwi-net: a cross-domain convolutional neural network for undersampled magnetic resonance image reconstruction. Magn Reson Imaging 73:1–10. https://doi.org/10.1016/j.mri.2020.06.015

    Article  MATH  Google Scholar 

  21. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867. https://doi.org/10.1109/TMI.2019.2959609

    Article  Google Scholar 

  22. Schlemper J, Caballero J, Hajnal JV, Price A, Rueckert D (2017) A deep cascade of convolutional neural networks for mr image reconstruction. In: Information processing in medical imaging: 25th International Conference, IPMI 2017, Boone, NC, USA, June 25-30, 2017, Proceedings 25. Springer, pp 647–658. https://doi.org/10.1007/978-3-319-59050-9_51

  23. Qin C, Schlemper J, Caballero J, Price AN, Hajnal JV, Rueckert D (2018) Convolutional recurrent neural networks for dynamic mr image reconstruction. IEEE Trans Med Imaging 38(1):280–290. https://doi.org/10.1109/TMI.2018.2863670

    Article  Google Scholar 

  24. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

  25. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27:2672–2680. https://doi.org/10.48550/arXiv.1406.2661

    Article  MATH  Google Scholar 

  26. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  27. Aghabiglou A, Eksioglu EM (2021) Mr image reconstruction using densely connected residual convolutional networks. Comput Biol Med 139:105010. https://doi.org/10.1016/j.compbiomed.2021.105010

    Article  MATH  Google Scholar 

  28. Yang G, Yu S, Dong H, Slabaugh G, Dragotti PL, Ye X, Liu F, Arridge S, Keegan J, Guo Y et al (2017) Dagan: deep de-aliasing generative adversarial networks for fast compressed sensing mri reconstruction. IEEE Trans Med Imaging 37(6):1310–1321. https://doi.org/10.1109/TMI.2017.2785879

    Article  Google Scholar 

  29. Quan TM, Nguyen-Duc T, Jeong W-K (2018) Compressed sensing mri reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging 37(6):1488–1497. https://doi.org/10.1109/TMI.2018.2820120

    Article  MATH  Google Scholar 

  30. Zhang T, Wan P, Zhang D (2019) Segan: structure-enhanced generative adversarial network for compressed sensing mri reconstruction. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 1012–1019. https://doi.org/10.1609/aaai.v33i01.33011012

  31. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30. https://doi.org/10.48550/arXiv.1706.03762

  32. Huang J, Fang Y, Wu Y, Wu H, Gao Z, Li Y, Del Ser J, Xia J, Yang G (2022) Swin transformer for fast mri. Neurocomputing 493:281–304. https://doi.org/10.1016/j.neucom.2022.04.051

    Article  MATH  Google Scholar 

  33. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022. https://doi.org/10.1109/ICCV48922.2021.00986

  34. Huang J, Xing X, Gao Z, Yang G (2022) Swin deformable attention u-net transformer (sdaut) for explainable fast mri. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 538–548. https://doi.org/10.1007/978-3-031-16446-0_51

  35. Huang J, Wu Y, Wu H, Yang G (2022) Fast mri reconstruction: how powerful transformers are? In: 2022 44th Annual international conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, pp 2066–2070. https://doi.org/10.1109/EMBC48229.2022.9871475

  36. Fabian Z, Tinaz B, Soltanolkotabi M (2022) Humus-net: hybrid unrolled multi-scale network architecture for accelerated mri reconstruction. Adv Neural Inf Process Syst 35:25306–25319. https://doi.org/10.48550/arXiv.2203.08213

    Article  MATH  Google Scholar 

  37. Han X, Chen Y, Liu Q, Liu Y, Chen K, Lin Y, Zhang W (2024) Reconstruction of cardiac cine mri using motion-guided deformable alignment and multi-resolution fusion. Int J Imaging Syst Technol 34(4):23131. https://doi.org/10.1002/ima.23131

    Article  MATH  Google Scholar 

  38. Guo P, Mei Y, Zhou J, Jiang S, Patel VM (2023) Reconformer: accelerated mri reconstruction using recurrent transformer. IEEE Trans Med Imaging. https://doi.org/10.1109/TMI.2023.3314747

    Article  Google Scholar 

  39. Zhao X, Yang T, Li B, Zhang X (2023) Swingan: a dual-domain swin transformer-based generative adversarial network for mri reconstruction. Comput Biol Med 153:106513. https://doi.org/10.1016/j.compbiomed.2022.106513

    Article  Google Scholar 

  40. Tong C, Pang Y, Wang Y (2022) Hiwdnet: a hybrid image-wavelet domain network for fast magnetic resonance image reconstruction. Comput Biol Med 151:105947. https://doi.org/10.1016/j.compbiomed.2022.105947

    Article  Google Scholar 

  41. Aghabiglou A, Eksioglu EM (2022) Densely connected wavelet-based autoencoder for mr image reconstruction. In: 2022 45th International conference on Telecommunications and Signal Processing (TSP). IEEE, pp 212–215. https://doi.org/10.1109/TSP55681.2022.9851354

  42. Yu S, Park B, Jeong J (2019) Deep iterative down-up cnn for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 0–0. https://doi.org/10.1109/CVPRW.2019.00262

  43. Tian Y, Zhang Y, Fu Y, Xu C (2020) Tdan: temporally-deformable alignment network for video super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3360–3369. https://doi.org/10.1109/CVPR42600.2020.00342

  44. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 764–773. https://doi.org/10.1109/ICCV.2017.89

  45. Zbontar J, Knoll F, Sriram A, Murrell T, Huang Z, Muckley MJ, Defazio A, Stern R, Johnson P, Bruno M et al (2018) fastmri: an open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.08839. https://doi.org/10.48550/arXiv.1811.08839

  46. Souza R, Lucena O, Garrafa J, Gobbi D, Saluzzi M, Appenzeller S, Rittner L, Frayne R, Lotufo R (2018) An open, multi-vendor, multi-field-strength brain mr dataset and analysis of publicly available skull stripping methods agreement. Neuroimage 170:482–494. https://doi.org/10.1016/j.neuroimage.2017.08.021

  47. Ramzi Z, Ciuciu P, Starck J-L (2020) Xpdnet for mri reconstruction: an application to the 2020 fastmri challenge. arXiv preprint arXiv:2010.07290. https://doi.org/10.48550/arXiv.2010.07290

  48. Feng C-M, Yan Y, Fu H, Chen L, Xu Y (2021) Task transformer network for joint mri reconstruction and super-resolution. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VI 24. Springer, pp 307–317. https://doi.org/10.1007/978-3-030-87231-1_30

  49. Abbaszadeh Shahri A, Chunling S, Larsson S (2023) A hybrid ensemble-based automated deep learning approach to generate 3d geo-models and uncertainty analysis. Eng Comput, 1–16. https://doi.org/10.1007/s00366-023-01852-5

  50. Abbaszadeh Shahri A, Shan C, Larsson S (2022) A novel approach to uncertainty quantification in groundwater table modeling by automated predictive deep learning. Nat Resour Res 31(3):1351–1373. https://doi.org/10.1007/s11053-022-10051-w

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers and the associate editor for their constructive comments and suggestions that helped to improve both the technical content and the presentation quality of this paper. This work is supported by the National Natural Science Foundation of China under grant No.61801288.

Author information

Authors and Affiliations

Authors

Contributions

Qiaohong Liu is responsible for methodological research, paper writing, and funding acquisition. Xiaoxiang Han is responsible for methodological research, conducting experiments, and data analysis. Yang Chen oversees conceptual guidance, project organization, and paper review within the team.

Corresponding author

Correspondence to Yang Chen.

Ethics declarations

Ethical and informed consent for data used

This study was approved by the Institutional Review Board of Shanghai University of Medicine and Health Sciences with a patient exemption applied. All publicly available data has been subject to an exemption.

Competing Interests

The authors declare no potential conflicts of interest regarding any financial support, research, authorship, and publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Han, X. & Chen, Y. A multi-domain feature alignment and hierarchical cross feature enhancement network for under-sampled magnetic image reconstruction. Appl Intell 55, 45 (2025). https://doi.org/10.1007/s10489-024-06008-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06008-6

Keywords