Abstract
Magnetic resonance imaging (MRI) is widely used in clinical diagnosis due to its high resolution and non-invasive scanning capabilities. However, long scanning times limit its development. To reduce acquisition time and obtain high-quality reconstructed images, a novel multi-domain MRI reconstruction network that fully utilizes the image domain, k-space, and wavelet domain is proposed. This network includes a parallel convolutional neural network (CNN) with k-space and wavelet domain branches, as well as a U-shaped image domain network. Following the parallel dual-domain CNN, a dual-domain feature alignment module aligns features from the k-space and wavelet domains into a unified representation space, mitigating artifact impacts. This design enhances the model’s understanding of multi-domain signals and improves generalization. Additionally, in the image domain, a hierarchical cross-feature enhancement module, based on Nested UNet, incorporates two cross-attention modules into different hierarchical skip connections of the Nested U-Net to reduce information propagation loss and enhance feature representation. Deep supervision within the image domain network further boosts the network’s performance and robustness. Extensive experiments on two public MRI datasets, FastMRI and CC359, as well as the private clinical dataset, validate the proposed method. Compared to several state-of-the-art deep learning methods, our approach demonstrates good reconstruction performance in both numerical assessments and visual effects.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability and access
The data and code of the study are available from the corresponding author upon reasonable request.
References
Chatterjee S, Breitkopf M, Sarasaen C, Yassin H, Rose G, Nürnberger A, Speck O (2022) Reconresnet: regularised residual learning for mr image reconstruction of undersampled cartesian and radial data. Comput Biol Med 143:105321. https://doi.org/10.1016/j.compbiomed.2022.105321
Patel MR, Klufas RA (1999) Gradient-and spin-echo mr imaging of the brain. Am J Neuroradiol 20(7):1381–1383
Deshmane A, Gulani V, Griswold MA, Seiberlich N (2012) Parallel mr imaging. J Magn Reson Imaging 36(1):55–72. https://doi.org/10.1002/jmri.23639
Lustig M, Donoho D, Pauly JM (2007) Sparse mri: the application of compressed sensing for rapid mr imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 58(6):1182–1195. https://doi.org/10.1002/mrm.21391
Hu Y, Li P, Chen H, Zou L, Wang H (2021) High-quality mr fingerprinting reconstruction using structured low-rank matrix completion and subspace projection. IEEE Trans Med Imaging 41(5):1150–1164. https://doi.org/10.1109/TMI.2021.3133329
Eo T, Shin H, Jun Y, Kim T, Hwang D (2020) Accelerating cartesian mri by domain-transform manifold learning in phase-encoding direction. Med Image Anal 63:101689. https://doi.org/10.1016/j.media.2020.101689
Qu X, Guo D, Ning B, Hou Y, Lin Y, Cai S, Chen Z (2012) Undersampled mri reconstruction with patch-based directional wavelets. Magn Reson Imaging 30(7):964–977. https://doi.org/10.1016/j.mri.2012.02.019
Han X, Liu Y, Liu G, Lin Y, Liu Q (2023) Loanet: a lightweight network using object attention for extracting buildings and roads from uav aerial remote sensing images. PeerJ Comp Sci 9:1467. https://doi.org/10.7717/peerj-cs.1467
Liu Y, Han X, Liang T, Dong B, Yuan J, Hu M, Liu Q, Chen J, Li Q, Zhang Y (2023) Edmae: an efficient decoupled masked autoencoder for standard view identification in pediatric echocardiography. Biomed Signal Process Control 86:105280. https://doi.org/10.1016/j.bspc.2023.105280
Liu Y, Huang Q, Han X, Liang T, Zhang Z, Lu X, Dong B, Yuan J, Wang Y, Hu M et al (2024) Atrial septal defect detection in children based on ultrasound video using multiple instances learning. J Imaging Inform Med, 1–11. https://doi.org/10.1007/s10278-024-00987-1
Theerthagiri P, Ruby AU (2023) Seasonal learning based arima algorithm for prediction of brent oil price trends. Multimed Tools Appl 82(16):24485–24504. https://doi.org/10.1007/s11042-023-14819-x
Wang S, Su Z, Ying L, Peng X, Zhu S, Liang F, Feng D, Liang D (2016) Accelerating magnetic resonance imaging via deep learning. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 514–517. https://doi.org/10.1109/ISBI.2016.7493320
Eo T, Jun Y, Kim T, Jang J, Lee H-J, Hwang D (2018) Kiki-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn Reson Med 80(5):2188–2201. https://doi.org/10.1002/mrm.27201
Souza R, Frayne R (2019) A hybrid frequency-domain/image-domain deep network for magnetic resonance image reconstruction. In: 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, pp 257–264. https://doi.org/10.1109/SIBGRAPI.2019.00042
Souza R, Lebel RM, Frayne R (2019) A hybrid, dual domain, cascade of convolutional neural networks for magnetic resonance image reconstruction. In: International conference on medical imaging with deep learning. PMLR, pp 437–446
Sun L, Wu Y, Shu B, Ding X, Cai C, Huang Y, Paisley J (2020) A dual-domain deep lattice network for rapid mri reconstruction. Neurocomputing 397:94–107. https://doi.org/10.1016/j.neucom.2020.01.063
Ran M, Xia W, Huang Y, Lu Z, Bao P, Liu Y, Sun H, Zhou J, Zhang Y (2020) Md-recon-net: a parallel dual-domain convolutional neural network for compressed sensing mri. IEEE Trans Radiat Plasma Med Sci 5(1):120–135. https://doi.org/10.1109/TRPMS.2020.2991877
Wang S, Ke Z, Cheng H, Jia S, Ying L, Zheng H, Liang D (2022) Dimension: dynamic mr imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training. NMR Biomed 35(4):4131. https://doi.org/10.1002/nbm.4131
Liu Y, Pang Y, Liu X, Liu Y, Nie J (2023) Diik-net: a full-resolution cross-domain deep interaction convolutional neural network for mr image reconstruction. Neurocomputing 517:213–222. https://doi.org/10.1016/j.neucom.2022.09.048
Wang Z, Jiang H, Du H, Xu J, Qiu B (2020) Ikwi-net: a cross-domain convolutional neural network for undersampled magnetic resonance image reconstruction. Magn Reson Imaging 73:1–10. https://doi.org/10.1016/j.mri.2020.06.015
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867. https://doi.org/10.1109/TMI.2019.2959609
Schlemper J, Caballero J, Hajnal JV, Price A, Rueckert D (2017) A deep cascade of convolutional neural networks for mr image reconstruction. In: Information processing in medical imaging: 25th International Conference, IPMI 2017, Boone, NC, USA, June 25-30, 2017, Proceedings 25. Springer, pp 647–658. https://doi.org/10.1007/978-3-319-59050-9_51
Qin C, Schlemper J, Caballero J, Price AN, Hajnal JV, Rueckert D (2018) Convolutional recurrent neural networks for dynamic mr image reconstruction. IEEE Trans Med Imaging 38(1):280–290. https://doi.org/10.1109/TMI.2018.2863670
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27:2672–2680. https://doi.org/10.48550/arXiv.1406.2661
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
Aghabiglou A, Eksioglu EM (2021) Mr image reconstruction using densely connected residual convolutional networks. Comput Biol Med 139:105010. https://doi.org/10.1016/j.compbiomed.2021.105010
Yang G, Yu S, Dong H, Slabaugh G, Dragotti PL, Ye X, Liu F, Arridge S, Keegan J, Guo Y et al (2017) Dagan: deep de-aliasing generative adversarial networks for fast compressed sensing mri reconstruction. IEEE Trans Med Imaging 37(6):1310–1321. https://doi.org/10.1109/TMI.2017.2785879
Quan TM, Nguyen-Duc T, Jeong W-K (2018) Compressed sensing mri reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging 37(6):1488–1497. https://doi.org/10.1109/TMI.2018.2820120
Zhang T, Wan P, Zhang D (2019) Segan: structure-enhanced generative adversarial network for compressed sensing mri reconstruction. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 1012–1019. https://doi.org/10.1609/aaai.v33i01.33011012
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30. https://doi.org/10.48550/arXiv.1706.03762
Huang J, Fang Y, Wu Y, Wu H, Gao Z, Li Y, Del Ser J, Xia J, Yang G (2022) Swin transformer for fast mri. Neurocomputing 493:281–304. https://doi.org/10.1016/j.neucom.2022.04.051
Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022. https://doi.org/10.1109/ICCV48922.2021.00986
Huang J, Xing X, Gao Z, Yang G (2022) Swin deformable attention u-net transformer (sdaut) for explainable fast mri. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 538–548. https://doi.org/10.1007/978-3-031-16446-0_51
Huang J, Wu Y, Wu H, Yang G (2022) Fast mri reconstruction: how powerful transformers are? In: 2022 44th Annual international conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, pp 2066–2070. https://doi.org/10.1109/EMBC48229.2022.9871475
Fabian Z, Tinaz B, Soltanolkotabi M (2022) Humus-net: hybrid unrolled multi-scale network architecture for accelerated mri reconstruction. Adv Neural Inf Process Syst 35:25306–25319. https://doi.org/10.48550/arXiv.2203.08213
Han X, Chen Y, Liu Q, Liu Y, Chen K, Lin Y, Zhang W (2024) Reconstruction of cardiac cine mri using motion-guided deformable alignment and multi-resolution fusion. Int J Imaging Syst Technol 34(4):23131. https://doi.org/10.1002/ima.23131
Guo P, Mei Y, Zhou J, Jiang S, Patel VM (2023) Reconformer: accelerated mri reconstruction using recurrent transformer. IEEE Trans Med Imaging. https://doi.org/10.1109/TMI.2023.3314747
Zhao X, Yang T, Li B, Zhang X (2023) Swingan: a dual-domain swin transformer-based generative adversarial network for mri reconstruction. Comput Biol Med 153:106513. https://doi.org/10.1016/j.compbiomed.2022.106513
Tong C, Pang Y, Wang Y (2022) Hiwdnet: a hybrid image-wavelet domain network for fast magnetic resonance image reconstruction. Comput Biol Med 151:105947. https://doi.org/10.1016/j.compbiomed.2022.105947
Aghabiglou A, Eksioglu EM (2022) Densely connected wavelet-based autoencoder for mr image reconstruction. In: 2022 45th International conference on Telecommunications and Signal Processing (TSP). IEEE, pp 212–215. https://doi.org/10.1109/TSP55681.2022.9851354
Yu S, Park B, Jeong J (2019) Deep iterative down-up cnn for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 0–0. https://doi.org/10.1109/CVPRW.2019.00262
Tian Y, Zhang Y, Fu Y, Xu C (2020) Tdan: temporally-deformable alignment network for video super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3360–3369. https://doi.org/10.1109/CVPR42600.2020.00342
Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 764–773. https://doi.org/10.1109/ICCV.2017.89
Zbontar J, Knoll F, Sriram A, Murrell T, Huang Z, Muckley MJ, Defazio A, Stern R, Johnson P, Bruno M et al (2018) fastmri: an open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.08839. https://doi.org/10.48550/arXiv.1811.08839
Souza R, Lucena O, Garrafa J, Gobbi D, Saluzzi M, Appenzeller S, Rittner L, Frayne R, Lotufo R (2018) An open, multi-vendor, multi-field-strength brain mr dataset and analysis of publicly available skull stripping methods agreement. Neuroimage 170:482–494. https://doi.org/10.1016/j.neuroimage.2017.08.021
Ramzi Z, Ciuciu P, Starck J-L (2020) Xpdnet for mri reconstruction: an application to the 2020 fastmri challenge. arXiv preprint arXiv:2010.07290. https://doi.org/10.48550/arXiv.2010.07290
Feng C-M, Yan Y, Fu H, Chen L, Xu Y (2021) Task transformer network for joint mri reconstruction and super-resolution. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VI 24. Springer, pp 307–317. https://doi.org/10.1007/978-3-030-87231-1_30
Abbaszadeh Shahri A, Chunling S, Larsson S (2023) A hybrid ensemble-based automated deep learning approach to generate 3d geo-models and uncertainty analysis. Eng Comput, 1–16. https://doi.org/10.1007/s00366-023-01852-5
Abbaszadeh Shahri A, Shan C, Larsson S (2022) A novel approach to uncertainty quantification in groundwater table modeling by automated predictive deep learning. Nat Resour Res 31(3):1351–1373. https://doi.org/10.1007/s11053-022-10051-w
Acknowledgements
The authors would like to thank the anonymous reviewers and the associate editor for their constructive comments and suggestions that helped to improve both the technical content and the presentation quality of this paper. This work is supported by the National Natural Science Foundation of China under grant No.61801288.
Author information
Authors and Affiliations
Contributions
Qiaohong Liu is responsible for methodological research, paper writing, and funding acquisition. Xiaoxiang Han is responsible for methodological research, conducting experiments, and data analysis. Yang Chen oversees conceptual guidance, project organization, and paper review within the team.
Corresponding author
Ethics declarations
Ethical and informed consent for data used
This study was approved by the Institutional Review Board of Shanghai University of Medicine and Health Sciences with a patient exemption applied. All publicly available data has been subject to an exemption.
Competing Interests
The authors declare no potential conflicts of interest regarding any financial support, research, authorship, and publication of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, Q., Han, X. & Chen, Y. A multi-domain feature alignment and hierarchical cross feature enhancement network for under-sampled magnetic image reconstruction. Appl Intell 55, 45 (2025). https://doi.org/10.1007/s10489-024-06008-6
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06008-6