Abstract
Medical diagnosis benefits from multimodal Magnetic Resonance Imaging (MRI). However, multimodal MRI has an inherently slow acquisition process. For acceleration, recent studies explored using a fully-sampled side modality (fSM) as a guidance to reconstruct the fully-sampled query modalities (fQMs) from their undersampled k-space data via convolutional neural networks. However, even aided by fSM, the reconstruction of fQMs from highly undersampled QM data (uQM) is still suffering from aliasing artifacts. To enhance reconstruction quality, we suggest to fully use both uQM and fSM via a deep cascading network, which adopts an iterative Reconstruction-And-Refinement (iRAR) structure. The main limitation of the iRAR structure is that its intermediate reconstruction operators impede the feature flow across subnets and thus leads to short-term memory. We therefore propose two typical Peer-layer-wise Dense Connections (PDC), namely, inner PDC (iPDC) and end PDC (ePDC), to achieve long-term memory. Extensive experiments on different query modalities under different acceleration rates demonstrate that the deep cascading network equipped with iPDC and ePDC consistently outperforms the state-of-the-art methods and can preserve anatomical structure faithfully up to 12-fold acceleration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Commowick, O., Cervenansky, F., Ameli, R.: MSSEG challenge proceedings: multiple sclerosis lesions segmentation challenge using a data management and processing infrastructure. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2016)
Dar, S.U., Yurt, M., Shahdloo, M., Ildız, M.E., Tınaz, B., Çukur, T.: Prior-guided image reconstruction for accelerated multi-contrast MRI via generative adversarial networks. IEEE J. Select. Topics Signal Process. 14(6), 1072–1087 (2020)
Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp. 1486–1494 (2015)
Ehrhardt, M.J.: Multi-modality imaging with structure-promoting regularisers. arXiv:2007.11689 (2020)
Ehrhardt, M.J., Betcke, M.M.: Multicontrast MRI reconstruction with structure-guided total variation. SIAM J. Imaging Sci. 9(3), 1084–1106 (2016)
Eo, T., Jun, Y., Kim, T., Jang, J., Lee, H.J., Hwang, D.: KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn. Reson. Med. 80(5), 2188–2201 (2018)
Han, Y., Sunwoo, L., Ye, J.C.: k-space deep learning for accelerated MRI. IEEE Trans. Med. Imaging 39(2), 377–386 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. Las Vegas, USA (2016)
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1637–1645 (2016)
Kim, K.H., Do, W.J., Park, S.H.: Improving resolution of MR images with an adversarial network incorporating images with different contrast. Med. Phys. 45(7), 3120–3131 (2018)
Li, W., Feng, X., An, H., Ng, X.Y., Zhang, Y.J.: MRI reconstruction with interpretable pixel-wise operations using reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 792–799 (2020)
Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V., Rueckert, D.: Convolutional recurrent neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 38(1), 280–290 (2019)
Quan, T.M., Nguyen-Duc, T., Jeong, W.K.: Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans. Med. Imaging 37(6), 1488–1497 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2018)
Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In: IEEE International Conference on Computer Vision, pp. 4539–4547 (2017)
Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: The IEEE International Conference on Computer Vision (2017)
Wang, L., Huang, Z., Gong, Y., Pan, C.: Ensemble based deep networks for image super-resolution. Pattern Recogn. 68, 191–198 (2017)
Weizman, L., Eldar, Y.C., Ben Bashat, D.: Reference-based MRI. Med. Phys. 43(10), 5357–5369 (2016)
Xiang, L., et al.: Ultra-fast T2-weighted MR reconstruction using complementary T1-weighted information. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 215–223 (2018)
Xiang, L., et al.: Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Trans. Biomed. Eng. 66(7), 2105–2114 (2019)
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image restoration. IEEE Trans. Pattern Anal. Mach. Intell. p. online (2020)
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2472–2481 (2018)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, XX., Chen, Z., Lou, XJ., Yang, J., Chen, Y., Shen, D. (2021). Multimodal MRI Acceleration via Deep Cascading Networks with Peer-Layer-Wise Dense Connections. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12906. Springer, Cham. https://doi.org/10.1007/978-3-030-87231-1_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-87231-1_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87230-4
Online ISBN: 978-3-030-87231-1
eBook Packages: Computer ScienceComputer Science (R0)
