Abstract
Image registration of structural and microstructural data allows accurate alignment of anatomical and diffusion channels. However, existing techniques employ simple fusion-based approaches, which use a global weight for each modality, or empirically-driven approaches, which rely on pre-calculated local certainty maps. Here, we present a novel attention-based deep learning deformable image registration solution for aligning multi-channel neonatal MRI data. We learn optimal attention maps to weigh each modality-specific velocity field in a spatially varying fashion, thus allowing for local fusion of structural and microstructural images. We evaluate our proposed method on registrations of 30 multi-channel neonatal MRI to a standard structural and microstructural atlas, and compare it against models trained without the use of attention maps on either single or both modalities. We show that by combining the two channels through attention-driven image registration, we take full advantage of the two complementary modalities, and achieve the best overall alignment of both structural and microstructural data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Avants, B., Duda, J.T., Zhang, H., Gee, J.C.: Multivariate normalization with symmetric diffeomorphisms for multivariate studies. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007. LNCS, vol. 4791, pp. 359–366. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75757-3_44
Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)
Christiaens, D., et al.: Scattered slice SHARD reconstruction for motion correction in multi-shell diffusion MRI. Neuroimage 225, 117437 (2021)
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 729–738. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_82
Edwards, A.D., et al.: The developing human connectome project neonatal data release. Front. Neurosci. 16, 886772 (2022)
Forsberg, D., Rathi, Y., Bouix, S., Wassermann, D., Knutsson, H., Westin, C.-F.: Improving registration using multi-channel diffeomorphic demons combined with certainty maps. In: Liu, T., Shen, D., Ibanez, L., Tao, X. (eds.) MBIA 2011. LNCS, vol. 7012, pp. 19–26. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24446-9_3
Grigorescu, I., et al.: Diffusion tensor driven image registration: a deep learning approach. In: Špiclin, Ž, McClelland, J., Kybic, J., Goksel, O. (eds.) WBIR 2020. LNCS, vol. 12120, pp. 131–140. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50120-4_13
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Irfanoglu, M.O., et al.: DR-TAMAS: diffeomorphic registration for tensor accurate alignment of anatomical structures. Neuroimage 132, 439–454 (2016)
Kingma, D.P., Rezende, D.J., Mohamed, S., Welling, M.: Semi-supervised learning with deep generative models (2014). arXiv:1406.5298
Krebs, J., Mansi, T., Mailhé, B., Ayache, N., Delingette, H.: Unsupervised probabilistic deformation modeling for robust diffeomorphic registration. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 101–109. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_12
Lowekamp, B., Chen, D., Ibanez, L., Blezek, D.: The design of SimpleiTK. Front. Neuroinform. 7, 45 (2013)
Makropoulos, A., et al.: Automatic whole brain MRI segmentation of the developing neonatal brain. IEEE Trans. Med. Imaging 33(9), 1818–1831 (2014)
Pérez-García, F., Sparks, R., Ourselin, S.: TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Comput. Methods Programs Biomed. 208, 106236 (2021)
Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999)
Smith, L.N.: Cyclical learning rates for training neural networks (2015)
Tournier, J.D., et al.: MRtrix3: a fast, flexible and open software framework for medical image processing and visualisation. Neuroimage 202, 116137 (2019)
Uus, A., et al.: Multi-channel 4D parametrized atlas of macro- and microstructural neonatal brain development. Front. Neurosci. 15, 721 (2021)
Uus, A., et al.: Multi-channel registration for diffusion MRI: longitudinal analysis for the neonatal brain. In: Špiclin, Ž, McClelland, J., Kybic, J., Goksel, O. (eds.) WBIR 2020. LNCS, vol. 12120, pp. 111–121. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50120-4_11
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1
Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolutional network (2015). arXiv:1505.00853
Zhang, H., Yushkevich, P.A., Alexander, D.C., Gee, J.C.: Deformable registration of diffusion tensor MR images with explicit orientation optimization. Med. Image Anal. 10(5), 764–785 (2006). The Eighth International Conference on Medical Imaging and Computer Assisted Intervention - MICCAI 2005
Acknowledgements
This work was supported by the Academy of Medical Sciences Springboard Award [SBF004\(\backslash \)1040], Medical Research Council (Grant no. [MR/K006355/1]), European Research Council under the European Union’s Seventh Framework Programme [FP7/20072013]/ERC grant agreement no. 319456 dHCP project, the EPSRC Research Council as part of the EPSRC DTP (grant Ref: [EP/R513064/1]), the Wellcome/EPSRC Centre for Medical Engineering at King’s College London [WT 203148/Z/16/Z], the NIHR Clinical Research Facility (CRF) at Guy’s and St Thomas’, and by the National Institute for Health Research Biomedical Research Centre based at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Grigorescu, I. et al. (2022). Attention-Driven Multi-channel Deformable Registration of Structural and Microstructural Neonatal Data. In: Licandro, R., Melbourne, A., Abaci Turk, E., Macgowan, C., Hutter, J. (eds) Perinatal, Preterm and Paediatric Image Analysis. PIPPI 2022. Lecture Notes in Computer Science, vol 13575. Springer, Cham. https://doi.org/10.1007/978-3-031-17117-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-17117-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17116-1
Online ISBN: 978-3-031-17117-8
eBook Packages: Computer ScienceComputer Science (R0)
