Abstract
Exploring the relationship between the cognitive ability and infant cortical structural and functional development is critically important to advance our understanding of early brain development, which, however, is very challenging due to the complex and dynamic brain development in early postnatal stages. Conventional approaches typically use either the structural MRI or resting-state functional MRI and rely on the region-level features or inter-region connectivity features after cortical parcellation for predicting cognitive scores. However, these methods have two major issues: 1) spatial information loss, which discards the critical fine-grained spatial patterns containing rich information related to cognitive development; 2) modality information loss, which ignores the complementary information and the interaction between the structural and functional images. To address these issues, we unprecedentedly invent a novel framework, namely cortical surface-based multimodal learning framework (CSML), to leverage fine-grained multimodal features for cognition development prediction. First, we introduce the fine-grained surface-based data representation to capture spatially detailed structural and functional information. Then, a dual-branch network is proposed to extract the discriminative features for each modality respectively and further captures the modality-shared and complementary information with a disentanglement strategy. Finally, an age-guided cognition prediction module is developed based on the prior that the cognition develops along with age. We validate our method on an infant multimodal MRI dataset with 318 scans. Compared to state-of-the-art methods, our method consistently achieves superior performances, and for the first time suggests crucial regions and features for cognition development hidden in the fine-grained spatial details of cortical structure and function.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gao, W., et al.: Functional network development during the first year: relative sequence and socioeconomic correlations. Cereb. Cortex 25(9), 2919–2928 (2015)
Zhang, H., Shen, D., Lin, W.: Resting-state functional MRI studies on infant brains: a decade of gap-filling efforts. Neuroimage 185, 664–684 (2019)
Keunen, K., Counsell, S.J., Benders, M.J.: The emergence of functional architecture during early brain development. Neuroimage 160, 2–14 (2017)
Smyser, C.D., Snyder, A.Z., Neil, J.J.: Functional connectivity MRI in infants: exploration of the functional organization of the developing brain. Neuroimage 56(3), 1437–1452 (2011)
Cheng, J., et al.: Path signature neural network of cortical features for prediction of infant cognitive scores. IEEE Trans. Med. Imaging 41(7), 1665–1676 (2021)
Adeli, E., et al.: Multi-task prediction of infant cognitive scores from longitudinal incomplete neuroimaging data. Neuroimage 185, 783–792 (2019)
Zhang, C., et al.: Infant brain development prediction with latent partial multi-view representation learning. IEEE Trans. Med. Imaging 38(4), 909–918 (2018)
Hu, D., et al.: Existence of functional connectome fingerprint during infancy and its stability over months. J. Neurosci. 42(3), 377–389 (2022)
Zhao, F., et al.: Spherical deformable u-net: application to cortical surface parcellation and development prediction. IEEE Trans. Med. Imaging 40(4), 1217–1228 (2021)
Wu, Z., et al.: Construction of 4D infant cortical surface atlases with sharp folding patterns via spherical patch-based group-wise sparse representation. Hum. Brain Mapp. 40(13), 3860–3880 (2019)
Li, G., et al.: Construction of 4D high-definition cortical surface atlases of infants: Methods and applications. Med. Image Anal. 25(1), 22–36 (2015)
Li, G., et al.: Computational neuroanatomy of baby brains: a review. Neuroimage 185, 906–925 (2019)
Li, G., et al.: Measuring the dynamic longitudinal cortex development in infants by reconstruction of temporally consistent cortical surfaces. Neuroimage 90, 266–279 (2014)
Wang, L., Wu, Z., Chen, L., Sun, Y., Lin, W., Li, G.: iBEAT V2. 0: a multisite-applicable, deep learning-based pipeline for infant cerebral cortical surface reconstruction. Nat. Protoc. 18(5), 1488–1509 (2023)
Wang, F., et al.: Fine-grained functional parcellation maps of the infant cerebral cortex. eLife (2023)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Zhao, F., et al.: Harmonization of infant cortical thickness using surface-to-surface cycle-consistent adversarial networks. In: Shen, Dinggang, et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 475–483. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_52
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30, pp. 6000-6010 (2017)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representation (2021)
Howell, B.R., et al.: The UNC/UMN baby connectome project (BCP): an overview of the study design and protocol development. Neuroimage 185, 891–905 (2019)
Mullen, E.M.: Mullen scales of early learning. AGS Circle Pines, MN (1995)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Veličković, P., et al.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Shi, Y., et al.: Masked label prediction: Unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03509 (2020)
Fuster, J.M.: Frontal lobe and cognitive development. J. Neurocytol. 31(3), 373–385 (2002)
Kolk, S.M., Rakic, P.: Development of prefrontal cortex. Neuropsychopharmacology 47(1), 41–57 (2022)
Roelfsema, P.R., de Lange, F.P.: Early visual cortex as a multiscale cognitive blackboard. Ann. Rev. Vis. Sci. 2, 131–151 (2016)
Albers, A.M., et al.: Shared representations for working memory and mental imagery in early visual cortex. Curr. Biol. 23(15), 1427–1431 (2013)
Acknowledgements
The work of Gang Li was supported in part by NIH grants (MH116225, MH117943, MH127544, and MH123202). The work of Li Wang was supported by NIH grant (MH117943). This work also utilizes approaches developed by an NIH grant (1U01MH110274) and the efforts of the UNC/UMN Baby Connectome Project Consortium.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cheng, J. et al. (2023). Prediction of Infant Cognitive Development with Cortical Surface-Based Multimodal Learning. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14221. Springer, Cham. https://doi.org/10.1007/978-3-031-43895-0_58
Download citation
DOI: https://doi.org/10.1007/978-3-031-43895-0_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43894-3
Online ISBN: 978-3-031-43895-0
eBook Packages: Computer ScienceComputer Science (R0)