Abstract
The number of stroke patients is growing worldwide and half of them will suffer from cognitive impairment. Therefore, the prediction of Post-Stroke Cognitive Impairment (PSCI) becomes more and more important. However, the determinants and mechanisms of PSCI are still insufficiently understood, making this task challenging. In this paper, we propose a multi-modal graph fusion model to solve this task. First, dynamic graph neural representation is proposed to integrate multi-modal information, such as clinical data and image data, which separates them into node-level and global-level properties rather than processing them uniformly. Second, considering the variability of brain anatomy, a subject-specific undirected graph is constructed based on the connections among 131 brain anatomical regions segmented from image data, while first-order statistical features are extracted from each brain region and internal stroke lesions as node features. Meanwhile, a novel missing information compensation module is proposed to reduce the impact of missing or incomplete clinical data. In the dynamic graph neural representation, two kinds of attention mechanisms are used to encourage the model to automatically localize brain anatomical regions that are highly relevant to PSCI prediction. One is node attention established between global tabular neural representation and nodes, the other is multi-head graph self-attention which changes the static undirected graph into multiple dynamic directed graphs and optimizes the broadcasting process of the graph. The proposed method studies 418 stroke patients and achieves the best overall performance with a balanced accuracy score of 79.6% on PSCI prediction, outperforming the competing models. The code is publicly available at github.com/fightingkitty/MHGSA.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Avants, B.B., Tustison, N., Song, G., et al.: Advanced normalization tools (ANTS). Insight j 2(365), 1–35 (2009)
Ball, E.L., et al.: Predicting post-stroke cognitive impairment using acute CT neuroimaging: a systematic review and meta-analysis. Int. J. Stroke 17(6), 618–627 (2022)
Binzer, M., Hammernik, K., Rueckert, D., Zimmer, V.A.: Long-term cognitive outcome prediction in stroke patients using multi-task learning on imaging and tabular data. In: Rekik, I., Adeli, E., Park, S.H., Cintas, C. (eds.) PRIME 2022. LNCS, vol. 13564, pp. 137–148. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16919-9_13
Ebrahimighahnavieh, M.A., Luo, S., Chiong, R.: Deep learning to detect Alzheimer’s disease from neuroimaging: a systematic literature review. Comput. Methods Programs Biomed. 187, 105242 (2020)
Georgakis, M.K., et al.: Cerebral small vessel disease burden and cognitive and functional outcomes after stroke: a multicenter prospective cohort study. Alzheimer’s Dementia 19(4), 1152–1163 (2023)
Grothe, M., et al.: Reduction of basal forebrain cholinergic system parallels cognitive impairment in patients at high risk of developing Alzheimer’s disease. Cereb. Cortex 20(7), 1685–1695 (2010)
Heckemann, R.A., et al.: Brain extraction using label propagation and group agreement: pincram. PLoS ONE 10(7), e0129211 (2015)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Ledig, C., et al.: Robust whole-brain segmentation: application to traumatic brain injury. Med. Image Anal. 21(1), 40–58 (2015)
Lim, J.S., Lee, J.J., Woo, C.W.: Post-stroke cognitive impairment: pathophysiological insights into brain disconnectome from advanced neuroimaging analysis techniques. J. Stroke 23(3), 297–311 (2021)
Liu, M., et al.: A multi-model deep convolutional neural network for automatic hippocampus segmentation and classification in Alzheimer’s disease. Neuroimage 208, 116459 (2020)
Pölsterl, S., Wolf, T.N., Wachinger, C.: Combining 3D image and tabular data via the dynamic affine feature map transform. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 688–698. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_66
Ray, N.J., et al.: Cholinergic basal forebrain structure influences the reconfiguration of white matter connections to support residual memory in mild cognitive impairment. J. Neurosci. 35(2), 739–747 (2015)
Uysal, G., Ozturk, M.: Hippocampal atrophy based Alzheimer’s disease diagnosis via machine learning methods. J. Neurosci. Methods 337, 108669 (2020)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Verdelho, A., et al.: Cognitive impairment in patients with cerebrovascular disease: a white paper from the links between stroke ESO dementia committee. Eur. Stroke J. 6(1), 5–17 (2021)
Weaver, N.A., et al.: Strategic infarct locations for post-stroke cognitive impairment: a pooled analysis of individual patient data from 12 acute ischaemic stroke cohorts. Lancet Neurol. 20(6), 448–459 (2021)
Zheng, G., et al.: A transformer-based multi-features fusion model for prediction of conversion in mild cognitive impairment. Methods 204, 241–248 (2022)
Acknowledgements
The project was supported by the China Scholarship Council (File No. 202106210062) and ERC Grant Deep4MI (884622).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, S., Zhang, B., Fang, R., Rueckert, D., Zimmer, V.A. (2023). Dynamic Graph Neural Representation Based Multi-modal Fusion Model for Cognitive Outcome Prediction in Stroke Cases. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14227. Springer, Cham. https://doi.org/10.1007/978-3-031-43993-3_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-43993-3_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43992-6
Online ISBN: 978-3-031-43993-3
eBook Packages: Computer ScienceComputer Science (R0)
