Abstract
Multimodal fusion of different types of neural image data offers an invaluable opportunity to leverage complementary cross-modal information and has greatly advanced our understanding of mild cognitive impairment (MCI), a precursor to Alzheimer’s disease (AD). Current multi-modal fusion methods assume that both brain’s natural geometry and the related feature embeddings are in Euclidean space. However, recent studies have suggested that non-Euclidean hyperbolic space may provide a more accurate interpretation of brain connectomes than Euclidean space. In light of these findings, we propose a novel graph-based hyperbolic deep model with a learnable topology to integrate the individual structural network with functional information in hyperbolic space for the MCI/NC (normal control) classification task. We comprehensively compared the classification performance of the proposed model with state-of-the-art methods and analyzed the feature representation in hyperbolic space and its Euclidean counterparts. The results demonstrate the superiority of the proposed model in both feature representation and classification performance, highlighting the advantages of using hyperbolic space for multimodal fusion in the study of brain diseases. (Code is available here (https://github.com/nasyxx/MDF-HS).)
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allard, A., Serrano, M.Á.: Navigable maps of structural brain networks across species. PLoS Comput. Biol. 16(2), e1007584 (2020)
Association, A.: 2019 Alzheimer’s disease facts and figures. Alzheimer’s Dementia 15(3), 321–387 (2019)
Bachmann, G., Bécigneul, G., Ganea, O.: Constant curvature graph convolutional networks. In: International Conference on Machine Learning, pp. 486–496. PMLR (2020)
Benedetti, R., Petronio, C.: Lectures on hyperbolic geometry. Springer Science & Business Media (1992)
Bonnabel, S.: Stochastic gradient descent on Riemannian manifolds. IEEE Trans. Autom. Control 58(9), 2217–2229 (2013)
Chamberlain, B.P., Clough, J., Deisenroth, M.P.: Neural embeddings of graphs in hyperbolic space. arXiv preprint arXiv:1705.10359 (2017)
Chami, I., Ying, Z., Ré, C., Leskovec, J.: Hyperbolic graph convolutional neural networks. Adv. Neural Inform. Process. Syst. 32 (2019)
Dai, Z., et al.: Disrupted structural and functional brain net-works in Alzheimer’s disease. Neurobiol. Aging 75, 71–82 (2019)
Destrieux, C., Fischl, B., Dale, A., Halgren, E.: Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. Neuroimage 53(1), 1–15 (2010)
Fang, X., Liu, Z., Xu, M.: Ensemble of deep convolutional neural networks based multi-modality images for Alzheimer’s disease diagnosis. IET Image Proc. 14(2), 318–326 (2020)
Ganea, O., Bécigneul, G., Hofmann, T.: Hyperbolic neural networks. Adv. Neural Inform. Process. Syst. 31 (2018)
Greene, R.E.: S. gallot, d. hulin and j. lafontaine, riemannian geometry (1989)
Gromov, M.: Hyperbolic groups. In: Gersten, S.M., et al. (eds.) Essays in Group Theory, pp. 75–263. Springer, New York, NY (1987). https://doi.org/10.1007/978-1-4613-9586-7_3
Li, Y., Liu, J., Tang, Z., Lei, B.: Deep spatial-temporal feature fusion from adaptive dynamic functional connectivity for mci identification. IEEE Trans. Med. Imaging 39(9), 2818–2830 (2020)
Liu, Q., Nickel, M., Kiela, D.: Hyperbolic graph neural networks. Adv. Neural Inform. Process. Syst. 32 (2019)
Mathieu, E., Le Lan, C., Maddison, C.J., Tomioka, R., Teh, Y.W.: Continuous hierarchical representations with poincaré variational auto-encoders. Adv. Neural Informa. Process. Syst. 32 (2019)
Muscoloni, A., Thomas, J.M., Ciucci, S., Bianconi, G., Cannistraci, C.V.: Machine learning meets complex networks via coalescent embedding in the hyperbolic space. Nat. Commun. 8(1), 1615 (2017)
Newman, M.E.: Power laws, pareto distributions and Zipf’s law. Contemp. Phys. 46(5), 323–351 (2005)
Peng, W., Varanka, T., Mostafa, A., Shi, H., Zhao, G.: Hyperbolic deep neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(12), 10023–10044 (2021)
Shi, Y., et al.: Asmfs: Adaptive-similarity-based multi-modality feature selection for classification of Alzheimer’s disease. Pattern Recogn. 126, 108566 (2022)
Shimizu, R., Mukuta, Y., Harada, T.: Hyperbolic neural networks++. arXiv preprint arXiv:2006.08210 (2020)
Zhang, J., Zheng, B., Gao, A., Feng, X., Liang, D., Long, X.: A 3D densely connected convolution neural net-work with connection-wise attention mechanism for Alzheimer’s disease classification. Magn. Reson. Imaging 78, 119–126 (2021)
Zhang, L., et al.: Roimaging Initiative, A.D.N., et al.: Deep fusion of brain structure-function in mild cognitive impairment. Medical Image Anal. 72, 102082 (2021)
Zhang, L., Wang, L., Zhu, D.: Jointly analyzing Alzheimer’s disease related structure-function using deep cross-model attention net-work. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 563–567. IEEE (2020)
Zhang, L., Wang, L., Zhu, D.: Recovering brain structural connectivity from functional connectivity via multi-GCN based generative adversarial network. In: Martel, A.L., et al. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VII, pp. 53–61. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_6
Zhang, L., Wang, L., Zhu, D., Initiative, A.D.N., et al.: Predicting brain structural network using functional connectivity. Med. Image Anal. 79, 102463 (2022)
Zhang, L., Zaman, A., Wang, L., Yan, J., Zhu, D.: A cascaded multi-modality analysis in mild cognitive impairment. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 557–565. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32692-0_64
Zheng, M., Allard, A., Hagmann, P., Alemán-Gómez, Y., Serrano, M.Á.: Geometric renormalization unravels self-similarity of the multiscale human connectome. Proc. Natl. Acad. Sci. 117(33), 20244–20253 (2020)
Zhou, P., et al.: Use of a sparse-response deep belief network and extreme learning machine to discriminate Alzheimer’s dis-ease, mild cognitive impairment, and normal controls based on amyloid pet/mri images. Front. Med. 7, 621204 (2021)
Acknowledgement
This work was supported by National Institutes of Health (R01AG075582 and RF1NS128534).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, L., Na, S., Liu, T., Zhu, D., Huang, J. (2023). Multimodal Deep Fusion in Hyperbolic Space for Mild Cognitive Impairment Study. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14224. Springer, Cham. https://doi.org/10.1007/978-3-031-43904-9_65
Download citation
DOI: https://doi.org/10.1007/978-3-031-43904-9_65
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43903-2
Online ISBN: 978-3-031-43904-9
eBook Packages: Computer ScienceComputer Science (R0)
