Abstract
Alzheimer’s Disease (AD) is the most common form of dementia and often difficult to diagnose due to the multifactorial etiology of dementia. Recent works on neuroimaging-based computer-aided diagnosis with deep neural networks (DNNs) showed that fusing structural magnetic resonance images (sMRI) and fluorodeoxyglucose positron emission tomography (FDG-PET) leads to improved accuracy in a study population of healthy controls and subjects with AD. However, this result conflicts with the established clinical knowledge that FDG-PET better captures AD-specific pathologies than sMRI. Therefore, we propose a framework for the systematic evaluation of multi-modal DNNs and critically re-evaluate single- and multi-modal DNNs based on FDG-PET and sMRI for binary healthy vs. AD, and three-way healthy/mild cognitive impairment/AD classification. Our experiments demonstrate that a single-modality network using FDG-PET performs better than MRI (accuracy 0.91 vs 0.87) and does not show improvement when combined. This conforms with the established clinical knowledge on AD biomarkers, but raises questions about the true benefit of multi-modal DNNs. We argue that future work on multi-modal fusion should systematically assess the contribution of individual modalities following our proposed evaluation framework. Finally, we encourage the community to go beyond healthy vs. AD classification and focus on differential diagnosis of dementia, where fusing multi-modal image information conforms with a clinical need.
M. Narazani and I. Sarasua—These authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aisen, P.S., Cummings, J., Jack, C.R., Morris, J.C., Sperling, R.: On the path to 2025: understanding the Alzheimer’s disease continuum. Alzheimers Res. Ther. 9(1), 60 (2017)
Basaia, S., et al.: Automated classification of Alzheimer’s disease and mild cognitive impairment using a single MRI and deep neural networks. NeuroImage Clin. 21, 101645 (2019)
Bloudek, L.M., Spackman, D.E., Blankenburg, M., Sullivan, S.D.: Review and meta-analysis of biomarkers and diagnostic imaging in Alzheimer’s disease. J. Alzheimer’s Dis. 26(4), 627–645 (2011)
Borson, S., et al.: Improving dementia care: the role of screening and detection of cognitive impairment. Alzheimer’s Dement 9(2), 151–159 (2013)
Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The balanced accuracy and its posterior distribution. In: 20th International Conference on Pattern Recognition, pp. 3121–3124 (2010)
Ding, Y., Sohn, J.H., Kawczynski, M.G., Trivedi, H., Harnish, R., et al.: A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain. Radiology 290(2), 456–464 (2019)
Ebrahimighahnavieh, M.A., Luo, S., Chiong, R.: Deep learning to detect Alzheimer’s disease from neuroimaging: a systematic literature review. Comput. Methods Programs Biomed. 187, 105242 (2020)
Esmaeilzadeh, S., Belivanis, D.I., Pohl, K.M., Adeli, E.: End-to-end Alzheimer’s disease diagnosis and biomarker identification. In: MLMI, pp. 337–345 (2018)
Farooq, A., Anwar, S., Awais, M., Rehman, S.: A deep CNN based multi-class classification of Alzheimer’s disease using MRI. In: IST, pp. 1–6 (2017)
Feng, C., et al.: Deep learning framework for Alzheimer’s disease diagnosis via 3D-CNN and FSBi-LSTM. IEEE Access 7, 63605–63618 (2019)
Fonov, V., et al.: Unbiased average age-appropriate atlases for pediatric studies. Neuroimage 54(1), 313–327 (2011)
Frisoni, G.B., et al.: Imaging markers for Alzheimer disease: which vs how. Neurology 81(5), 487–500 (2013)
Gaser, C., Dahnke, R., et al.: Cat-a computational anatomy toolbox for the analysis of structural MRI data. HBM 2016, 336–348 (2016)
Hosseini-Asl, E., Gimel’farb, G., El-Baz, A.: Alzheimer’s disease diagnostics by a deeply supervised adaptable 3D convolutional network. arXiv preprint arXiv:1607.00556 (2016)
Huang, Y., et al.: Diagnosis of Alzheimer’s disease via multi-modality 3D convolutional neural network. Front. Neurosci. 13, 509 (2019)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, pp. 448–456 (2015)
Jack, C.R., Jr., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Resonan. Imaging 27(4), 685–691 (2008)
Korolev, S., Safiullin, A., Belyaev, M., Dodonova, Y.: Residual and plain convolutional neural networks for 3D brain MRI classification. In: ISBI, pp. 835–838 (2017)
Li, F., Cheng, D., Liu, M.: Alzheimer’s disease classification based on combination of multi-model convolutional networks. In: IST, pp. 1–5 (2017)
Liu, M., Cheng, D., Yan, W., et al.: Classification of Alzheimer’s disease by combination of convolutional and recurrent neural networks using FDG-PET images. Front. Neuroinform. 12, 35 (2018)
Livingston, G., Sommerlad, A., Orgeta, V., Costafreda, S.G., Huntley, J., et al.: Dementia prevention, intervention, and care. The Lancet 390(10113), 2673–2734 (2017)
Marcus, C., Mena, E., Subramaniam, R.M.: Brain PET in the diagnosis of Alzheimer’s disease. Clin. Nucl. Med. 39(10), e413 (2014)
Payan, A., Montana, G.: Predicting Alzheimer’s disease: a neuroimaging study with 3D convolutional neural networks. arXiv preprint arXiv:1502.02506 (2015)
Song, J., Zheng, J., Li, P., Lu, X., Zhu, G., Shen, P.: An effective multimodal image fusion method using MRI and PET for Alzheimer’s disease diagnosis. Front. Digit Health 3, 19 (2021)
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: ICML, pp. 3319–3328 (2017)
Teipel, S., Kilimann, I., Thyrian, J.R., Kloppel, S., Hoffmann, W.: Potential role of neuroimaging markers for early diagnosis of dementia in primary care. Curr. Alzheimer Res. 15(1), 18–27 (2017)
Wang, Y., Huang, W., Sun, F., Xu, T., Rong, Y., Huang, J.: Deep multimodal fusion by channel exchanging. NeurIPS 33, 4835–4845 (2020)
Yee, E., et al.: Quantifying brain metabolism from FDG-PET images into a probability of Alzheimer’s dementia score. Hum. Brain Mapp. 41(1), 5–16 (2020)
Zhang, D., et al.: Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage 55(3), 856–867 (2011)
Zhou, T., Thung, K.H., Zhu, X., Shen, D.: Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis. Hum. Brain Mapp. 40(3), 1001–1016 (2019)
Acknowledgment
This research was partially supported by the Bavarian State Ministry of Science and the Arts and coordinated by the bidt, and the Federal Ministry of Education and Research in the call for Computational Life Sciences (DeepMentia, 031L0200A). We gratefully acknowledge the computational resources provided by the Leibniz Supercomputing Centre (www.lrz.de).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Narazani, M., Sarasua, I., Pölsterl, S., Lizarraga, A., Yakushev, I., Wachinger, C. (2022). Is a PET All You Need? A Multi-modal Study for Alzheimer’s Disease Using 3D CNNs. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13431. Springer, Cham. https://doi.org/10.1007/978-3-031-16431-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-16431-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16430-9
Online ISBN: 978-3-031-16431-6
eBook Packages: Computer ScienceComputer Science (R0)
