Skip to main content

Advertisement

SpringerLink
Log in

Deep learning and multimodal feature fusion for the aided diagnosis of Alzheimer's disease

  • S.I: Deep learning modelling in real life: (Anomaly Detection, Biomedical, Concept Analysis, Finance, Image analysis, Recommendation)
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The accurate diagnosis of Alzheimer’s disease (AD) in the early stages, such as significant memory concern (SMC) and mild cognitive impairment (MCI), is essential in order to slow its progression through timely treatment. Recent achievements have shown that fusing multimodal neuroimaging data effectively facilitates AD diagnosis. However, most proposed fusion methods simply add or concatenate multimodal features and do not make full use of nonlinear features and texture features across the range of modalities. This paper proposes a diagnostic model that effectively diagnoses AD in different stages by fusing functional magnetic resonance imaging (fMRI) and structural MRI (sMRI) information. First, fMRI and sMRI scans are preprocessed, and mean regional homogeneity (mReHo) transformation is performed for the preprocessed fMRI scans. Then, 3DMR-PCANet extracts features of mReHo images. The basic ResNet module is stacked to build a 3DResNet-10 model for feature extraction of sMRI scans. Next, two image features are fused by kernel canonical correlation analysis. Finally, a support vector machine (SVM) is utilized for the classification of fused features. Experimental results on the Alzheimer's Disease Neuroimaging dataset demonstrate the effectiveness of the proposed method. Specifically, this method improves on the accuracy, specificity, sensitivity, F1 value and area under the curve (AUC) of existing methods in comparisons of the normal control (NC) versus SMC, NC versus MCI, NC versus AD, SMC versus MCI, SMC versus AD, and MCI versus AD groups, which confirms that the proposed method can mine information on the correlation between fMRI and sMRI data of the same subject and can effectively classify AD patients in different stages.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Chung S, Yang J, Kim HJ et al (2021) Plexin-A4 mediates amyloid-β–induced tau pathology in Alzheimer’s disease animal model. Prog Neurobiol 2021(203):3592–3604

    Google Scholar 

  2. Jia J, Wei C, Chen S et al (2018) The cost of Alzheimer’s disease in China and re-estimation of costs worldwide. Alzheimers Dement 14(4):483–491

    Article  MathSciNet  Google Scholar 

  3. Leifer BP (2009) Alzheimer’s disease: Seeing the signs early. J Am Acad Nurse Pract 21(11):588–595

    Article  Google Scholar 

  4. Clark CM, Karlawish JHT (2003) Alzheimer disease: current concepts and emerging diagnostic and therapeutic strategies. Ann Intern Med 138(5):400–410

    Article  Google Scholar 

  5. Reisberg B, Shulman MB, Torossian C et al (2010) Outcome over seven years of healthy adults with and without subjective cognitive impairment. Alzheimer’s Dementia 6(1):11–24

    Article  Google Scholar 

  6. Mitchell AJ, Beaumont H, Ferguson D et al (2015) Risk of dementia and mild cognitive impairment in older people with subjective memory complaints: meta-analysis. Acta Psychiatr Scand 130(6):439–451

    Article  Google Scholar 

  7. Ronnlund M, Sundstro A, Adolfsson R et al (2015) Self-Reported Memory Failures: Associations with Future Dementia in a Population-Based Study with Long-Term Follow-Up. J Am Geriatr Soc 63(9):1766–1773

    Article  Google Scholar 

  8. Sanford AM (2017) Mild Cognitive Impairment. Clin Geriatr Med 33(3):325–337

    Article  Google Scholar 

  9. Weiner MW, Veitch DP, Aisen PS et al (2013) The Alzheimer’s Disease Neuroimaging Initiative: a review of papers published since its inception. Alzheimers Dement 8(1):1–68

    Google Scholar 

  10. Winblad B, Amouyel P, Andrieu S et al (2016) Defeating Alzheimer’s disease and other dementias: a priority for European science and society. The Lancet Neurology 15(5):455–532

    Article  Google Scholar 

  11. Choi BK, Madusanka N, Choi HK et al (2020) Convolutional Neural Network-based MR Image Analysis for Alzheimer’s Disease Classification. Current Medical Imaging Reviews 16(1):27–35

    Article  Google Scholar 

  12. Jain R, Jain N, Aggarwal A et al (2019) Convolutional neural network based Alzheimer’s disease classification from magnetic resonance brain images. Cogn Syst Res 2019(57):147–159

    Article  Google Scholar 

  13. Zhu X, Suk HI, Shen D (2014) A novel matrix-similarity based loss function for joint regression and classification in AD diagnosis. Neuroimage 100:91–105

    Article  Google Scholar 

  14. Wang XY, Xie J, Chen XJ (2021) Applications of non-invasive and novel methods of low-field nuclear magnetic resonance and magnetic resonance imaging in aquatic products. Front Nutr 8:651–804

    Google Scholar 

  15. Yamanakkanavar N, Choi JY, Lee B (2020) MRI segmentation and classification of human brain using deep learning for diagnosis of alzheimer’s disease: a survey. Sensors 20(11):3243

    Article  Google Scholar 

  16. Scarpelli ML, Healey DR, Mehta S et al (2020) A practical method for multimodal registration and assessment of whole-brain disease burden using PET, MRI, and optical imaging. Sci Rep 10(1):17324

    Article  Google Scholar 

  17. Woo YJ, Roussos P, Haroutunian V et al (2020) Comparison of brain connectomes by MRI and genomics and its implication in Alzheimer’s disease. BMC Med 18(1):23

    Article  Google Scholar 

  18. Provenzano D, Washington SD, Rao YJ et al (2020) logistic regression algorithm differentiates gulf war illness (GWI) functional magnetic resonance imaging (fMRI) data from a sedentary control. Brain Sci 10(5):319

    Article  Google Scholar 

  19. Faria D, Marques AJ, Balardin J et al (2020) Task-related brain activity and functional connectivity in upper limb dystonia: a functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS) study. Neurophotonics 7(4):45004–45001

    Article  Google Scholar 

  20. Acm A, Tb C, Gw A et al (2021) Cloud-based functional magnetic resonance imaging neurofeedback to reduce the negative attentional bias in depression: a proof-of-concept study - sciencedirect. Biol Psychiatry Cognitive Neurosci Neuroimag 6(4):490–497

    Article  Google Scholar 

  21. Farheen RM et al (2019) A Deep Learning approach for automated diagnosis and multi-class classification of alzheimer's disease stages using resting-state fmri and residual neural networks. J Med Syst 44(2):37–37

  22. Khullar V, Salgotra K, Singh HP et al (2021) Deep learning-based binary classification of ADHD using resting state MR images. Augment Human Res 6(1):1–9

  23. Sun J, Ding Y, Zhao K et al (2020) Predicting Alzheimer’s disease based on network topological latent representations. J Med Imag Health Inf 10(3):667–671

    Google Scholar 

  24. Chen Z, Zhao R, Wang Q et al (2020) Functional connectivity changes of the visual cortex in the cervical spondylotic myelopathy patients: a resting-state fMRI study. Spine 5(45):272–279

    Article  Google Scholar 

  25. Ju R, Hu C, Zhou P et al (2017) Early diagnosis of Alzheimer’s disease based on resting-state brain networks and deep learning. IEEE/ACM Trans Comput Biol Bioinf 16(1):244–257

    Article  Google Scholar 

  26. Kam TE, Zhang H, Jiao Z et al(2019) Deep Learning of Static and Dynamic Brain Functional Networks for Early MCI Detection. IEEE Trans Med Imag (99):1–1.

  27. Bi X, Zhao X, Huang H et al (2020) Functional brain network classification for Alzheimer’s disease detection with deep features and extreme learning machine. Cogn Comput 12(2):513–527

    Article  Google Scholar 

  28. Yu M, Nho K, Risacher SL et al (2020) Transcriptomic profiles underlying functional brain networks at different stages of Alzheimer’s disease: Genetics/genetic factors of Alzheimer’s disease. Alzheimers Dement 16(S3):1–3

    Article  Google Scholar 

  29. Jalilianhasanpour R, Beheshtian E, Sherbaf G et al (2019) Functional connectivity in neurodegenerative disorders: Alzheimer’s disease and frontotemporal dementia. Top Magn Reson Imaging 28(6):317–324

    Article  Google Scholar 

  30. Jiang L, Zuo XN (2016) Regional homogeneity. Neuroscientist 22(5):486–505

    Article  Google Scholar 

  31. Huang Y, Xu J, Zhou Y et al (2010) Diagnosis of alzheimer’s disease via multi-modality 3d convolutional neural network. Front Neurosci 13:509

    Article  Google Scholar 

  32. Manhua LD et al (2018) Multi-Modality Cascaded Convolutional Neural Networks for Alzheimer's Disease Diagnosis.. Neuroinformatics 16(3):295–308

  33. Chan TH, Jia K, Gao S et al (2015) PCANet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017–5032

    Article  MathSciNet  MATH  Google Scholar 

  34. Tong LS, Bin X, Sheng LW et al (2018) Aided diagnosis of Alzheimer’s disease based on 3D-PCANET. Comput Sci 45(6):153–155

    Google Scholar 

  35. Lee S, Kim KW (2020) Associations between texture of T1-weighted magnetic resonance imaging and radiographic pathologies in Alzheimer’s disease. Eur J Neurol 28(3):735–744

    Article  MathSciNet  Google Scholar 

  36. Yoa B, Tia B, Ts C et al (2020) Visual texture agnosia influences object identification in dementia with Lewy bodies and Alzheimer’s disease - ScienceDirect. Cortex 129:23–32

    Article  Google Scholar 

  37. Cai JH, He Y, Zhong XL et al (2020) Magnetic Resonance Texture Analysis in Alzheimer’s disease. Acad Radiol 27(12):1774–1783

    Article  Google Scholar 

  38. Leandrou S, Lamnisos D, Kyriacou PA et al (2020) Comparison of 1.5 T and 3 T MRI hippocampus texture features in the assessment of Alzheimer's disease. Biomed Signal Process Control 62:102098

  39. He K, Zhang X, Ren S et al (2016) Identity mappings in deep residual networks.In: European conference on computer vision. Springer International Publishing, pp 630–645

  40. Ning J, Zhao H, Lan L et al (2019) A computer-aided detection system for the detection of lung nodules based on 3D-ResNet. Appl Sci 9(24):5544

    Article  Google Scholar 

  41. Ashkan E, Dalboni D, Nagaraju DB et al (2017) Ensemble classification of Alzheimer’s disease and mild cognitive impairment based on complex graph measures from diffusion tensor images. Frontiers in Neuroence 11:11–56

    Google Scholar 

  42. Beheshti I, Maikusa N, Daneshmand M, et al (2017) Classification of Alzheimer's disease and prediction of mild cognitive impairment conversion using histogram-based analysis of patient-specific anatomical brain connectivity networks. J Alzheimer\"s Dis 60(1):295–304

  43. Vos FD, Schouten TM, Koini M et al (2020) Pre-trained MRI-based Alzheimer's disease classification models to classify memory clinic patients. NeuroImage Clin 27:102303

  44. Korolev S, Safiullin A, Belyaev M et al (2017) Residual and plain convolutional neural networks for 3D brain MRI classification. In: IEEE International Symposium on Biomedical Imaging 2017, Melbourne, VIC,pp 1945–8452

  45. Ren F, Yang C, Qiu Q et al (2019) Exploiting discriminative regions of brain slices based on 2D CNNs for Alzheimer’s disease classification. IEEE Access 7(99):181423–181433

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Big Data Technology for Food Safety, School of Artificial Intelligence, Beijing Technology and Business University, Beijing, 100048, China

    Hongfei Jia

  2. School of Artificial Intelligence, Guangxi Minzu University, Nanning, Guangxi, 530004, People’s Republic of China

    Huan Lao

Authors
  1. Hongfei Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huan Lao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongfei Jia.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest regarding the publication of this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jia, H., Lao, H. Deep learning and multimodal feature fusion for the aided diagnosis of Alzheimer's disease. Neural Comput & Applic 34, 19585–19598 (2022). https://doi.org/10.1007/s00521-022-07501-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07501-0

Keywords

Search

Navigation