Skip to main content

Advertisement

SpringerLink
Log in

Re-transfer learning and multi-modal learning assisted early diagnosis of Alzheimer’s disease

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Nowadays more and more elderly people are suffering from Alzheimer’s disease (AD). Finely recognizing mild cognitive impairment (MCI) in early stage of the symptom is vital for AD therapy. However, brain image samples are relatively scarce, meanwhile have multiple modalities, which makes finely classifying brain images by computers extremely difficult. This paper proposes a fine-grained brain image classification approach for diagnosing Alzheimer’s disease, with re-transfer learning and multi-modal learning. First of all, an end-to-end deep neural network classifier CNN4AD is designed to finely classify diffusion tensor image (DTI) into four categories. And according to the characteristics of multi-modal brain image dataset, the re-transfer learning method is proposed based on transfer learning and multi-modal learning theories. Experimental results show that the proposed approach obtain higher accuracy with less labeled training samples. This could help doctors diagnose Alzheimer’s disease more timely and accurately.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Aderghal K, Boissenin M, Benois-Pineau J, Catheline G, Afdel K (2017) Classification of sMRI for AD diagnosis with convolutional neuronal networks: a pilot 2D study on ADNI. In: International conference on multimedia modeling, Springer, pp 690–701

  2. Afzal S, Maqsood M, Nazir F, Khan U, Aadil F, Awan KM, Mehmood I, Song O (2019) A data augmentation-based framework to handle class imbalance problem for Alzheimer’s stage detection. IEEE Access 7:115528–115539

    Article  Google Scholar 

  3. Ahmed OB, Benois-Pineau J, Allard M, Catheline G, Amar CB (2015) Alzheimer’s disease diagnosis on structural MR images using circular harmonic functions descriptors on hippocampus and posterior cingulate cortex. Comput Med Imaging Graph 44:13–25

    Article  Google Scholar 

  4. Ahmed OB, Benois-Pineau J, Allard M, Catheline G, Amar CB (2017) Recognition of Alzheimer’s disease and mild cognitive impairment with multimodal image-derived biomarkers and multiple kernel learning. Neurocomputing 220:98–110

    Article  Google Scholar 

  5. Ali A, Zhu Y, Chen Q, Yu J, Cai H (2019) Leveraging spatio-temporal patterns for predicting citywide traffic crowd flows using deep hybrid neural networks. In: 2019 IEEE 25Th international conference on parallel and distributed systems (ICPADS). IEEE, pp 125–132

  6. Ali A, Zhu Y, Zakarya M (2021) A data aggregation based approach to exploit dynamic spatio-temporal correlations for citywide crowd flows prediction in fog computing. Multimedia Tools and Applications 1–33

  7. Alnæs D, Kaufmann T, Richard G, Duff EP, Sneve MH, Endestad T, Nordvik JE, Andreassen OA, Smith SM, Westlye LT (2015) Attentional load modulates large-scale functional brain connectivity beyond the core attention networks. Neuroimage 109:260–272

    Article  Google Scholar 

  8. Andre E, Alexandre R, Bharath R, Volodymyr K, Mark D, Katherine C, Claire C, Greg C, Sebastian T, Jeff D (2019) A guide to deep learning in healthcare. Nature Medcine 24–29

  9. Bakkouri I, Afdel K, Benois-Pineau J, Catheline G (2019) Recognition of Alzheimer’s disease on sMRI based on 3D multi-scale CNN features and a gated recurrent fusion unit. In: 2019 international conference on content-based multimedia indexing (CBMI), pp 1–6

  10. Bihan DL, Johansen-Berg H (2012) Diffusion MRI at 25: Exploring brain tissue structure and function. NeuroImage 61(2):324–341

    Article  Google Scholar 

  11. Cheng D, Liu M, Fu J, Wang Y (2017) Classification of MR brain images by combination of multi-CNNs for AD diagnosis. In: ninth international conference on digital image processing (ICDIP 2017), international society for optics and photonics, vol 10420, p 1042042

  12. Ebadi A, Dalboni da Rocha JL, Nagaraju DB, Tovar-Moll F, Bramati I, Coutinho G, Sitaram R, Rashidi P (2017) Ensemble classification of Alzheimer’s disease and mild cognitive impairment based on complex graph measures from diffusion tensor images. Front Neurosci 11:56

    Article  Google Scholar 

  13. Glozman T, Liba O (2016) Hidden cues: Deep learning for Alzheimer’s disease classification CS331B project final report

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  15. Heppner FL, Ransohoff RM, Becher B (2015) Immune attack: the role of inflammation in Alzheimer disease. Nat Rev Neurosci 16(6):358

    Article  Google Scholar 

  16. Hosseini-Asl E, Keynton R, El-Baz A (2016) Alzheimer’s disease diagnostics by adaptation of 3D convolutional network. In: 2016 IEEE international conference on image processing (ICIP), pp 126–130

  17. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  18. Islam J, Zhang Y (2017) A novel deep learning based multi-class classification method for Alzheimer’s disease detection using brain MRI data. In: Brain informatics. Springer International Publishing, Cham, pp 213–222

  19. Jack Jr CR, Bernstein MA, Fox NC, Thompson P, Alexander G, Harvey D, Borowski B, Britson PJ, L Whitwell J, Ward C, Dale AM, Felmlee JP, Gunter JL, Hill DL, Killiany R, Schuff N, Fox-Bosetti S, Lin C, Studholme C, DeCarli CS, Krueger G, Ward HA, Metzger GJ, Scott KT, Mallozzi R, Blezek D, Levy J, Debbins JP, Fleisher AS, Albert M, Green R, Bartzokis G, Glover G, Mugler J, Weiner MW (2008) The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J Magn Reson Imaging 27(4):685–691

    Article  Google Scholar 

  20. Khvostikov A, Aderghal K, Krylov AS, Catheline G, Benois-Pineau J (2018) 3D inception-based CNN with sMRI and MD-DTI data fusion for alzheimer’s disease diagnostics. arXiv:1809.03972

  21. Korolev IO (2014) Alzheimer disease: a clinical and basic science review. Med Stud Res J 4(1):24–33

    Google Scholar 

  22. Korolev S, Safiullin A, Belyaev M, Dodonova Y (2017) Residual and plain convolutional neural networks for 3D brain MRI classification. In: 2017 IEEE 14Th international symposium on biomedical imaging (ISBI), vol 2017, pp 835–838

  23. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  24. Lei B, Chen S, Ni D, Wang T (2016) Discriminative learning for Alzheimer’s disease diagnosis via canonical correlation analysis and multimodal fusion. Front Aging Neurosci 8:77

    Article  Google Scholar 

  25. Lella E, Lombardi A, Amoroso N, Diacono D, Maggipinto T, Monaco A, Bellotti R, Tangaro S (2020) Machine learning and dwi brain communicability networks for Alzheimer’s disease detection. Appl Sci 10(3):934

    Article  Google Scholar 

  26. Lella E, Vessio G (2020) Ensembling complex network ‘perspectives’ for mild cognitive impairment detection with artificial neural networks. Pattern Recognition Letters

  27. Lu S, Xia Y, Cai TW, Feng DD (2015) Semi-supervised manifold learning with affinity regularization for Alzheimer’s disease identification using positron emission tomography imaging. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 2251–2254

  28. Luo S, Li X, Li J (2017) Automatic Alzheimer’s disease recognition from MRI data using deep learning method. J Appl Math Phys 5(09):1892

    Article  Google Scholar 

  29. Magnin B, Mesrob L, Kinkingnéhun S, Pélégrini-Issac M, Colliot O, Sarazin M, Dubois B, Lehéricy S, Benali H (2009) Support vector machine-based classification of Alzheimer’s disease from whole-brain anatomical MRI. Neuroradiology 51(2):73–83

    Article  Google Scholar 

  30. Mengqing M, Fazhi H, Shan X (2020) Attention deep residual networks for MR image analysis. Neural Comput & Applic 1–10

  31. Mwangi B, Tian TS, Soares JC (2014) A review of feature reduction techniques in neuroimaging. Neuroinformatics 12(2):229–244

    Article  Google Scholar 

  32. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E (1999) Mild cognitive impairment: clinical characterization and outcome. Arch Neurol 56(3):303–308

    Article  Google Scholar 

  33. Suk HI, Wee CY, Lee SW, Shen D (2016) State-space model with deep learning for functional dynamics estimation in resting-state fMRI. NeuroImage 129:292–307

    Article  Google Scholar 

  34. Tufail AB, Ma YK, Zhang QN (2020) Binary classification of Alzheimer’s disease using sMRI imaging modality and deep learning. J Digit Imaging 1–18

  35. Vu TD, Yang HJ, Nguyen VQ, Oh AR, Kim MS (2017) Multimodal learning using convolution neural network and sparse autoencoder. In: 2017 IEEE International conference on big data and smart computing (BigComp). IEEE, pp 309–312

  36. Weiner MW, Aisen P, Mackin S (2017) Alzheimer’s Disease Neuroimaging Initiative. http://adni.loni.usc.edu/, accessed 1 Aug 2019

  37. Woo S, Park J, Lee JY, So Kweon I (2018) CBAM: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  38. Wood D, Cole J, Booth T (2019) NEURO-DRAM: a 3D recurrent visual attention model for interpretable neuroimaging classification. arXiv:191004721

  39. Yang J, Li S, Xu W (2017) Active learning for visual image classification method based on transfer learning. IEEE Access 6:187–198

    Article  Google Scholar 

  40. Yann L, Yoshua B, Geoffrey H (2015) Deep learning. Nature 436–444

  41. Yong P, Bao-Liang L (2017) Discriminative extreme learning machine with supervised sparsity preserving for image classification. Neurocomputing 242–252

  42. Zu C, Jie B, Liu M, Chen S, Shen D, Zhang D (2016) The Alzheimer’s Disease Neuroimaging Initiative. Label-aligned multi-task feature learning for multimodal classification of Alzheimer’s disease and mild cognitive impairment. Brain Imaging and Behavior 10(4):1148–1159

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by National Natural Science Foundation of China (Nos. 62072126, 61772164, 61972121, 61971173, U1909210), Zhejiang Provincial Natural Science Foundation of China (Nos. L221F020008, LY21F020015). The authors would like to thank the reviewers for their comments and suggestions in advance.

Author information

Authors and Affiliations

  1. School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, China

    Meie Fang

  2. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China

    Zhuxin Jin, Feiwei Qin & Yong Peng

  3. Engineering Research Center of Cognitive Healthcare of Zhejiang Province, Sir Run Run Shaw Hospital, Zhejiang, China

    Feiwei Qin & Chao Jiang

  4. School of Artifical Intelligence, Nanjing University of Information Science & Technology, Nanjing, China

    Zhigeng Pan

  5. Digital Media, Interaction, Hangzhou Normal University, Hangzhou, China

    Zhigeng Pan

Authors
  1. Meie Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhuxin Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feiwei Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhigeng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feiwei Qin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, M., Jin, Z., Qin, F. et al. Re-transfer learning and multi-modal learning assisted early diagnosis of Alzheimer’s disease. Multimed Tools Appl 81, 29159–29175 (2022). https://doi.org/10.1007/s11042-022-11911-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-11911-6

Keywords

Search

Navigation