Abstract
The classification and identification of brain diseases with multimodal information have attracted increasing attention in the domain of computer-aided. Compared with traditional method which use single modal feature information, multiple modal information fusion can classify and diagnose brain diseases more comprehensively and accurately in patient subjects. Existing multimodal methods require manual extraction of features or additional personal information, which consumes a lot of manual work. Furthermore, the difference between different modal images along with different manual feature extraction make it difficult for models to learn the optimal solution. In this paper, we propose a multimodal 3D convolutional neural networks framework for classification of brain disease diagnosis using MR images data and PET images data of subjects. We demonstrate the performance of the proposed approach for classification of Alzheimer’s disease (AD) versus mild cognitive impairment (MCI) and normal controls (NC) on the Alzheimer’s Disease National Initiative (ADNI) data set of 3D structural MRI brain scans and FDG-PET images. Experimental results show that the performance of the proposed method for AD vs. NC, MCI vs. NC are 93.55% and 78.92% accuracy respectively. And the accuracy of the results of AD, MCI and NC 3-classification experiments is 68.86%.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Thung, K.H., Wee, C., Yap, P.T., Shen, D.: Neurodegenerative disease diagnosis using incomplete multi-modality data via matrix shrinkage and completion. NeuroImage 91, 386–400 (2014)
Zhan, L., et al.: Comparison of 9 tractography algorithms for detecting abnormal structural brain networks in Alzheimers disease. Front. Aging Neurosci. 48(7), 401–408 (2015)
The State of The Art of Dementia Research: New Frontiers. https://www.alz.co.uk/research/world-report-2018/. Accessed Sept 2018
Manivannan, W., Li, S., Akbar, S., Zhang, J., Trucco, E., McKenna, S.J.: Gland segmentation in colon histology images using hand-crafted features and convolutional neural networks. In: IEEE International Symposium on Biomedical Imaging, vol. 4, pp. 1405–1408 (2016)
Jia, W., Li, F., Hu, Q.: Automatic segmentation of liver tumor in CT images with deep convolutional neural networks. J. Comput. Commun. 11(3), 146–151 (2015)
Litjens, G., et al.: Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Nat. Sci. Rep. 6, 26286 (2016)
Haiwei, P., Pengyuan, L., Qing, L., Qilong, H., Xiaoning, F., Linlin, G.: Brain CT image similarity retrieval method based on uncertain location graph. IEEE J. Biomed. Health Inform. 18(2), 574–584 (2014)
Linlin, G., Haiwei, P., Xiaoqin, X., Zhiqiang. Z., Qing L., Qilong, H.: Graph modeling and mining methods for brain images. Multimedia Tools Appl. 75(15), 9333–9369 (2016)
Linlin, G., et al.: Brain medical image diagnosis based on corners with importance-values. BMC Bioinform. 18(1), 505 (2017)
Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Ann. Rev. Biomed. Engl. 19(1), 221–248 (2017)
Gray, K.R., Wolz, R., Heckemann, R.A., Aljabar, P., Hammmers, A., Rueckert, D.: Multi-region analysis of longitudinal FDG-PET for the classification of Alzheimer’s disease. NeuroImage 60(1), 221–229 (2012)
Gray, K.R., Wolz, R., Keihaninejad, S.: Regional analysis of FDG-PET for using in the classification of Alzheimer’s disease. In: IEEE International Symposium on Biomedical Imaging, vol. 3, pp. 1082–1085 (2011)
Garali, I., Adel, M., Bourennane, S., Guedj, E.: Region-based brain selection and classification on pet images for Alzheimer’s disease computer aided diagnosis. In: IEEE International Conference on Image Processing, vol. 10, no. 9, pp. 27–30 (2015)
Silveira, M., Marques, J.: Boosting Alzheimer disease diagnosis using PET images. In: IEEE International Conference on Pattern Recognition (ICPR), vol. 8, pp. 2556–2559 (2015)
Liu, M., Cheng, D., Yan, W.: Classification of Alzheimer’s disease by combination of convolutional and recurrent neural networks using FDG-PET images. Front. Neuroinf. 12(6), 1–12 (2018)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Hosseini-Asl, E., Keynton, R., El-Baz, A.: Alzheimer’s disease diagnostics by adaptation of 3D fonvolutional network. In: IEEE International Conference on Image Processing, vol. 7, pp. 126–130 (2016)
Jianxu, C., Liu, Y., Yizhe, Z., Alber, M.: Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation. Neural Inf. Process. Syst. 6(9), 1–9 (2016)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. Neural Inf. Process. Syst. 21(6), 1–8 (2016)
Korolev, S., Safiullin, A., Belyaev, M., Dodonova, Y.: Residual and plain convolutional neural networks for 3D brain MRI classification. In: IEEE International Symposium on Biomedical Imaging, vol. 6 (2017)
Kaiming, H., Xiangyu, Z., Shaoqing, R., Jian, S.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 6, pp. 770–778 (2016
Voxresnet: deep voxelwise residual networks for volumetric brain segmentation. https://arxiv.org/abs/1608.05895/. Accessed Aug 2016
Daoqiang, Z., Dinggang, S.: Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. Neuroimage 18(9), 895–907 (2012)
Acknowledgements
This work was supported by the National Natural Science Foundation of China under Grant No. 61672181, No. 51679058, Natural Science Foundation of Heilongjiang Province under Grant No. F2016005. We would like to thank our teacher for guiding this paper. We would also like to thank classmates for their encouragement and help.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Han, K., Pan, H., Gao, R., Yu, J., Yang, B. (2019). Multimodal 3D Convolutional Neural Networks for Classification of Brain Disease Using Structural MR and FDG-PET Images. In: Cheng, X., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2019. Communications in Computer and Information Science, vol 1058. Springer, Singapore. https://doi.org/10.1007/978-981-15-0118-0_51
Download citation
DOI: https://doi.org/10.1007/978-981-15-0118-0_51
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-0117-3
Online ISBN: 978-981-15-0118-0
eBook Packages: Computer ScienceComputer Science (R0)