Abstract
Deep learning-based brain disease diagnoses utilizing magnetic resonance (MR) images has attracted increasing attention in the field of computer-aided diagnosis. However, most existing methods require computationally expensive preprocessing before feature extraction, such as 3D MR image registration and landmark detection. Additionally, these methods only employ cross-sectional MR images. Recent studies have demonstrated that longitudinal images acquired at different time points can comprehensively reflect the pathological changes of diseases. To date, effectively capturing information from variable numbers of longitudinal MR images has not been adequately investigated. In this study, we propose a deep learning method taking advantage of longitudinal MR images for disease diagnoses. In particular, we first extract features from slice images employing a Deep Convolutional Neural Network (DCNN) in an end-to-end manner. This avoids 3D image registration and landmark detection. We then generate longitudinal-level features by using the Bag-of-Words (BoW) model. Lastly, we devise a Recurrent Neural Network (RNN) to capture the pathological changes for facilitating disease diagnoses. We evaluate the proposed method on the public Alzheimer’s Disease National Initiative (ADNI) dataset. Extensive experiments show that the proposed method is superior to baseline methods and is robust to both the Alzheimer’s disease (AD) and mild cognitive impairment (MCI) diagnoses. Moreover, the proposed method can effectively learn pathological changes from the longitudinal MR images for disease diagnosis.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Liu, S., Liu, S., Cai, W., Pujol, S., Kikinis, R., Feng, D.: Early diagnosis of Alzheimer’s disease with deep learning. In: ISBI, pp. 1015–1018. IEEE (2014)
Shi, J., Zheng, X., Li, Y., Zhang, Q., Ying, S.: Multimodal neuroimaging feature learning with multimodal stacked deep polynomial networks for diagnosis of Alzheimer’s disease. IEEE J. Biomed. Health Inf. 22(1), 173–183 (2018)
Suk, H.-I., Lee, S.-W., Shen, D., et al.: Deep ensemble learning of sparse regression models for brain disease diagnosis. Med. Image Anal. 37, 101–113 (2017)
Korolev, S., Safiullin, A., Belyaev, M., Dodonova, Y.: Residual and plain convolutional neural networks for 3D brain MRI classification. arXiv preprint arXiv:1701.06643 (2017)
Suk, H.-I., Shen, D.: Deep learning-based feature representation for AD/MCI classification. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 583–590. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_72
Suk, H.-I., Lee, S.-W., Shen, D., et al.: Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage 101, 569–582 (2014)
Chincarini, A., Sensi, F., Rei, L., Gemme, G., Squarcia, S., Longo, R., Brun, F., Tangaro, S., Bellotti, R., Amoroso, N., et al.: Integrating longitudinal information in hippocampal volume measurements for the early detection of Alzheimer’s disease. Neuroimage 125, 834–847 (2016)
Farzan, A., Mashohor, S., Ramli, A.R., Mahmud, R.: Boosting diagnosis accuracy of Alzheimer’s disease using high dimensional recognition of longitudinal brain atrophy patterns. Behav. Brain Res. 290, 124–130 (2015)
Zhang, J., Liu, M., An, L., Gao, Y., Shen, D.: Alzheimer’s disease diagnosis using landmark-based features from longitudinal structural mr images. IEEE J. Biomed. Health Inf. 21, 1067–1616 (2017)
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on CVPR, pp. 770–778 (2016)
Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: Null, p. 1470. IEEE (2003)
Yang, J., Price, B., Cohen, S., Lee, H., Yang, M.-H.: Object contour detection with a fully convolutional encoder-decoder network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 193–202 (2016)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248–255. IEEE (2009)
Noh, H., Hongsuck Seo, P., Han, B.: Image question answering using convolutional neural network with dynamic parameter prediction. In: Proceedings of the IEEE Conference on CVPR, pp. 30–38 (2016)
Cai, J., Lu, L., Xie, Y., Xing, F., Yang, L.: Pancreas segmentation in MRI using graph-based decision fusion on convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 674–682. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_77
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C Appl. Stat. 28(1), 100–108 (1979)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural comput. 9(8), 1735–1780 (1997)
Hinton, G., Srivastava, N., Swersky, K.: RmsProp: divide the gradient by a running average of its recent magnitude. Neural Networks For Machine Learning, Coursera Lecture (2012)
Matthews, B.W.: Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochim. et Biophys. Acta (BBA) Protein Struct. 405(2), 442–451 (1975)
Doran, G., Ray, S.: A theoretical and empirical analysis of support vector machine methods for multiple-instance classification. Mach. Learn. 97(1–2), 79–102 (2014)
Joachims, T.: Making large-scale SVM learning practical. SFB 475: Komplexitätsreduktion in Multivariaten Datenstrukturen, Universität Dortmund, Technical Report (1998)
Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)
Acknowledgement
The paper is partly supported by the National Natural Science Foundation of China under Grant No. 61672181, 51679058, Natural Science Foundation of Heilongjiang Province under Grant No. F2016005, the Numerical Tank Innovative Project (Phase I) and the Council Scholarship of China. Data collection and sharing was funded by ADNI (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions. We are also thankful for Mason Mcgough (his introduction is here https://www.bme.ufl.edu/labs/yang/group.html) for his serious presentation modification.
Author information
Authors and Affiliations
Consortia
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Gao, L. et al. (2018). Brain Disease Diagnosis Using Deep Learning Features from Longitudinal MR Images. In: Cai, Y., Ishikawa, Y., Xu, J. (eds) Web and Big Data. APWeb-WAIM 2018. Lecture Notes in Computer Science(), vol 10987. Springer, Cham. https://doi.org/10.1007/978-3-319-96890-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-96890-2_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-96889-6
Online ISBN: 978-3-319-96890-2
eBook Packages: Computer ScienceComputer Science (R0)