Abstract
Degenerative disorders such as Parkinson’s disease (PD) have an influence on daily activities due to rigidity of muscles, tremor or cognitive impairment. Micrographia, speech intensity, and deficient generation of voluntary saccadic eye movements (Pretegiani and Optican in Front Neurol 8:592, 2017) are manifestations of PD that can be used to devise noninvasive and low cost clinical tests. In this context, we have collected a multimodal dataset that we call Parkinson’s disease Multi-Modal Collection (PDMultiMC), which includes online handwriting, speech signals, and eye movements recordings. We present here the handwriting dataset that we call HandPDMultiMC that will be made publicly available. The HandPDMultiMC dataset includes handwriting samples from 42 subjects (21 PD and 21 controls). In this work we investigate the application of various Deep learning architectures, namely the CNN and the CNN-BLSTM, to PD detection through time series classification. Various approaches such as Spectrograms have been applied to encode pen-based signals into images for the CNN model, while the raw time series are directly used in the CNN-BLSTM. In order to train these models for PD detection on large scale data, various data augmentation approaches for pen-based signals are proposed. Experimental results on our dataset show that the best performance for early PD detection (97.62% accuracy) is reached by a combination of CNN-BLSTM models trained with Jittering and Synthetic data augmentation approaches. We also illustrate that deep architectures can surpass the models trained on pre-engineered features even though the available data is small.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atienza R (2017) LSTM by example using Tensor-flow. https://towardsdatascience.com/lstm-by-example-using-tensorflow-feb0c1968537. Accessed 3 July 2019
Brownlee J (2019) Train neural networks with noise to reduce overfitting. Machine learning mastery. https://machinelearningmastery.com/train-neural-networks-with-noise-to-reduce-overfitting. Accessed 11 Sept 2019
Drotar P et al (2015) Contribution of different handwriting modalities to differential diagnosis of Parkinson’s disease. In: IEEE international symposium on medical measurements and applications (MeMeA), pp 344–348
Drotar P et al (2016) Evaluation of handwriting kinematics and pressure for differential diagnosis of Parkinson’s disease. Artif Intell Med 67:39–46
Fawaz HI, Forestier G, Weber J, Idoumghar L, Muller PA (2018) arxiv.org/abs/1808.02455
Gallicchio C, Micheli A, Pedrelli L (2018) Deep tree echo state networks. IJCNN. https://doi.org/10.1109/ijcnn.2018.8489464
Gamboa JCB (2017) Deep learning for time-series analysis. https://arxiv.org/pdf/1701.01887.pdf. Accessed 3 July 2019
Gómez-Vilda P et al (2017) Parkinson disease detection from speech articulation neuromechanics. Front Neuroinform 11:56
Himmetoglu B (2017) Time series classification with Tensor flow. https://burakhimmetoglu.com/2017/08/22/time-series-classification-with-tensorflow/. Accessed 3 July 2019
Huzaifah M (2017) Comparison of time-frequency representations for environmental sound classification using convolutional neural networks. arxiv.org/abs/1706.07156
Khan NA, Jafri MN, Qazi SA (2011) Improved resolution short time Fourier transform. In: 7th International conference on emerging technologies
Khatamino P, Cantürk I, Özyılmaz L (2018) A deep learning—CNN based system for medical diagnosis: an application on Parkinson’s disease handwriting drawings. In: Proceedings of the 6th international conference control engineering information technology, pp 1–6
Marcelino P (2019) Transfer learning from pre-trained models. https://towardsdatascience.com/transfer-learning-from-pre-trained-models-f2393f124751. Accessed 11 Sept 2019
Moetesum M, Siddiqi I, Vincent N, Cloppet F (2019) Assessing visual attributes of handwriting for prediction of neurological disorders: a case study on Parkinson’s disease. Pattern Recognit Lett 121:19–27
Mormont R, Geurts P, Andmarée R (2018) Comparison of deep transfer learning strategies for digital pathology. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 2262–2271
Mucha J et al (2018) Identification and monitoring of Parkinson’s disease dysgraphia based on fractional-order derivatives of online handwriting. In: 41st International conference on telecommunications and signal processing, pp 1–4
Naumov E (2017) A convolutional network on EEG spectrograms for sleep staging. M.S.thesis, College of CS. McGill Univ., Montreal
Nilashi M, Ibrahim O, Ahani A (2016) Accuracy improvement for predicting Parkinson’s. Dis Prog Sci Rep 6(1):1–18
Nisar S, Khan OU, Tariq M (2016) An efficient adaptive window size selection method for improving spectrogram visualization. Comput Intell Neurosci. https://doi.org/10.1155/2016/6172453
Ordóñez F, Roggen D (2016) Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1):115–115
Pretegiani E, Optican LM (2017) Eye movements in Parkinson’s disease and inherited Parkinsonian syndromes. Front Neurol 8:592
Pereira CR et al (2018) Handwritten dynamics assessment through convolutional neural networks: an application to Parkinson’s disease identification. Artif Intell Med 87:67–77
Sharma RK, Gupta AK (2015) Voice analysis for telediagnosis of Parkinson disease using artificial neural networks and support vector machines. Int J Intell Syst Appl 7(6):41–47
Taleb C, Likforman L, Khachab M, Mokbel C (2017) Feature selection for an improved Parkinson’s disease identification based on handwriting. In: 1st International workshop on arabic script analysis and recognition. ASAR, Nancy, France
Taleb C, Likforman L, Khachab M, Mokbel C (2019) Visual representation of online handwriting time series for deep learning Parkinson’s disease detection. In: 3rd International workshop on arabic and derived script analysis and recognition, ASAR@ICDAR 2019. Sydney, Australia
Um T et al (2017) Data augmentation of wearable sensor data for Parkinson’s disease monitoring using convolutional neural networks. In: Proceedings of the 19th ACM international conference on multimodal interaction-ICMI
Wang F, Zhong S, Peng J, Jiang J, Liu Y (2018) Data augmentation for EEG-based emotion recognition with deep convolutional neural networks. MultiMedia Model Lect Notes Comput Sci. https://doi.org/10.1007/978-3-319-73600-6_8
Zeiler DM, Fergus R (2013) Visualizing and understanding convolutional networks. Comput Vision – ECCV 2014. Lect Notes Comput Sci. https://doi.org/10.1007/978-3-319-10590-1_53
Acknowledgements
This study has been approved by the institutional review board (IRB) of the University of Balamand and Saint George Hospital University Medical Center.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Taleb, C., Likforman-Sulem, L., Mokbel, C. et al. Detection of Parkinson’s disease from handwriting using deep learning: a comparative study. Evol. Intel. 16, 1813–1824 (2023). https://doi.org/10.1007/s12065-020-00470-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12065-020-00470-0