Assessing visual attributes of handwriting for prediction of neurological disorders—A case study on Parkinson’s disease

Highlights

• Investigation of visual attributes of handwriting to predict Parkison’s Disease.

• Use of Convolutional Neural Networks for automatic feature extraction.

• Multiple representations of raw data to enhance feature extraction step.

• Evaluations on a standard template including drawing and writing tasks.

• Fusion of predictions from multiple tasks to enhance performance.

Abstract

Parkinson’s disease (PD) is a degenerative disorder that progressively affects the central nervous system causing muscle rigidity, tremors, slowed movements and impaired balance. Sophisticated diagnostic procedures like SPECT scans can detect changes in the brain caused by PD but are only effective once the disease has advanced considerably. Analysis of subtle variations in handwriting and speech can serve as potential tools for early prediction of the disease. While traditional techniques mostly rely on dynamic (kinematic and spatio-temporal) features of handwriting, in this study, we quantitatively evaluate the visual attributes in characterization of graphomotor samples of PD patients. For this purpose, Convolutional Neural Networks are employed to extract discriminating visual features from multiple representations of various graphomotor samples produced by both control and PD subjects. The extracted features are then fed to a Support Vector Machine (SVM) classifier. Evaluations are carried out on a dataset of 72 subjects using early and late fusion techniques and an overall accuracy of 83% is realized with solely visual information.

Introduction

Parkinson’s Disease (PD) is a neurodegenerative disorder that affects the coordinated movements of a person due to loss of dopamine producing neurons in substantia nigra [4]. According to studies [6], [71], it is one of the most prevalent neurological diseases after Alzheimer’s [69] with an average onset age of 60. Patients with PD experience symptoms like posture deformation, rigidity, tremors and vocal impairments, etc. Traditional diagnostic procedures for determination of the disease include costly, invasive methods like SPECT and CT scans, which are usually effective when the disease has already progressed to a mature stage. Clinical practitioners therefore, first opt for manual, non-invasive screening tests like Unified Parkinson’s Disease Rating Scale (UPDRS) [17], for early detection of the disease. While this process is quite established and has been modified over years of experience, it remains relatively subjective.

With the advent of technology, a number of computerized systems have been proposed to identify the early symptoms of Parkinson’s and similar neurological diseases. Some of these studies analyze voice or speech patterns to observe subtle but progressive changes which are indicative of PD [62], while others monitor muscular movements using wearable sensors [38]. Over the period of time, a substantial number of studies [10], [36], [63] have suggested that handwriting, a product of perceptive, cognitive and fine motor skills [3], [16], [61], can also be employed as an effective tool for early diagnosis of PD.

Currently most of the research [9], [25], [49], [65] focuses on analyzing the kinematic and pressure aspects of handwriting to determine PD. Although these dynamic features are effective, they mostly require additional temporal information for prediction which can only be acquired by utilizing special equipment like digitizer tablets and customized electronic pens [64]. Recent advancements in image analysis and pattern classification techniques have encouraged researchers to re-investigate static features from offline samples for improved detection of PD as well [70]. Hypothetically believing in the importance of visual features, we propose a novel method of assessing their contribution in characterization of PD. We designed an enhanced system that extracts useful visual features from handwriting and drawing samples of subjects and applies various fusion techniques to accurately discriminate between the control and PD groups. Feature learning based classification has previously been applied to sensor based handwriting movement signals [43] for detection of symptomatic signs of PD. Nevertheless, to the best of our knowledge, feature learning of visual attributes has not been explored to its full potential. The main contributions of the paper are listed in the following.

• Investigation of the effectiveness of visual attributes of handwriting by employing machine learning techniques, in characterizing PD, as opposed to the dynamic online attributes traditionally considered in the literature.

• Use of multiple representations of raw data as input to learn discriminative patterns from handwriting samples.

• Fusion of results from multiple samples acquired from various graphomotor (handwritten & hand drawn) tasks for improved overall classification.

The rest of the paper is organized as follows. Section 2 presents an overview of related works. Section 3 describes the proposed methodology and experimental setup. Section 4 summarizes the results and their analysis. Finally conclusion and future directions are discussed in Section 5.