Extracting a diagnostic gait signature

Abstract

This research addresses the question of the existence of prominent diagnostic signatures for human walking extracted from kinematics gait data. The proposed method is based on transforming the joint motion trajectories using wavelets to extract spatio-temporal features which are then fed as input to a vector quantiser; a self-organising map for classification of walking patterns of individuals with and without pathology. We show that our proposed algorithm is successful in extracting features that successfully discriminate between individuals with and without locomotion impairment.

Introduction

Normal walking in humans may be defined as a method of locomotion involving the use of two legs, alternately, to provide both support and propulsion, with at least one foot in contact with the ground at all times. Walking is a periodic process and gait describes the manner or the style of walking—rather than the walking process itself [1]. Fig. 1 illustrates the repetitive events of gait. The stance phase starts by heel strike (or foot contact with ground) passes through midstance and ends by taking the toe off the ground to start the swing phase. The time interval between two successive occurrences of one of the repetitive events of walking is known as the gait cycle and it is usually defined between two consecutive heel-strikes of the same foot. One characteristic phase of walking is the double support interval, i.e. when both feet are in contact with the ground. This time interval decreases as the velocity of the subject increases until it vanishes; the subject is then considered to be running.

The development of photographic methods of recording a series of displacements during locomotion by the end of the 19th century encouraged researchers from different disciplines to study human motion. The images were so useful as studies of the human form in motion that the noted poet and physician Oliver Wendell Holmes, who was interested in providing artificial limbs for veterans of the American Civil War, proclaimed that it was photography which assisted him in the study of the “complex act” of walking [2]. Experiments of the American photographer Eadweard Muybridge of photographing animals (e.g. horse during trotting) and humans in motion (e.g. athletes while practising a sport) perfected the study of animal and human locomotion [3]. Early experiments to study human locomotion were done by Marey, the first scientist in Europe to study motion and its visual implications [4]. Marey observed an actor dressed in a black body stocking with white strips on his limbs. He studied the motion through observing the traces left on photographic plates as the actor walked laterally across the field of view of the camera [5]. Later, at the end of the century, two German scientists—Braune and Fischer—used a similar approach to study human motion [6], but they used light rods attached to the actor's limbs instead.

Following those pioneers, lots of researchers from different disciplines studied human locomotion. Since the early seventies of the last century, biomechanics researchers have used a technique similar to the ones used by Marey for gait analysis and assessment. In Ref. [7] a method is described for measurement of gait movement from a motion picture film where three cameras were placed such that two are on the sides and one is at the front of a walkway—and a barefooted subject walked across the walkway. Measurements of the flexion/extension of knee and ankle in the sagittal plane and rotation of the pelvis, femur and foot in the transverse plane were measured with this system which had the advantage of being painless and did not involve any encumbering apparatus attached to the patient. In Ref. [8] a television/computer system is designed to estimate the spatial coordinates of markers attached to a subject indicating anatomical landmarks. The system was designed and tested for human locomotion analysis. Another attempt at kinematic analysis using a video camera, frame grabbers and a PC was proposed in Ref. [9]. The approach was based on tracking passive markers attached on specific body landmarks and the results were depicted as an animated stick diagram as well as graphs of joints’ flexion/extension. Interest of researchers was not confined to patterns of normal subjects, it extended to the study of the pathological gait [10], [11].

The method of using markers attached to joints or points of interest of a subject or an articulated object is similar to what is known in the literature of motion perception as Moving light displays $(\text{<mi mathvariant="italic" is="true">MLDs</mi>})$. In Refs. [12], [13], Johansson used MLDs in psychophysical experiments to show that humans can recognise gaits representing different activities such as walking, stair climbing, etc. when watching a sequence of frames of subjects having lights attached to them (sometimes referred to in the literature as Johansson's figures). One experiment had a sequence of 36 motion-picture frames in which two persons were dancing together with 12 lights attached to each one: two at each shoulder, elbow, wrist, hip, knee and ankle. He reported that “naïve” subjects, when shown the sequence, were able to recognise in a fraction of a second that two persons were moving. However, they were not able to identify what a single stationary frame represented. Cutting and Koslowski also showed that using MLDs, one can recognise one's friends [14] and can also determine the gender of a walker [15].

Human motion analysis is a multidisciplinary field which attracts the attention of a wide range of researchers [16]. The nature of the motion analysis research is dictated by the underlying application [17], [18], [19], [20], [21], [22], [23], [24].

Motion trajectories are the most widely used features in motion analysis. Most of human motion is periodic, as reflected in changes in joint angle and vertical displacement trajectories, functions involving motion are represented using transformation representing the spatio-temporal characteristics of these trajectories [20] or the volume [25]. In Ref. [26], a computer program that generated absolute motion variables of the human gait from predetermined relative motions was described. Kinematics data during free and forced-speed walking were collected and trajectories were analysed using fast Fourier transform (FFT). It was found that the spectrum of the variables was concentrated in the low frequency range while high frequencies components (above the 15th harmonic) resembled those of white noise.

FFT analysis was also used in Ref. [27]. FFT components of joint displacement trajectories were used as feature vectors to recognise people from their gait. In Ref. [28] the medial axis transformation was used to extract a stick figure model to simulate the lower extremities of the human body under certain conditions. Three male subjects were involved in the study where their 3D kinematic data were averaged to derive a reference sequence for the stick figure model. Two segments of the lower limb (thigh and shank) were modelled and the model was valid only for subjects walking parallel to the image plane. Factors affecting kinematics patterns were explored by studying subjects walking with bare feet and high heels, with folded arms and with arms freely swinging. It was concluded that there was almost no difference in the kinematics patterns. Eigenspace representation was used in Refs. [29], [30], [31]. This representation reduced computation of correlation-based comparison between image sequences. In Ref. [29], the proposed template-matching technique was applied for lip reading and gait analysis. The technique was useful in recognising different human gait. In Ref. [30] a combination of eigenspace transformation and canonical space transformation was used to extract features to recognise six people from their gait.

The majority of systems implemented for understanding human motion focus on learning, annotation or recognition of a subject or a human movement or activity.

For recognising activities, independent of the actor, it is necessary to define a set of unique features that would identify the activity (from other activities) successfully. The authors in Refs. [32], [33], [34] presented a general non-structural method for detecting and localising periodic activities such as walking, running, etc. from low-level grey scale image sequences. In their approach, a periodicity measure is defined and associated with the object tracked or the activity in the scene. This measure determined whether or not there was any periodic activity in the scene. A feature vector extracted from a spatio-temporal motion magnitude template was classified by comparing it to reference templates of predefined activity sets. The algorithm tracked a particular subject in a sequence where two subjects were moving. One nice feature of their algorithm was accounting for spatial scale changes in frames, so it was not restricted to motion parallel to the plane of the image. On the other hand, the effect of changing the viewing angle was not addressed in their work. In Ref. [35] a three-level framework for recognition of activities was described in which probabilistic mixture models for segmentation from low-level cluttered video sequences were initially used. Computing spatio-temporal gradient, colour similarity and spatial proximity for blobs representing limbs, a hidden Markov model (HMM) was trained for recognising different activities. The training sequences were either tracked MLD sequences or were hand-labelled joints. The Kalman filter used for tracking coped with short occlusions yet some occlusions resulted in misclassification of activity. Sequences seemed to have only one subject in the scene moving parallel to the image plane. In Ref. [36] features from displacements of the body parts in the vertical and horizontal directions were extracted and a classifier based on HMM was used to identify different activities (walking, hopping, running and limping).

A different perspective for recognising activities was portrayed in Ref. [37] by Johansson. His approach focused on high level representations through modelling human recognition of MLDs. He showed that recognition of gait can be achieved through multiresolution feature hierarchies extracted from motion rather than shape information. He applied his approach to recognise three gaits (walking, running and skipping) performed by four different subjects. A similar approach was used in Ref. [38] where techniques based on space curves were developed assuming the availability of 3D Cartesian tracking data to represent movements of ballet dancers. The system learned and recognised nine movements from an un-segmented stream of motion. The idea was based on representing each movement with a set of unique constraints which were extracted from a phase-space that related the independent variables of the body motion. A potential application given by the authors was video annotation for the ever increasing video databases in, for example, entertainment companies and sports teams.

For recognising individuals, examples of research attempts to develop systems to recognise individuals from their gait have been previously discussed. One other attempt [39] computed the optical flow for an image sequence of a walking person and characterised the shape of the motion with a set of sinusoidally varying scalars. Extracting feature vectors composed of the phases of the sinusoids which have shown significant statistical variation, the system was able to discriminate among five subjects. The representation is model-free and only considered subjects walking across the field of view of a stationary camera.

Following the above and other studies, gait has been considered as a biometric for individual authentication [40], [41], [42]. The idea of the existence of human gait signatures has been widely accepted, e.g. Refs. [43], [44]. In this article, we extend this idea to explore the existence of clinical diagnostic signatures from gait data.

Normal walking depends on a continual interchange between mobility and stability. Free passive mobility and appropriate muscle action are basic constituents. Any abnormality restricting the normal free mobility of a joint or altering either the timing or intensity of muscle action creates an abnormal gait. Abnormal gait may be due to an injury, disease, pain or problems of motor control. The subject's ability to compensate for the abnormality determines the amount of functionality retained.

However, when these compensations introduce penalties in joint strain, muscle overuse, lack of normal muscle growth or soft tissue contracture, then clinical intervention becomes a necessity. In determining appropriate intervention, gait analysis is used to identify gait defects.

Clinical gait analysis comprises visual assessment, measurement of stride and temporal parameters such as stance, cadence and walking velocity, kinematics dealing with the analysis of joint movements, angles of rotations, etc. and kinetics involving analysis of forces and moments acting on joints and electromyography (EMG) measuring muscle activity [1]. Gait analysis mainly is to document deviations from normal pattern (deviations might be due to habit, pathological reasons or old age), to determine the abnormalities, their severity and their causes, to plan for future treatment which might involve surgery, physiotherapy, or the use of braces, orthosis or any other walking aid, to evaluate the effect of intervention, and finally to measure and assess the change over time with treatment. Gait analysis instrumentation based on infra-red (IR) cameras and computer-aided systems recording the 3D positions of markers attached to the subject has been used to record gait cycles of the patient and produce patterns and plots for clinicians to assess and hence diagnose. To measure kinematics, the subject is filmed while walking using cameras placed on both sides and in front of the walkway such that each marker is seen by at least two cameras at any instant. For kinetics measurements, the subject activates force plates embedded in the walkway. The output of the cameras and the force plates is fed to the computer which estimates the 3D coordinates of the markers and the moments and torques applied on each joint. Kinematics, kinetics measurements and movement trajectories of the joints in the three different planes of movement are plotted for each patient. The pathological traces are plotted together with normal ones to show the variations resulting from the impairment and at the upper left corner of the figure, different gait parameters, e.g. cadence, velocity, etc. are also estimated for the patient. These graphs are then assessed by the specialists. The number of graphs plotted for gait analysis is immense and extracting useful information from such graphs to accomplish the gait analysis objectives mentioned earlier is a demanding and challenging task. This is due to various reasons that include the complexity of the walking process itself, variability of patients’ response to treatment, uncertainty in data quality and the difficulty in distinguishing between primary abnormalities and compensations in the gait pattern.

Gait interpretation involves evaluation of all measurements of kinematics, kinetics and EMG to identify abnormalities in gait and hence suggesting and assessing treatment alternatives. The experience of the clinicians’ team is the key element for a successful interpretation and this must include the understanding of normal gait, efficient and rigorous data collection and adequate data reduction [45]. Early studies by Murray [46], [47] aimed to establish ranges of normal values for normal human walking for both men and women from kinematics data analysis. Her studies involved 60 men of 20–65 years of age and 30 women of 20–80 years of age. The main aim of the study was to provide standards concerning speed, stride dimensions as well as angular and linear displacement of the trunk and extremities with which abnormal gait patterns could be compared. Moreover, the study went further trying to find correlations between different parameters, e.g. height and gait parameters; age and displacement patterns.

Developing automatic systems for clinical gait analysis provides objective analysis and interpretation of gait signals. In developing an automatic system for analysis and recognition of gait signals, signal processing not only forms a key element in the analysis extraction and interpretation of information but also plays an important role in the dimensionality reduction [41]. Some artificial intelligence (AI) methods, as artificial neural networks, due to their inherent abilities of generalisation, interpolation and fault tolerance offer means to assist in dealing with the challenge of processing huge amounts of data, classifying it through extracting generic diagnostic signatures. A review of the use of these techniques in analysing clinical gait data can be found in Refs. [48], [49]. Moreover, psychophysical experiments carried out by Johansson [12], [13] and others showed that humans can recognise activities from a sequence of frames containing only points corresponding to specific body landmarks of a subject.

The research presented in this article is motivated by the capabilities of humans to perceive gait from reduced spatio-temporal trajectories, and attempts to give machines a similar gait perception capability. It builds upon Murray's ideas of setting standards for normal walking and investigates the existence of diagnostic signatures that can be extracted from kinematics-based features for both normal and pathological subjects. Our objective is to automatically find salient features within trajectories of locomotion from which normal gait standards could be set. Similarly, for abnormal gait, those features could be used for diagnosis of abnormal walking or for establishing relationships among spatio-temporal parameters, gaits and impairment. The long term objective of this work is to provide clinicians with tools for data reduction, feature extraction and gait profile understanding.