Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques

Highlights

• STFT can be used as a single informative feature for movement identification.

• Large training data reveals high classification rates for single subject setting.

• Diffusion Maps outperformed PCA in case of limited training data.

• The alignment algorithm enables robust classification of a new subject's data.

• The proposed alignment algorithm can be adopted for many EMG classification tasks.

Abstract

Surface electromyography (EMG) is non-invasive signal acquisition technique that plays a central role in many application, including clinical diagnostics, control for prosthetic devices and for human-machine interactions. The processing typically begins with a feature extraction step, which may be followed by the application of a dimensionality reduction technique. The obtained reduced features are input for a machine learning classifier. The constructed machine learning model may then classify new recorded movements.

The features extracted for EMG signals usually capture information both from the time and from the frequency domain. Short time Fourier transform (STFT) is commonly used for signal processing and in particular for EMG processing since it captures the temporal and the frequency characteristics of the data. Since the number of calculated STFT features is large, a common approach in signal processing and machine learning applications is to apply a linear or a nonlinear dimensionality reduction technique for simplifying the feature space. Another aspect that arises in medical applications in general and in EMG based hand classification in particular, is the large variability between subjects. Due to this variability, many studies focus on single subject classification. This requires acquiring a large training set for each tested participant which is not practical in real life application.

The objectives of this study were first to compare between the performances of a nonlinear dimensionality technique to a standard linear dimensionality method when applied for single subject EMG based hand movement classification, and to examined their performances in case of limited amount of training data samples. The second objective was to propose an algorithm for multi-subjects classification that utilized a data alignment step for overcoming the large variability between subjects.

The data set included EMG signals from 5 subjects who perform 6 different hand movements. STFT was calculated for feature extraction, principal component analysis (PCA) and diffusion maps (DM) were compared for dimension reductions. An affine transformation for aligning between the reduced feature spaces of two subjects, was investigated. K-nearest neighbors (KNN) was used for single and multi-subject classification.

The results of this study clearly show that the DM outperformed the PCA in case of limited training data. In addition, the multi-subject classification approach, which utilizes dimension reduction methods along with an alignment algorithm enable robust classification of a new subject based on another subjects’ data sets. The proposed framework is general and can be adopted for many EMG classification task.

Introduction

Machine learning is integrated nowadays in many data driven application, such as signal and image processing tasks, due to its generic and flexible methodologies. The rapid development of advanced and sophisticated signal acquisition tools, which results with large amounts of complex data, gave rise to new branches in machine learning that deal with compact modeling of these data sets. Coupling dimensional reduction methods with time series processing is a natural match and recent work in diverse fields (Dov, Talmon, & Cohen, 2016; Dsilva, Talmon, Coifman, & Kevrekidis, 2018; Duncan, Talmon, Zaveri, & Coifman, 2013; Rabin, Bregman, Lindenbaum, Ben-Horin, & Averbuch, 2016; Talmon, Cohen, Gannot, & Coifman, 2013) justify this claim. Algorithms that utilize machine learning for signal processing typically include a number of consecutive steps. First, features are extracted from the time series signal. Next, when the dimension of the feature vector is large, a dimensionality reduction method is used to construct a compact representation of the dataset. Last, a supervised or an unsupervised method is applied to predict the class of a new recorded signal.

The use of electromyography (EMG) signal allows convenience and non-invasive access signal acquisition resulting in new developed methods for classification and feature extraction. Identification of hand movements based on EMG measurements have been largely used in the field of computer and automatic video games, robotic exoskeleton, operative devices and for power prostheses (Batzianoulis et al., 2017; Gailey, Artemiadis, & Santello, 2017; Na, Kim, Jo, & Kim, 2017) and thus, has been the subject of many studies over the past few years.

A large number of these studies focus in features selection for EMG movement classifications and included in their methods is a dimensionality reduction step followed by a machine learning based classifier. The dimensionality reduction step is typically performed by a linear dimensionality reduction technique such as Principal Component Analysis (PCA), which is limited in representing complex data that has in it nonlinear relationships. Nonlinear dimensionality techniques enable to preserve the local structures in the original feature space and by that overcome some of PCA's limitations, due to the method's global and linear nature. However, while the extension to new data points in the linear dimensionality reduction methods is straightforward, methods based on the Nyström technique (Coifman and Lafon, 2006b) are used for the extension of nonlinear models.

Selection of significant discriminative features for EMG hand movement classification is discussed in several recent papers. Phinyomark et al. (2013) computed 50 features from EMG data that was measured in a period of 21 days from a single male subject, to identify a feature set that is stable over a long period of time. Using a linear discriminate analysis classification algorithm it was found that the sample entropy was the most stable feature. In another study (Tsai, Luh, & Lin, 2015) it was found that the Short time Fourier transform (STFT) ranking feature gives higher classification rates (93.9%) compared with conventional EMG features as mean absolute value, zero crossing, slope sign change, waveform length, auto regression, median and mean spectrum frequency (33–90.8%) for motion pattern recognition from multi-channel EMG signals of 6 muscles. The linear PCA method was applied in this work for dimensionality reduction and the Support Vector Machines (SVM) for the classification. Other studies investigate the best features selection for pattern recognition classification of myoelectric limb prostheses (Al-Angari, Kanitz, Tarantino, & Cipriani, 2016; Khushaba, Al-Timemy, Kodagoda, & Nazarpour, 2016). Khushaba et al. (2016) studied the effect of both forearm orientation and muscle contraction on the classification accuracy using wavelet transform based features and time and frequency domain features. Al-Angari et al. (2016) investigate the best pairs of features and channels for the classification of 5 hand postures at 9 different arm positions. Using 10 features from the time and frequency domain two methods for feature-channels pairs selection were compared, distance based and correlation-based feature selection. In both studies the results were evaluated for each subject separately, based only on his/her train data. The performance of a convolutional neural network classifier was compared with two other classifiers (linear discriminant analysis and stacked sparse autoencoders) for the classification of eight hand motions that were repeatedly performed by each of seven subjects. Features from the time domain were computed and fed as inputs to the classifiers and the results, in line with the previous all above works were evaluated for each subject separately.

Several studies have also been conducted over the years to investigate the ability to identify hand movements using different time and frequency domain EMG features. These EMG features were used in a study aimed to identify six daily hand movements (Sapsanis, Georgoulas, & Tzes, 2013). Two dimensionality reduction techniques, PCA (Pearson, 1901) and RELIEF (Kira et al., 1992) were used in this study to reduce the dimension of the feature space. Then, the classification was carried out on each subject separately, where 50%–70% of the data were used as train points. The reduced input was then inserted into a linear classifier and resulted with an average of 85% success classification rate without significant difference between the two methods. The dataset used by Sapsanis et al. (2013), which is publicly available in the UCI repository (UCI datasets), was further tested in several recent studies (Gu, Zhang, Zhao, & Luo, 2017; Ramírez-Martínez, Alfaro-Ponce, Pogrebnyak, Aldape-Pérez, & Argüelles-Cruz, 2019; Ruangpaisarn & Jaiyen, 2015). Gu et al. (2017) used the above dataset to compare between several machines learning algorithms for classification after calculating featured with Empirical Mode Decomposition (EDM). The compared classification methods were Neural Network, Adaptive Boosting, Linear Discriminant Analysis, Random Forest and Random Forest with PCA. Using 80% of the data from all the subjects as the training set, the Neural Network classifier resulted in accuracy of 85% while the Adaptive Boosting and the Linear Discriminant Analysis achieved only a 55% and 65% correct classification rate, respectively. An increase in the accuracy rates to 91% and 94% was achieved with the Random Forest and Random Forest with PCA, correspondingly.

In Ruangpaisarn and Jaiyen (2015), where Singular Value Decomposition (SVD) was applied for feature extraction, Naive Bayes, Radial Basis Function Network, k-nearest neighbors (KNN) and SVM trained by Sequential minimal optimization (SMO) (Platt, 1999), were compared for classification. The classification step was performed on each subject separately achieving accuracies of 91.66%, 94%, 94.77% and 98.22%, correspondingly. More recently, Ramírez-Martínez et al. (2019) examined the use of Burg reflection coefficients and tested many combinations of different feature-based datasets with machine learning algorithms for classification of the UCI hands movement dataset. A ten-fold cross-validation procedure was applied to the full dataset that held all of the subject's data. Results for a specific setting of an instance based classifier reached the accuracy of 100%, using 60 features. The results degraded a bit when the number of features was reduced.

Several other public and not-public dataset were tested in several papers, with the task being gesture or hand movement classification. In (Tang, Liu, Lv, & Sun, 2012), the authors collected EMG signals from multiple channels in order to discriminate between eleven hand gestures. In this study the dimensionality reduction step was bypassed by extracting only a small number of informative EMG features such as energy ratios and concordance correlation coefficient. A cascaded-classifier that was based on 25 train movements of each type was applied on each subject separately to the feature set and resulted in an accuracy rate of 89%. A real-time EMG recognition algorithm for the control of multifunction myoelectric hand was developed based on wavelets for feature extraction (Chu, Moon, & Mun, 2006). PCA followed by Self Organization Maps and a multi-layer neural network was used for dimensionality reduction while using data from the same subject for the train and for the test. Discrete Wavelet feature along with Artificial Neural Network classifier was applied in Mane, Kambli, Kazi, & Singh, 2015, and was optimized to each of four subjects separately that performed three basic hand movements, with an overall correct classification rate of 93%. A PCA based algorithm was also used to drive an under actuated prosthetic hand prototype having a two dimensional control input (Matrone, Cipriani, Secco, Magenes, & Carrozza, 2010) and for classification of 52 hand movements using an individual classifier that was constructed for each subject (Isaković, Miljković, & Popović, 2014).

In all of the above previous studies the computed classification algorithms, as far as we know, were tested on each subject separately, and were usually based on a relatively large number of training samples. The presented classification results in these works represent the average correct classification rates from all of the subjects. In other words, data fusion for merging between several subjects, is not performed. In order to achieve high classification rates for a single subject classification algorithm, one is required to record a large amount of training examples. In real life application, this request is not practical. Constructing a fused multi-subject representation is challenging. The variance between the signals recorded from two subjects that perform the same set of hand movements is large due to the different physical characteristics of the subjects. Thus, a simple step of merging between training data of different subjects is bound to fail. Recently, deep-learning methods were applied for creating multi-subject models.

In Côté-Allard et al. (2019), a deep-learning algorithm, which uses transfer learning, was applied for automatic hand movement classification. Time and frequency information was computed in the features extraction step. Linear Discriminant Analysis was tested for dimensionality reduction. Two datasets, which comprise of 19 and 17 participants were used as input for the deep learning method. Transfer learning implemented the data fusion between different subjects.

Other deep learning based methods were proposed for multi-subject classification (Phinyomark & Scheme, 2018; Tsinganos, Cornelis, Cornelis, Jansen, & Skodras, 2018). These methods typically compute STFT features (similarly to what is proposed in this paper), and treat them as input images for Convolutional Neural Networks.

Although deep learning based models, and in particular those which include transfer learning, are suitable for fusing samples recorded from different subjects, large amounts of data is required for the model construction. This study proposes a different approach, which is also suited for training data of limited size, and is easy to implement (when compared to deep learning), for constructing unified, multi-subject models. Implementation of the proposed model is done by solving a least square optimization problem, which is a common computational task in many numerical analysis and machine learning algorithms.

The goal of this paper was, therefore, twofold; to compare between the performances of nonlinear dimensionality techniques to standard linear dimensionality methods for single subject EMG hand movement classification. In particular, to examine the classification rates of a PCA and a diffusion maps (DM) based algorithm as the size of the training set decreases. Second, to propose an algorithm, which utilized data alignment, for classification of one subject's hand movements based on other subjects’ hand movements based on limited size of training data. This fusion procedure aimed to bypass the variance between subjects, has rarely been studied or addressed in most of the papers in this field.