Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier

https://doi.org/10.1016/j.bspc.2018.07.010Get rights and content

Highlights

  • Minimalist two channel surface EMG system for hand gesture detection is demonstrated.

  • Support vector machine classification can detect five hand gestures with two sEMG channels.

  • Overall classification rate of over 90% is demonstrated.

  • A gesture is defined for locking/unlocking of the system which adds to the robustness.

Abstract

Integration of surface EMG sensors as an input source for Human Machine Interfaces (HMIs) is getting an increasing attention due to their application in wearable devices such as armbands. For a wearable device, comfort and lightness are important factors. Therefore, in this article we focus on a minimalistic approach, in which we try to classify four gestures with only 2 EMG channels installed on the flexor and extensor muscles of the forearm. We adopted a two-channel EMG system, together with a high dimensional feature-space and a support vector machine (SVM) as a classifier. In addition, tolerance of the system for rejection of unsolicited gestures during the body movement was evaluated, and the two methods were implemented to ensure this; one based on an SVM threshold and another one based on the addition of a locking gesture. The resulting system is able to recognize up to 5 gestures (hand closing, hand opening, wrist flexion, wrist extension and double wrist flexion), presenting a classification accuracy of between 95% and 100% for a trained user and robustness against different body movements, guaranteed with the locking feature. We showed that misclassification of other gestures as the unlocking never happened for expert users.

Introduction

Surface EMGs have been mostly used for control of prosthetic hands. Full control of a highly articulated hands still requires a high number of EMG sensors, raising significantly the cost and complexity of the system. It is a common configuration to utilize four EMG channels, as adopted in [1] and [2], in order to be able to detect wrist and 4-fingers flexions and extensions (Fig. 1); other authors, like Yang [3] and Bugmann [4], used 6 bipolar electrodes for recognizing up to 19 and 15 hand movement respectively and controlling a highly dexterous hand. These control systems require a high number of control inputs. Some other studies reported application of eight [5] or ten [6] bipolar electrodes positioned on the forearm.

Other examples of utilizing surface EMG sensors include finger joint angle estimation using a 8 channel EMG system [7] and hand gesture recognition using a 6 channel EMG armband [8]. Also, in a recent study authors discussed the selection of best subsets of EMG electrode pairs for classification of hand movements when performing 5 hand postures at 9 different arm positions [9].

Surface EMG sensors that were previously exploited only for prosthetic devices, were considered as an important source of input for general purpose Human Machine Interfaces (HMIs) for wearable devices. For such devices, it is always interesting to received as many inputs as possible, and classify as many gestures as possible, to enrich the human control over the machine. BioSLeeve [10] is an example of a wearable device that implements 16 electrodes.

Recently, a gesture recognition armband, the Myo armband was commercialized [11]. As can be seen in Fig. 2, this armband is able to detect five gestures, which are open, wave out, wave in, fist and pinch. Myo armband embeds eight EMG channels, and is designed as a wearable armband. Yet, Myo is relatively bulky as a wearable device.

The purpose of this article is then to explore a minimalistic approach in order to achieve hand gestures. Therefore, we explore recognition of four hand gestures (open, close, wave in, wave out) with only two EMG channels.

While, the “Open” and “Close” gestures, or more generally two gestures recognition with a single EMG channel have been already explored [12], [13], adding the “wave in” and “wave out” gesture provides new possibilities for the user. The goals is then to build a real-time classification algorithm for recognizing these 4 gestures and measuring the success rate in real-time classification. Therefore the following constraints are fixed:

  • (1)

    The number of inputs is reduced to two, that is, the 4 necessary gestures must be recognized by using only two signals.

  • (2)

    The classifier must achieve good gestures recognition percentage, e.g. over 90%.

  • (3)

    The processing time for calibration must not exceed 30 s; moreover, the algorithm must be trainable with a small training set, asking so the user to perform a minimum number of gestures for the calibration.

  • (4)

    The system response to a gesture performing must not exceed 300 ms for user comfort.

  • (5)

    The classifier should allow to be used without re-training it in every session; even when the algorithm is not re-calibrated.

  • (6)

    The system must provide robustness against other limb movements, that is, the disturbances coming from these motions must not be classified as one of the three gestures, letting the user move freely when wearing the EMG device.

In the following chapters, we describe the system design, the features of the signal that were extracted and the classifier which was selected and implemented. We then show the result of the system implementation for 7 subjects (four males and three females) that tested the system. Subjects were all healthy, with an average age of 25.6 years (SD = 5.7), with different levels of previous acquaintance with the system. The last constraint (tolerance to disturbances) actually found to be the most challenging for some of the beginner users, which enforced integration of a fifth gesture, as an option, for locking the system. Tests were repeated for beginner subjects and the results are presented.

Section snippets

EMG system design and implementation

As described before, the goal of the design is to recognize hand gestures with satisfying reliability, for which we set at 90% the lower bound of correctly classified gestures. Moreover, we decided to use dry electrodes instead of gelled ones, to increase the comfort and durability of the system. Therefore we used stainless steel electrodes. The positioning of the two channels should follow the scheme shown in Fig. 3; however, we experienced during the test sessions that the optimal positioning

Test subjects

Seven volunteers were selected to test the system; two subjects that used the system previously several times, and at least for eight hours (here called expert user), two users who were allowed to try the system and see the signals for 2 h before the test(here called intermediate user), and the other three at their first usage(beginners). The subjects were 4 males and 3 females, all normally-limbed, with an average age of 25.6 years (SD = 5.7). Every test set was repeated three times for every

Classification accuracy results (First test)

Table 1, Table 2, Table 3, Table 4 report the results for the 4 subjects, differentiated for each of the three sessions. In each of them, the results of the both tests are reported, by building a proper confusion matrix.

As can be seen, in all cases the accuracy for all tests was always better than 82%. It can be also seen that the expert user had excellent results of classification, always better than 95% of accuracy. In fact, there were some errors due to noise, but no misclassification

Tolerance to noise and body movement (Second test)

Test 2 revealed that daily body movement would generate signals that could be interpreted as Gestures. In fact the expert user had very satisfactory results and had less than 3 misclassifications in each session. However the result from beginner users was not satisfactory. While it can be discussed that with training such results will improve, we decided to evaluate the possibility of adding a fifth gesture for locking/unlocking the system for two reasons:

  • First, because unwanted gesture

Discussion and conclusions

The overall results were satisfactory in terms of classification success, calibration time and session independence which were demonstrated. The real-time classification constraints were respected, since the above satisfying results were obtained with a small training set and feature space dimensions, which brought to a reasonable processing time for the calibration stage (<30 s) and a fast system response to a gesture performing (∼10 ms after completion of the gesture). In addition, we studied

Acknowledgements

This work was partially supported by the national funds of the Foundation of Science and Technology of Portugal through the CMU-Portugal project Stretchtronics (Nr. CMUP-ERI/TIC/0021/2014), and MATIS(CENTRO-01-0145-FEDER-000014) co-financed by the European Regional Development Fund (FEDER) through “Programa Operacional Regional do Centro” (CENTRO2020).

References (23)

  • C. Cipriani et al.

    Online myoelectric control of a dexterous hand prosthesis by transradial amputees

    IEEE Trans. Neural Syst. Rehabil. Eng.

    (2011)
  • Cited by (120)

    View all citing articles on Scopus
    View full text