Online myoelectric pattern recognition based on hybrid spatial features

https://doi.org/10.1016/j.bspc.2021.102482Get rights and content

Highlights

  • Three types of spatial feature sets are proposed named as H, HI and AIH features. H features correspond to extracting HOG features by Histogram Oriented Gradient method from HD-sEMG map. HI features are obtained by concatenating H features and scalar intensity feature. Finally, The hybrid spatial features AIH are obtained by combining H features and the intensity features matrix (AI). AI features are proposed as a modification of I feature.

  • The integration of HD-sEMG electrodes technology and reliable spatial features can guarantee an efficient classification performance.

  • The feature sets are evaluated in both intra-session and inter-session evaluation. The online classification reports the robustness of these features to overcome the variability of EMG signals across time and between sessions with accurate classification performance.

  • Reducing the sampling rate to a certain extent without degrading the classifier’s performance shows the reliability of the proposed features.

Abstract

Although Electromyography (EMG) signals are sources of neural information that are essential in controlling the prosthetic hand, many confounding factors caused the variation of EMG signals properties over time. These factors degraded the performance of myoelectric prosthesis and made it unstable over time, across subjects and sessions such as stress, fatigue, muscular dystrophy, shifting electrodes locations, etc. The spatial information of muscle activity can be augmented using high-density surface electromyography (HD-sEMG) electrodes technology. In this study, HD-sEMG electrodes technology are integrated with the robust hybrid spatial features to improve the performance of myoelectric prostheses towards the non-stationary characteristics of EMG signals over time and across sessions. Three types of spatial feature sets are proposed using histogram oriented gradient (HOG) algorithm and intensity features. Three sub databases are used for evaluating the SVM classifier based on the proposed features. Intra-session and inter-session evaluation in offline manner show the potential of the proposed feature sets to improve the classification performance. The classification performance based on hybrid spatial features achieved precision of 97.9 %, sensitivity of 97.5 % for intra-session evaluation and a classification accuracy about 92.18 % for inter-session evaluation. Online classification results exhibit the robustness of hybrid spatial features (i.e. it achieved a classification accuracy based on hybrid spatial features of 92 % for intra-session evaluation and 89.9 % between sessions). Further, reducing the sampling rate to a certain extent without affecting the classification accuracy indicates the robustness and reliability of the proposed features. The results confirm that the robust spatial features have a significant effect on the classification accuracy more than that of the classifier algorithm.

Introduction

Myoelectric pattern recognition (MPR) has been used for different applications such as upper limb prostheses, exoskeletons, and rehabilitation robots [[1], [2], [3]]. Controlling the prostheses can expand the abilities of amputees that suffer from physical or cognitive failures to perform their daily living activity. However, many amputees refuse the use of commercial prosthesis due to several reasons, either non-intuitive control, incomplete functionality or unsatisfactory performance in real life environments [4].

Electromyography (EMG) signals are recorded from electrical activities of a muscle by means of surface, needle or implantable electrodes. EMG signals are widely used for controlling the prosthesis and in the identification of human's motion intent. They have higher signal to noise ratio than the brain recordings (i.e. Electroencephalogram (EEG)). Brain and nerve recordings usually require an invasive procedure for placing the electrodes, which limits their applications in a laboratory environment. However, EMG signals are prone to be faulty due to fatigue. Some researchers have suggested using a hybrid brain computer interface (BCI) system consisting of EEG and EMG signals to achieve a reliable device for controlling the prosthesis [5,6].

The system that uses surface EMG signals to control the prostheses is called myoelectric control system. Myoelectric control systems can be categorized into: conventional control systems and pattern recognition systems. The conventional control systems are limited to proportional control. These systems allow easy real time implementation, intuitive control but at the cost of their limited degree of freedom functions (DOF) (i.e. joints' movements). Pattern recognition techniques overcome the disadvantages of the conventional proportional control. They control more DOF functions intuitively (i.e. dexterity prosthetic) [[7], [8], [9]]. In the last decades, pattern recognition techniques enticed the attention of researchers for controlling the prosthetics hand [[10], [11], [12], [13]]. Using appropriate features and suitable classifier algorithms have influenced the classification performance. Two methods are used for recording EMG signals, either using sparse multichannel in which electrodes are placed precisely over a specific muscle [14,15], or electrodes are arranged in two dimensional array with closely spaced electrode to cover a specific area of muscle (i.e. known as HD-sEMG signals) [16,17]. HD-sEMG signals allow the extraction of the spatial features. The spatial distribution of channels has lots of information about muscle. So, spatial features are considered as valuable features in gestures recognition. Many myoelectric pattern recognition researches achieve high accuracy for offline experiments under lab conditions; however, their performance in real-life environments is not as accurate and authoritative. This causes amputees to abandon their prostheses [18]. Many studies show that there is no engagement between improving offline performance and online performance [19,20]. This confirms the necessity of evaluating the myoelectric prosthesis in a real-time environment. The disparity of subjects and environments cause the variations of EMG signals characteristics' over time, which in turn affect the performance of myoelectric pattern recognition and system robustness. Some of these factors are attributed to physiological reasons like muscle fatigue, muscle atrophy and physical changes like electrode shift and electrode conductivity [21].

Section snippets

Related work

Myoelectric pattern recognition algorithms are efficient for controlling upper limb prostheses. Pattern recognition approaches are responsible for sampling EMG signals, extracting appropriate features, then classifying them into a set of commands. Many high performance classifiers are employed for gesture recognition such as hidden Markov model [22], support vector machines (SVM) [23,24], artificial neural networks (ANN) [25], linear discriminant analysis (LDA) [26], and recently deep learning [

Myoelectric Pattern Recognition (MPR)

MPR is the control system that uses EMG signals to control the prosthesis. HD-sEMG signals are recorded from electrical activities of a wide area of a muscle. HD-sEMG signals increased the spatial information of muscle activity. Accordingly, HD-sEMG data can be analysed in both the temporal and spatial domains. This gives the possibility for using image-processing technique [41]. HD-sEMG data are analysed either as instantaneous image [42] or as activation map [43]. Several features are

Intra-session assessment

In intra-session, the classifier is trained on a part of data during one session and tested on the remaining part of the data at the same session. DB-a are used for training the SVM classifier using 50 % of samples as training set and the remaining half are used as testing set for each subject. The performance of SVM classifier based on three feature sets (i.e. H, HI and AIH features) are displayed in Fig. 5, Fig. 6, Fig. 7 to classify eight gestures. The classification performance is

Discussion

In real life setting, training the classifier efficiently allows an amputee to recognize gestures at a specific time. However, the classification of gestures cannot be accommodated over time. The classifier performance is degraded over time. The temporal changes of EMG signals cause a major challenge that constrains the commercialization of upper limb devices. Using robust spatial features and suitable myoelectric pattern recognition algorithm, motion can be decoded with outstandingly high

Conclusion

The EMG signals are stochastic and non-stationary signals. However, it is considered as an important input for controlling the myoelectric control system. The HD-sEMG signals augmented the spatial information of muscle activity by increasing the density and convergence of the electrodes. Two-dimensional electrodes array provides the ability to use EMG signals in both temporal and spatial domains. Three types of spatial features are proposed using histogram oriented gradient method and intensity

CRediT authorship contribution statement

Hanadi Abbas Jaber: Conceptualization, Formal analysis, Resources, Software, Visualization, Writing - original draft. Mofeed Turky Rashid: Conceptualization, Formal analysis, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing. Luigi Fortuna: Validation, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (51)

  • M. Jordanic et al.

    Spatial distribution of HD-EMG improves identification of task and force in patients with incomplete spinal cord injury

    J. Neuroeng. Rehabil.

    (2016)
  • A.L. Edwards et al.

    “Application of real time machine learning to myoelectric prostheses control: a case series in adaptive switching”

    Prosthet. Orthot. Int.

    (2015)
  • N. Parajuli et al.

    Real-time EMG based pattern recognition control for and prostheses: a review on existing methods, challenges and future implementation

    Sensors

    (2019)
  • E. Campbell et al.

    Current trends and confounding factors in myoelectric control: limb position and contraction intensity

    Sensors

    (2020)
  • D.K. Vishwakarma et al.

    Simple and intelligent system to recognize the expression of speech-disabled person

    4th International Conference on Intelligent Human Computer Interaction (IHCI)

    (2012)
  • J.Y. Chang

    “Nonparametric feature matching based conditional random fields for gesture recognition from multi-modal video”

    (2016)
  • D.K. Vishwakarma, R. Maheshwari and R. Kapoor, "An Efficient Approach for the Recognition of Hand Gestures from Very...
  • A.H. Al-Timemy et al.

    Adaptive windowing framework for surface Electromyography – based pattern recognition system for transradial amputees

    Sensors

    (2018)
  • J. Liu et al.

    Reduced daily recalibration of myoelectric prostheses classifier based on domain adaptation

    IEEE J. Biomed. Health Inform.

    (2016)
  • M. Rojas-Martínez et al.

    High-density surface EMG maps from upper –arm and forearm muscles

    J. Neuroeng. Rehabil.

    (2012)
  • F. Nougarou et al.

    Pattern recognition based on HD-sEMG spatial features extraction for an efficient proportional control of a robotic arm

    Biomed. Signal Process. Control

    (2019)
  • N. Jiang et al.

    Is accurate mapping of EMG signals on kinematics needed for precise online myoelectric control

    IEEE Trans. Neural Syst. Rehabil. Eng.

    (2014)
  • M. Ortiz-Catalan et al.

    Offline accuracy: a potentially misleading metric in myoelectric pattern recognition for prosthetic control

    Conf. Proc. IEEE Eng. Med. Biol. Soc.

    (2015)
  • I. Vujaklija et al.

    Translating research on myoelectric control into clinics are the performance assessment methods adequate?

    Front. Neurorobot.

    (2017)
  • K.H. Park et al.

    Position-independent decoding of movement intention for proportional myoelectric interfaces

    IEEE Trans. Neural Syst. Rehabil. Eng.

    (2016)
  • Cited by (11)

    • Short-term hand gestures recognition based on electromyography signals

      2023, IAES International Journal of Artificial Intelligence
    • Features selection for estimating hand gestures based on electromyography signals

      2023, Bulletin of Electrical Engineering and Informatics
    View all citing articles on Scopus
    View full text