Online myoelectric pattern recognition based on hybrid spatial features
Introduction
Myoelectric pattern recognition (MPR) has been used for different applications such as upper limb prostheses, exoskeletons, and rehabilitation robots [[1], [2], [3]]. Controlling the prostheses can expand the abilities of amputees that suffer from physical or cognitive failures to perform their daily living activity. However, many amputees refuse the use of commercial prosthesis due to several reasons, either non-intuitive control, incomplete functionality or unsatisfactory performance in real life environments [4].
Electromyography (EMG) signals are recorded from electrical activities of a muscle by means of surface, needle or implantable electrodes. EMG signals are widely used for controlling the prosthesis and in the identification of human's motion intent. They have higher signal to noise ratio than the brain recordings (i.e. Electroencephalogram (EEG)). Brain and nerve recordings usually require an invasive procedure for placing the electrodes, which limits their applications in a laboratory environment. However, EMG signals are prone to be faulty due to fatigue. Some researchers have suggested using a hybrid brain computer interface (BCI) system consisting of EEG and EMG signals to achieve a reliable device for controlling the prosthesis [5,6].
The system that uses surface EMG signals to control the prostheses is called myoelectric control system. Myoelectric control systems can be categorized into: conventional control systems and pattern recognition systems. The conventional control systems are limited to proportional control. These systems allow easy real time implementation, intuitive control but at the cost of their limited degree of freedom functions (DOF) (i.e. joints' movements). Pattern recognition techniques overcome the disadvantages of the conventional proportional control. They control more DOF functions intuitively (i.e. dexterity prosthetic) [[7], [8], [9]]. In the last decades, pattern recognition techniques enticed the attention of researchers for controlling the prosthetics hand [[10], [11], [12], [13]]. Using appropriate features and suitable classifier algorithms have influenced the classification performance. Two methods are used for recording EMG signals, either using sparse multichannel in which electrodes are placed precisely over a specific muscle [14,15], or electrodes are arranged in two dimensional array with closely spaced electrode to cover a specific area of muscle (i.e. known as HD-sEMG signals) [16,17]. HD-sEMG signals allow the extraction of the spatial features. The spatial distribution of channels has lots of information about muscle. So, spatial features are considered as valuable features in gestures recognition. Many myoelectric pattern recognition researches achieve high accuracy for offline experiments under lab conditions; however, their performance in real-life environments is not as accurate and authoritative. This causes amputees to abandon their prostheses [18]. Many studies show that there is no engagement between improving offline performance and online performance [19,20]. This confirms the necessity of evaluating the myoelectric prosthesis in a real-time environment. The disparity of subjects and environments cause the variations of EMG signals characteristics' over time, which in turn affect the performance of myoelectric pattern recognition and system robustness. Some of these factors are attributed to physiological reasons like muscle fatigue, muscle atrophy and physical changes like electrode shift and electrode conductivity [21].
Section snippets
Related work
Myoelectric pattern recognition algorithms are efficient for controlling upper limb prostheses. Pattern recognition approaches are responsible for sampling EMG signals, extracting appropriate features, then classifying them into a set of commands. Many high performance classifiers are employed for gesture recognition such as hidden Markov model [22], support vector machines (SVM) [23,24], artificial neural networks (ANN) [25], linear discriminant analysis (LDA) [26], and recently deep learning [
Myoelectric Pattern Recognition (MPR)
MPR is the control system that uses EMG signals to control the prosthesis. HD-sEMG signals are recorded from electrical activities of a wide area of a muscle. HD-sEMG signals increased the spatial information of muscle activity. Accordingly, HD-sEMG data can be analysed in both the temporal and spatial domains. This gives the possibility for using image-processing technique [41]. HD-sEMG data are analysed either as instantaneous image [42] or as activation map [43]. Several features are
Intra-session assessment
In intra-session, the classifier is trained on a part of data during one session and tested on the remaining part of the data at the same session. DB-a are used for training the SVM classifier using 50 % of samples as training set and the remaining half are used as testing set for each subject. The performance of SVM classifier based on three feature sets (i.e. H, HI and AIH features) are displayed in Fig. 5, Fig. 6, Fig. 7 to classify eight gestures. The classification performance is
Discussion
In real life setting, training the classifier efficiently allows an amputee to recognize gestures at a specific time. However, the classification of gestures cannot be accommodated over time. The classifier performance is degraded over time. The temporal changes of EMG signals cause a major challenge that constrains the commercialization of upper limb devices. Using robust spatial features and suitable myoelectric pattern recognition algorithm, motion can be decoded with outstandingly high
Conclusion
The EMG signals are stochastic and non-stationary signals. However, it is considered as an important input for controlling the myoelectric control system. The HD-sEMG signals augmented the spatial information of muscle activity by increasing the density and convergence of the electrodes. Two-dimensional electrodes array provides the ability to use EMG signals in both temporal and spatial domains. Three types of spatial features are proposed using histogram oriented gradient method and intensity
CRediT authorship contribution statement
Hanadi Abbas Jaber: Conceptualization, Formal analysis, Resources, Software, Visualization, Writing - original draft. Mofeed Turky Rashid: Conceptualization, Formal analysis, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing. Luigi Fortuna: Validation, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (51)
- et al.
"Brain-computer interfaces in medicine"
Mayo Clin. Proc.
(2012) - et al.
"Bidirectional interfaces with the peripheral nervous system"
Int. Rev. Neurobiol.
(2009) - et al.
Robust EMG pattern recognition in the presence of confounding factors: features, classifiers and adaptive learning
Expert Syst. Appl.
(2018) - et al.
A visual cognizance based multi-resolution descriptor for human action recognition using key pose
Int. J. Electron. Commun. (AEÜ)
(2019) - et al.
A training strategies to reduce classification degradation due to electrode displacements in pattern recognition based myoelectric control
Biomed. Signal Process. Control
(2008) - et al.
Robust EMG pattern recognition in the presence of confounding factors: features, classifiers and adaptive learning
Expert Syst. Appl.
(2018) - et al.
“Classification of simultaneous movements using surface EMG pattern recognition”
IEEE Trans. Biomed. Eng.
(2013) - et al.
“Exoskeleton technology in rehabilitation: Towards an EMG-based orthosis system for upper limb neuro motor rehabilitation”
J. Robotics
(2013) - et al.
“Review of control strategies for robotic movement training after neurologic injury”
J. Neuro. Eng. Rehab.
(2009) - et al.
“Adaptive artificial limbs, a real time approach to prediction and anticipation”
IEEE Robot. Automat. Mag.
(2013)
Spatial distribution of HD-EMG improves identification of task and force in patients with incomplete spinal cord injury
J. Neuroeng. Rehabil.
“Application of real time machine learning to myoelectric prostheses control: a case series in adaptive switching”
Prosthet. Orthot. Int.
Real-time EMG based pattern recognition control for and prostheses: a review on existing methods, challenges and future implementation
Sensors
Current trends and confounding factors in myoelectric control: limb position and contraction intensity
Sensors
Simple and intelligent system to recognize the expression of speech-disabled person
4th International Conference on Intelligent Human Computer Interaction (IHCI)
“Nonparametric feature matching based conditional random fields for gesture recognition from multi-modal video”
Adaptive windowing framework for surface Electromyography – based pattern recognition system for transradial amputees
Sensors
Reduced daily recalibration of myoelectric prostheses classifier based on domain adaptation
IEEE J. Biomed. Health Inform.
High-density surface EMG maps from upper –arm and forearm muscles
J. Neuroeng. Rehabil.
Pattern recognition based on HD-sEMG spatial features extraction for an efficient proportional control of a robotic arm
Biomed. Signal Process. Control
Is accurate mapping of EMG signals on kinematics needed for precise online myoelectric control
IEEE Trans. Neural Syst. Rehabil. Eng.
Offline accuracy: a potentially misleading metric in myoelectric pattern recognition for prosthetic control
Conf. Proc. IEEE Eng. Med. Biol. Soc.
Translating research on myoelectric control into clinics are the performance assessment methods adequate?
Front. Neurorobot.
Position-independent decoding of movement intention for proportional myoelectric interfaces
IEEE Trans. Neural Syst. Rehabil. Eng.
Cited by (11)
Descriptive Statistical Features-Based Improvement of Hand Gesture Identification
2024, Biomedical Signal Processing and ControlMyoelectric Model Reference Adaptive Control with Adaptive Kalman Filter for a soft elbow exoskeleton
2024, Control Engineering PracticeShort-term hand gestures recognition based on electromyography signals
2023, IAES International Journal of Artificial IntelligenceFeatures selection for estimating hand gestures based on electromyography signals
2023, Bulletin of Electrical Engineering and InformaticsAn Extended Spatial Transformer Convolutional Neural Network for Gesture Recognition and Self-Calibration Based on Sparse sEMG Electrodes
2022, IEEE Transactions on Biomedical Circuits and Systems