Towards online myoelectric control based on muscle synergies-to-force mapping for robotic applications☆
Introduction
In the era of the human-centered design, myoelectric control is continuously gaining attention since it allows for a natural, intuitive and tailored human–machine interface. Myoelectric control, or myo-control, consists in decoding the human motor intention, through the analysis of electromyographic signals (EMG), and computing a set of control signals that drive the machine the human is interacting with. The scientific research in EMG-based control has been mainly driven by the need for more intuitive prosthetic devices [23]. However, myo-electric schemes have also showed promising results when applied in different scenarios, featuring the simultaneous control of multiple degrees of freedom (DoF), e.g. the control of orthoses and exoskeletons [8], [6], [7].
Although the neuro-muscular theory behind the EMG signals generation hypothesizes the existence of applicable schemes [14], still no robust and reliable solutions appeared among both scientific literature and commercial devices [1]. In the last decades, the biological inspired approaches have earned an increasing approval by the scientific community even though how the human central nervous system (CNS) copes with a complex neuromusculoskeletal system is not fully understood yet [12], [28].
The bio-inspired myo-control paradigms could be mainly categorized in model-based and synergies-based approaches. The former relies on the mathematical modeling of all physiological and mechanical processes that are involved in the human movement generation and it is usually used to accurately estimate the articulations torque at the expense of both a long lasting calibration phase and high computational cost [7]. The latter aims at minimizing the computational cost by mimicking the CNS through the identification of specific muscle activation patterns, also called muscle synergies, exploited during task-related movements [3].
In the last decade, some researchers have focused their attention on the development of muscle synergies-based approaches for the human motor activity detection, that represents the basis for an effective myoelectric control [17]. For the sake of brevity, the two most representative papers are reported and described.
Denise Berger et al. [2] demonstrated that muscle synergies represented a valid technique to continuously estimate isometric forces generated by the hand in real-time through the surface EMG acquired from upper limb muscles. The setup proposed by Berger et al. considered a single arm pose and isometric forces applied at the hand in multiple directions on the horizontal plane. The authors demonstrated that the proposed synergies-based myo-controller was able to ensure the same level of accuracy achieved using the force sensor during online tasks. Ning Jiang et al. [19] successfully developed a strategy for achieving an accurate simultaneous and proportional control of a 2-Degrees of Freedom (DoF) wrist prostheses by concurrently extracting synergies-based control signals from each independent DoF. However, they documented a substantial decrease of the performance when introducing the 3rd DoF. In both the above mentioned works, the proposed setups for the experimental validation were based on a single arm/wrist pose and considered isometric multi-DOF forces generated at the hand/wrist. It is worth noting that the two cited works successfully proposed synergies-based myo-controllers that are able to estimate the human motor intention during tasks involving few DoFs and a fixed arm pose. However, to the best of the authors’ knowledge, how such approaches might be extended for setups featuring a high number of limb poses has been not studied yet. In fact, the already proposed paradigms rely on a specific synergies-to-force mapping that has been computed in a pre-defined limb pose.
The research question faced in this paper is: do we need to compute a synergies-to-DOF mapping for each potential pose the limb might assume during the task? In particular, it is interesting understanding how to build new models that, after being trained in few fixed upper-limb poses, are able to estimate the force from EMG signals acquired in non-trained upper-limb poses (generalization capabilities). The authors hypothesize a potential usage of a linear interpolation on trained models, as a first and simple attempt to solve the previous mentioned problems. Moreover, since synergies represent a set of motor primitives, they might be suitable to be shared across different upper limb poses, eventually leading to interesting outcomes with respect to a full muscles-to-force mapping. In this work, the authors propose a method that, with the knowledge of the synergies-to-force mapping related to few limb poses, is able to compute the synergies-to-force mapping of a new upper limb pose through the interpolation of regression matrices. Upper-limb muscles activity and isometric hand forces have been acquired during virtual planar reaching tasks in different 3D points (or pose) of a large workspace (). Then, for each tested point in the workspace, the synergies-to-force mapping computed with the data acquired in that specific pose has been compared with the synergies-to-force mapping computed with the novel proposed method. More in detail, the proposed interpolation-based approach has been evaluated with three different kind of mappings: muscle-to-force, pose-shared synergies-to-force and pose-related synergies-to-force. The muscle-to-force mapping considers a direct map between muscles and hand force. Both the synergies-to-force methods consider a map between muscle synergies and hand force with the only difference that the “Pose-Shared” assumes that the muscle patterns can be factorized using data coming from different points, whereas the “Pose-Related” assumes that each point has its own set of muscle primitives that can be clustered together with the synergies extracted in the other points [5]. This work extends a previous paper [9] under three main points: (I) the workspace in this study is three times bigger than the previous one; (II) a larger number of subjects is involved and (III) it also introduces a preliminary online evaluation session.
Section snippets
Participants
Five right-handed healthy subjects (four males, aged years, weight kg) were involved in the study. At the moment of the experiment, they had not any previous experience with myo-electric controlled applications. All the experiments have been approved by the Ethical Review Board of Scuola Superiore Sant’Anna (Approval Number: 1292) and conducted following the World Medical Association (WMA) Declaration of Helsinki. All subjects signed the written consent form before joining the
Theory
In this section, all the developed algorithms are described from a mathematical point of view, highlighting the main difference in including or not muscle synergies in the estimation process. Moreover the basis for comparisons are stated, detailing how the performance indexes are computed. Referring to the training and the test phases in the Specific condition, the related EMG datasets, used for the force estimations process, involve either the first or the second contraction (see acquisition
Results
As reported in author’s previous works [9], [5], and in line with the works of D’Avella et al. [2], [13], the authors found that a small number of synergies, with respect to the number of acquired muscles, was sufficient for building a functional subset of grouped muscles. In this work, five synergies have been selected for reconstructing the muscle activations, as found to be enough for representing more than the of the variance of the EMG signals in a similar task [9], [5]. Regarding the
Discussion and conclusions
This work presents an extended study (with respect to a previous work [9]) on the generalization abilities of different myoelectric control methods in a large upper-limb workspace (). The aim of the study was building a myoelectric control scheme that, after being trained on a reduced set of points placed at the vertexes of a rectangle drawn on the horizontal plane, was able to estimate the force at the hand in untrained points placed inside the rectangle. The offline results obtained
CRediT authorship contribution statement
Cristian Camardella: Conceptualization, Methodology, Software, Validation, Visualization, Formal analysis, Investigation, Data curation, Writing - original draft, Writing - review & editing. Michele Barsotti: Conceptualization, Methodology, Validation, Visualization, Formal analysis, Investigation, Writing - original draft, Writing - review & editing. Domenico Buongiorno: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Writing - original draft, Writing - review &
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Cristian Camardella is a PhD Student from Scuola Superiore Sant’Anna in Perceptual Robotics. He received the bachelor and master degree in Automation Engineering from Polytechnic University of Bari in 2017. In 2019 he was a visiting student at the Chinese University of Hong Kong, in collaboration with Prof. Raymond Tong on the post-stroke muscle synergies analysis and rehabilitation robotics strategies development. His interests cover the development of human-machine interfaces, bio-signals
References (28)
- et al.
A human–robot interaction perspective on assistive and rehabilitation robotics
Front. Neurorobot.
(2017) - et al.
Effective force control by muscle synergies
Front. Comput. Neurosci.
(2014) - et al.
Book review: modular organization of spinal motor systems
Neuroscientist
(2002) - et al.
Neuromusculoskeletal modeling: estimation of muscle forces and joint moments and movements from measurements of neural command
J. Appl. Biomech.
(2004) - et al.
Evaluation of a pose-shared synergy-based isometric model for hand force estimation: towards myocontrol
- et al.
A linear optimization procedure for an emg-driven neuromusculoskeletal model parameters adjusting: validation through a myoelectric exoskeleton control
- et al.
A linear approach to optimize an emg-driven neuromusculoskeletal model for movement intention detection in myo-control: a case study on shoulder and elbow joints
Front. Neurorobot.
(2018) - et al.
A neuromusculoskeletal model of the human upper limb for a myoelectric exoskeleton control using a reduced number of muscles
- C. Camardella, M. Barsotti, L.P. Murciego, D. Buongiorno, V. Bevilacqua, A. Frisoli, Evaluating generalization...
- et al.
Muscle synergy patterns as physiological markers of motor cortical damage
Proc. Nat. Acad. Sci.
(2012)
Modularity in motor control: from muscle synergies to cognitive action representation
Front. Media SA
Control of fast-reaching movements by muscle synergy combinations
J. Neurosci.
Man/machine interface based on the discharge timings of spinal motor neurons after targeted muscle reinnervation
Nat. Biomed. Eng.
Cited by (5)
Myoelectric Model Reference Adaptive Control with Adaptive Kalman Filter for a soft elbow exoskeleton
2024, Control Engineering PracticeOn the Robustness of Muscle Synergies in the Lower Extremity for Human-Exoskeletons Interaction
2023, Procedia Computer ScienceA Proposal of Bioinspired Soft Active Hand Prosthesis
2023, BiomimeticsHow Many Muscles? Optimal Muscles Set Search for Optimizing Myocontrol Performance
2021, Frontiers in Computational Neuroscience
Cristian Camardella is a PhD Student from Scuola Superiore Sant’Anna in Perceptual Robotics. He received the bachelor and master degree in Automation Engineering from Polytechnic University of Bari in 2017. In 2019 he was a visiting student at the Chinese University of Hong Kong, in collaboration with Prof. Raymond Tong on the post-stroke muscle synergies analysis and rehabilitation robotics strategies development. His interests cover the development of human-machine interfaces, bio-signals processing, robotics, machine learning and control strategies. Within these topics, he worked on muscle synergies and myoelectric control applications, in the rehabilitation context, as one of the main research topic.
Michele Barsotti received the bachelor and master degree (M.Sc.) in Biomedical Engineering from the University of Pisa in 2009 and 2011 respectively. He obtained his PhD degree in Emerging Digital Technologies from the Scuola Superiore Sant’Anna of Pisa on October 27th, 2016 with a dissertation on “Human-Machine interfaces based on electro-biological signals for robotic application and neurorehabilitation” with the final evaluation score of full marks and honor. In 2016 he was a visiting student at the Institute of Neurorehabilitation Systems at the University Medical Center G?ttingen, Georg-August University, Germany. He works as a post-doc researcher at the PercRo laboratory and his research interests included bio-signal processing, robotic, wearable robots, neuro-rehabilitation and neural control of movements. He is currently working as a Machine Learning DSP Software Engineer at the Camlin Italy SRL.
Domenico Buongiorno is a Post-Doc Researcher in Electronic and Information Bioengineering at the Department of Electrical and Information Engineering of Polytechnic University of Bari. He received the B.Sc. and M.Sc. (cum laude) degrees in automation and control theory engineering from Politecnico di Bari, Bari, Italy, in 2011 and 2014, respectively. His bachelor and master theses, both supervised by the Prof. Vitoantonio Bevilacqua, concerned machine learning-based optimization techniques for energy consumption optimization and human neuromusculoskeletal modeling, respectively.
In 2017, he received the Ph.D. degree in Emerging Digital Technologies from Scuola Superiore Sant?Anna, Pisa, Italy (PERCeptual RObotics laboratory – PERCRO). His PhD thesis concerns the control of robotic interfaces for interaction with virtual environments, robot-aided neurorehabilitation (EMG based control – Myocontrol) and bilateral multi-DoF teleoperation in disaster scenario. In 2017, he has been a visiting PhD Student at the Biorobotics Laboratory, UCI Irvine, California, under the sypervision of the Prof. David Reinkensmeyer. He is currently working on the research projects SOS and RoboVir, and on two industrial projects in collaboration with COMAU Robot Company.
His research interests include human-machine and human-robot interaction, assistive technologies, muscle synergy analysis and myocontrol for neuro-rehabilitation. He teaches Computer Science at the Polytechnic University of Bari (a.y. 2017/2018 and 2018/2019) and Electronic and Information Bioengineering at the Medical School of the University of Bari (a.y. 2018/2019).
In March 2019, he founded Apulian Bioengineering srl, a spin-off company of the Polytechnic University of Bari.
Antonio Frisoli (Eng., PhD) is Full Professor of Robotics and Engineering Mechanics at Scuola Superiore Sant?Anna, where he leads the Human-Robot Interaction area and he is responsible of the Sant’Anna Macronode for the Artes National Competence Center on Collaborative Robotics. He received his PhD (2002) with honors in Industrial and Information Engineering from Scuola Superiore Sant?Anna, Italy and the MSc (1998) in Mechanical Engineering, minor Robotics, from University of Pisa-Italy. His interests in advanced kinematics started with the study of screw theory and Lie algebra applied to the synthesis and study of dynamics of novel parallel kinematics. His research interests are in the field of wearable robotics and haptic interfaces, advanced robotic solutions for inspection, maintenance and disaster scenarios, rehabilitation robotics and advanced kinematics.
Vitoantonio Bevilacqua is Associate Professor of Electronic and Information Bioengineering at the Department of Electrical and Information Engineering of Polytechnic University of Bari (DEI-POLIBA) where obtained the Laurea Degree in Electronic Engineering, Ph.D. in Electrical Engineering and Post-Doc in Industrial Informatics and is the Head of Industrial Informatics Lab. In 2019 he got the National Scientific Habilitation (ASN) as Full Professor of Information Processing Systems.
Since 1996 he has been working and investigating in the field of computer vision and image processing, bioengineering, human-machine interaction based on machine learning and soft computing techniques (neural networks, evolutionary algorithms, hybrid expert systems, deep learning). The main applications of his research are in medicine, in biometry, in bioinformatics in ambient assisted living and industry.
- ☆
This work has been partially founded by the PRIN-2015 ModuLimb (Prot. 2015HFWRYY) and supported by the Italian project RoboVir within the BRIC INAIL-2016 program.