Abstract
Hand gesture recognition (HGR) based on electromyography (EMG) has been a research topic of great interest in recent years. Designing an HGR to be robust enough to the variation of EMGs is a challenging problem and most of the existing studies have explored supervised learning to design HGRs methods. However, reinforcement learning, which allows an agent to learn online while taking EMG samples, has barely been investigated. In this work, we propose a HGR system composed of the following stages: pre-processing, feature extraction, classification and post-processing. For the classification stage, we use Q-learning to train an agent that learns to classify and recognize EMGs from five gestures of interest. At each step of training, the agent interacts with a defined environment, obtaining thus a reward for the action taken in the current state and observing the next state. We performed experiments using a public EMGs dataset, and the results were evaluated for user-specific HGR models by using a method that is robust to the rotations of the EMG bracelet device. The results showed that the classification accuracy reach up to 90.78% and the recognition up to 87.51% for two different test-sets for 612 users in total. The results obtained in this work show that reinforcement learning methods such as Q-learning can learn a policy from online experiences to solve both the hand gesture classification and the recognition problem based on EMGs.
Supported by Escuela Politécnica Nacional.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
EMG Gesture Recognition Evaluator. https://aplicaciones-ia.epn.edu.ec/webapps/home/session.html?app=EMG Gesture Recognition Evaluator
Barona López, L.I., Valdivieso Caraguay, Á.L., Vimos, V.H., Zea, J.A., Vásconez, J.P., Álvarez, M., Benalcázar, M.E.: An energy-based method for orientation correction of emg bracelet sensors in hand gesture recognition systems. Sensors 20(21), 6327 (2020)
Benalcázar, M.E., Jaramillo, A.G., Zea, A., Páez, A., Andaluz, V.H., et al.: Hand gesture recognition using machine learning and the myo armband. In: 2017 25th European Signal Processing Conference (EUSIPCO), pp. 1040–1044. IEEE (2017)
Benalcázar, M.E., et al.: Real-time hand gesture recognition using the myo armband and muscle activity detection. In: 2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM), pp. 1–6. IEEE (2017)
Benalcázar, M., Barona, L., Valdivieso, L., Aguas, X., Zea, J.: Emg-epn-612 dataset (2020). https://doi.org/10.5281/zenodo.4027874
Englehart, K., Hudgins, B.: A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 50(7), 848–854 (2003)
Jaramillo-Yánez, A., Benalcázar, M.E., Mena-Maldonado, E.: Real-time hand gesture recognition using surface electromyography and machine learning: a systematic literature review. Sensors 20(9), 2467 (2020)
Kapturowski, S., Ostrovski, G., Quan, J., Munos, R., Dabney, W.: Recurrent experience replay in distributed reinforcement learning. In: International Conference on Learning Representations (2018)
Kukker, A., Sharma, R.: Neural reinforcement learning classifier for elbow, finger and hand movements. J. Intell. Fuzzy Syst. 35(5), 5111–5121 (2018)
McGill, K.: Surface electromyogram signal modelling. Med. Biol. Eng. Comput. 42(4), 446–454 (2004)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Seok, W., Kim, Y., Park, C.: Pattern recognition of human arm movement using deep reinforcement learning. In: 2018 International Conference on Information Networking (ICOIN), pp. 917–919. IEEE (2018)
Song, C., Chen, C., Li, Y., Wu, X.: Deep reinforcement learning apply in electromyography data classification. In: 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS), pp. 505–510. IEEE (2018)
Sugiyama, M., Kawanabe, M.: Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation. MIT Press, Cambridge (2012)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Wang, F., et al.: Robot learning by demonstration interaction system based on multiple information. In: 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 138–143. IEEE (2018)
Acknowledgment
The authors gratefully acknowledge the financial support provided by the Escuela Politécnica Nacional (EPN) for the development of the research project “PIGR-19-07 Reconocimiento de gestos de la mano usando señales
electromiográficas e inteligencia artificial y su aplicación para la implementación de interfaces humano—máquina y humano—humano”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Vásconez, J.P., López, L.I.B., Caraguay, Á.L.V., Cruz, P.J., Álvarez, R., Benalcázar, M.E. (2021). A Hand Gesture Recognition System Using EMG and Reinforcement Learning: A Q-Learning Approach. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12894. Springer, Cham. https://doi.org/10.1007/978-3-030-86380-7_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-86380-7_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86379-1
Online ISBN: 978-3-030-86380-7
eBook Packages: Computer ScienceComputer Science (R0)