Abstract
This paper presents the application of hand gestures and arm movements to control a dual rotor testbench. A multimodal control method is developed for a 3-degrees-of-freedom (DOF) tandem helicopter based on surface electromyography sensors and an inertial measurement unit (IMU) included in the Myo Armband sensor. The recognition system can classify five different hand gestures which are used for switching between flight modes and generating set point values for the helicopter. The 3-DOF helicopter testbench is fully designed and implemented as a low cost alternative for assessing the effectiveness of flight controls for unmanned aerial vehicles. The position of the helicopter is regulated by a cascade-dual-PID control scheme that allows a fast switching between a gesture mode and an IMU mode. Experimental results show the effectiveness of using hand gesture recognition and arm movement for controlling an aerial test bench with a fast and accurate response.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Antonius, R., Tjahyadi, H.: Electromyography gesture identification using CNN-RNN neural network for controlling quadcopters. In: Journal of Physics: Conference Series, vol. 1858, p. 012075. IOP Publishing (2021)
Barona López, L.I., et al.: An energy-based method for orientation correction of EMG bracelet sensors in hand gesture recognition systems. Sensors 20(21), 6327 (2020)
Benalcazar, M.E., Barona, L., Valdivieso, L., Aguas, X., Zea, J.: EMG-EPN-612 dataset, November 2020. https://doi.org/10.5281/ZENODO.4023305. https://laboratorio-ia.epn.edu.ec/en/resources/dataset/2020_emg_dataset_612
Benalcázar, M.E., Jaramillo, A.G., Zea, A., Páez, A., Andaluz, V.H., et al.: Hand gesture recognition using machine learning and the Myo armband. In: 2017 25th European Signal Processing Conference (EUSIPCO), pp. 1040–1044. IEEE (2017)
Choudhary, S.K.: Negative imaginary feedback control for a 3-DOF helicopter system. In: 2019 8th International Conference on Modeling Simulation and Applied Optimization (ICMSAO), pp. 1–6. IEEE (2019)
Jiang, D., et al.: Gesture recognition based on binocular vision. Clust. Comput. 22(6), 13261–13271 (2018). https://doi.org/10.1007/s10586-018-1844-5
Jiang, S., et al.: Feasibility of wrist-worn, real-time hand, and surface gesture recognition via sEMG and IMU sensing. IEEE Trans. Ind. Inform. 14(8), 3376–3385 (2017)
Kim, M., Cho, J., Lee, S., Jung, Y.: IMU sensor-based hand gesture recognition for human-machine interfaces. Sensors 19(18), 3827 (2019)
Ma, Y., et al.: Hand gesture recognition with convolutional neural networks for the multimodal UAV control. In: 2017 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 198–203. IEEE (2017)
Nuzzi, C., Pasinetti, S., Lancini, M., Docchio, F., Sansoni, G.: Deep learning based machine vision: first steps towards a hand gesture recognition set up for collaborative robots. In: 2018 Workshop on Metrology for Industry 4.0 and IoT, pp. 28–33. IEEE (2018)
OYMotion: Gforcepro + EMG armband. http://www.oymotion.com/en/product32/149
Park, J.S., Na, H.J.: Front-end of vehicle-embedded speech recognition for voice-driven multi-UAVs control. Appl. Sci. 10(19), 6876 (2020)
Pérez-Ventura, U., Fridman, L., Capello, E., Punta, E.: Fault tolerant control based on continuous twisting algorithms of a 3-DOF helicopter prototype. Control Eng. Pract. 101, 104486 (2020)
Schreck, B., Gross, L.: Gesture controlled UAV proposal. In: Web. mit. edu., vol. 29 (2014)
Visconti, P., Gaetani, F., Zappatore, G., Primiceri, P., et al.: Technical features and functionalities of Myo armband: an overview on related literature and advanced applications of myoelectric armbands mainly focused on arm prostheses. Int. J. Smart Sens. Intell. Syst 11(1), 1–25 (2018)
Wen, F., et al.: Machine learning glove using self-powered conductive superhydrophobic triboelectric textile for gesture recognition in VR/AR applications. Adv. Sci. 7(14), 2000261 (2020)
Yang, L., Chen, J., Zhu, W.: Dynamic hand gesture recognition based on a leap motion controller and two-layer bidirectional recurrent neural network. Sensors 20(7), 2106 (2020)
Zea, J.A., Benalcázar, M.E.: Real-time hand gesture recognition: a long short-term memory approach with electromyography. In: Nummenmaa, J., Pérez-González, F., Domenech-Lega, B., Vaunat, J., Oscar Fernández-Peña, F. (eds.) CSEI 2019. AISC, vol. 1078, pp. 155–167. Springer, Cham (2020) . https://doi.org/10.1007/978-3-030-33614-1_11
Acknowledgment
The authors gratefully acknowledge the financial support provided by the Escuela Politécnica Nacional (EPN) for the development of the research project “PIGR-19-07 Reconocimiento de gestos de la mano usando señales electromio- gráficas e inteligencia artificial y su aplicación para la implementación de interfaces humano-máquina y humano-humano”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Romero, R. et al. (2022). Hand Gesture and Arm Movement Recognition for Multimodal Control of a 3-DOF Helicopter. In: Kim, J., et al. Robot Intelligence Technology and Applications 6. RiTA 2021. Lecture Notes in Networks and Systems, vol 429. Springer, Cham. https://doi.org/10.1007/978-3-030-97672-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-97672-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97671-2
Online ISBN: 978-3-030-97672-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)