Skip to main content
SpringerLink
Log in

Hand Gesture Recognition Applied to the Interaction with Video Games

  • Conference paper
  • First Online:
Advances in Computational Intelligence (MICAI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14391))

Included in the following conference series:

  • 291 Accesses

Abstract

In this work, a hand gesture recognition system was created for 11 different gestures. The system employed CNN-LSTM artificial neural networks and followed the CRISP-ML(Q) process model. The aim was to incorporate software engineering practices into machine learning projects. The system uses Electromyography (EMG) and Inertial Measurement Unit (IMU) signals as input to compute a gesture label and the time of occurrence in the signal. The system is integrated with a video game that utilizes hand gestures as input. A system usability scale (SUS) survey was done by ten final users in order to measure the interaction with the video game using gestures as the main way of interaction. The complete application evaluation obtained a SUS score of 75, or a B grade.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Benalcazar, M.E., Motoche, C., Perez, M.: Real-time hand gesture recognition using the Myo armband and muscle activity detection, pp. 1–6, IEEE (2017)

    Google Scholar 

  2. Oudah, M., Al-Naji, A., Chahl, J.: Imaging hand gesture recognition based on computer vision: a review of techniques. J. Imaging 6(8), 73 (2020)

    Google Scholar 

  3. Jiménez, L.A.E., Benalcázar, M.E.: Gesture recognition and machine learning applied to sign language. IFMBE Proceedings 60, 233–236 (2017)

    Article  Google Scholar 

  4. Lamberti, L., Camastra, F.: Handy: a real-time three color glove-based gesture recognizer with learning vector quantization. Expert Syst. Appl. 39, 10489–10494 (2012)

    Google Scholar 

  5. Mantecón, T., del-Blanco, C.R., Jaureguizar, F., García, N.: Hand gesture recognition using infrared imagery provided by leap motion controller. In: Blanc-Talon, J., Distante, C., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2016. LNCS, vol. 10016, pp. 47–57. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48680-2_5

    Chapter  Google Scholar 

  6. Lu, W. Tong, Z., Chu, J.: Dynamic hand gesture recognition with leap motion controller. IEEE Signal Process. Lett. 23, 1188–1192 (2016)

    Google Scholar 

  7. Moschetti, A., Fiorini, L., Esposito, D., Dario, P., Cavallo, F.: Recognition of daily gestures with wearable inertial rings and bracelets. Sensors (Basel, Switzerland), 16, 8 (2016)

    Google Scholar 

  8. Geng, W., Du, Y., Jin, W., Wei, W., Hu, Y., Li, J.: Gesture recognition by instantaneous surface EMG images open. Sci. Rep. 6, 36571, (2016). https://doi.org/10.1038/srep36571

  9. Huang, Y., et al.: Preliminary testing of a hand gesture recognition wristband based on EMG and inertial sensor fusion. In: Liu, H., Kubota, N., Zhu, X., Dillmann, R., Zhou, D. (eds.) ICIRA 2015. LNCS (LNAI), vol. 9244, pp. 359–367. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22879-2_33

    Chapter  Google Scholar 

  10. Georgi, M., Amma, C., Schultz, T.: Recognizing hand and finger gestures with IMU based motion and EMG based muscle activity sensing (2015)

    Google Scholar 

  11. Chen, X., Zhang, X., Zhao, Z.Y., Yang, J.H., Lantz, V., Wang, K.Q.: Hand gesture recognition research based on surface EMG sensors and 2D-accelerometers. In: Proceedings - International Symposium on Wearable Computers, pp. 11–14 (2007)

    Google Scholar 

  12. Wolf, M., et al.: Decoding static and dynamic arm and hand gestures from the JPL BioSleeve. In: IEEE Aerospace Conference Proceedings (2013)

    Google Scholar 

  13. Algorithmia: 2020 state of enterprise machine learning

    Google Scholar 

  14. Sculley, D., et al.: Hidden technical debt in machine learning systems (2015)

    Google Scholar 

  15. Studer, S., et al.: Towards CRISP-ML(Q): a machine learning process model with quality assurance methodology (2020)

    Google Scholar 

  16. Zea, J., Benalcázar, M.E., López, L.I.B.: An open-source data acquisition and manual segmentation system for hand gesture recognition based on EMG. In: ETCM 2021–5th Ecuador Technical Chapters Meeting (2021)

    Google Scholar 

  17. Benalcázar, M.E., Jaramillo, A.G., Zea, J.A., Paéz, A., Andaluz, V.: Hand gesture recognition using machine learning and the Myo armband. In: 25th European Signal Processing Conference, EUSIPCO, pp. 1040–1044 (2017)

    Google Scholar 

  18. Bernhardt, P.: How i learned to stop worrying and love quaternions (2015)

    Google Scholar 

  19. Artificial Intelligence and Computer Vision Research Lab: EMG-IMU-EPN-100+. https://laboratorio-ia.epn.edu.ec/en/resources/dataset/emg-imu-epn-100. Accessed 01 Sept 2023

  20. López, L.I.B., Caraguay, A.L.V., Vóscornez, J.P., Benalcázar, M.E.: An energy-based method for orientation correction of EMG bracelet sensors in hand gesture recognition systems. Sensors 20, 6327 (2020)

    Google Scholar 

  21. Reaz, M., Mohd-Yasin, F.: Techniques of EMG signal analysis: detection, processing, classification and applications. Biol. Proced. 8, 11 (2006)

    Article  Google Scholar 

  22. Sironi, E., Wermter, S.: An analysis of convolutional long short-term memory recurrent neural networks. Neurocomputing 268, 76–86 (2017)

    Google Scholar 

  23. Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)

    Google Scholar 

  24. Ripalda, F.M.F.: Desarrollo de un modelo de reconocimiento de gestos de la mano utilizando señales EMG y Deep Learning (2021)

    Google Scholar 

  25. Tomaszewski, M.: Myo SDK MATLAB MEX wrapper (2022)

    Google Scholar 

  26. Nuñez, H.G.R.: Videojuego educativo en 3d para dispositivos móviles android, enfocado al aprendizaje de la lógica de programación para usuarios entre los 5 a 18 años de eda (2017)

    Google Scholar 

  27. Rojas, H.D.L.: Implementación de un juego serio multiplataforma para el desarrollo de la orientación espacial en niños de 6 a 8 añ0 (2017)

    Google Scholar 

  28. N. Acerenza, et al.: Una metodología para desarrollo de videojuegos: versión extendida (2009)

    Google Scholar 

  29. Pabon, F.M.P.: Diseño y desarrollo de un video juego educativo con agentes inteligentes aplicando la metodología sum. caso de estudio: TIC-TAC-TOE (2015)

    Google Scholar 

  30. Brooke, J.: Sus: a quick and dirty usability scale. Usability Eval. Ind. 189, 11 (1995)

    Google Scholar 

  31. Sauro, J.: 5 ways to interpret a Sus score - measuringu (2018)

    Google Scholar 

Download references

Acknowledgment

The authors gratefully acknowledge the financial support provided by the Escuela Politécnica Nacional (EPN) for the development of the research project “PIGR-22-09 Avances para el desarrollo de un prototipo de prótesis mioeléctrica de mano y control avanzado de su operación usando Inteligencia Artificial".

Author information

Authors and Affiliations

  1. Artificial Intelligence and Computer Vision Research Lab, Departamento de Informática Y Ciencias de la Computación (DICC), Escuela Politécnica Nacional, Quito, 170517, Ecuador

    Lorena Isabel Barona López, César Israel León Cifuentes, José Miguel Muñoz Oña, Angel Leonardo Valdivieso Caraguay & Marco E. Benalcázar

Authors
  1. Lorena Isabel Barona López
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. César Israel León Cifuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José Miguel Muñoz Oña
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Angel Leonardo Valdivieso Caraguay
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marco E. Benalcázar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lorena Isabel Barona López .

Editor information

Editors and Affiliations

  1. Center for Computing Research, Instituto Politécnico Nacional, Ciudad de México, Distrito Federal, Mexico

    Hiram Calvo

  2. Facultad de Ingeniería, Universidad Panamericana, Ciudad de México, Mexico

    Lourdes Martínez-Villaseñor

  3. Facultad de Ingeniería, Universidad Panamericana, Ciudad de México, Mexico

    Hiram Ponce

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Barona López, L.I., León Cifuentes, C.I., Muñoz Oña, J.M., Valdivieso Caraguay, A.L., Benalcázar, M.E. (2024). Hand Gesture Recognition Applied to the Interaction with Video Games. In: Calvo, H., Martínez-Villaseñor, L., Ponce, H. (eds) Advances in Computational Intelligence. MICAI 2023. Lecture Notes in Computer Science(), vol 14391. Springer, Cham. https://doi.org/10.1007/978-3-031-47765-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47765-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47764-5

  • Online ISBN: 978-3-031-47765-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation