Abstract
Due to recent improvements in Virtual Reality (VR) regarding the potential of full-body tracking, the number of VR-based exergames has been increasing. However, such applications often depend on additional tracking technology, e.g., markerless or marker-based. On the one hand, tracking approaches, such as the Kinect device are limited by either high latency or insufficient accuracy. On the other hand, motion capture suits are expensive and create discomfort. In this paper we present an accurate motion recognition approach, using only the HTC Vive HMD with their associated Controllers and Trackers. The recognition is based on an Hidden Markov Model, that has been trained in advance for a specific movement. The results suggest that our system is capable of detecting a complex full-body gesture, such as yoga Warrior I pose, with an accuracy of 88%. In addition, audible feedback is provided, so that the user can immediately hear if the particular exercise has been executed correctly. Such a system can be used to assist players in learning a particular movement and can be applied in various serious games applications, e.g., for training purposes or rehabilitation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://www.xbox.com/en-GB/xbox-one/accessories/kinect, last visited on May 25th, 2018.
- 2.
http://optitrack.com, last visited on May 29th, 2018.
- 3.
https://vicon.com, last visited on May 29th, 2018.
- 4.
https://docs.microsoft.com/en-us/previous-versions/windows/kinect/dn785304(v%3dieb.10), last visited on May 25th, 2018.
- 5.
https://www.leapmotion.com, last visited on May 28, 2018.
- 6.
https://www.myo.com, last visited on May 28th, 2018.
- 7.
https://www.xsens.com/functions/human-motion-measurement/, last visited on May 28th, 2018.
- 8.
https://github.com/CatCuddler/BodyTracking, last visited on May 22th, 2018.
- 9.
https://github.com/Kode/Kore/, last visited on May 22th, 2018.
- 10.
https://text-to-speech-demo.ng.bluemix.net, last visited on May 22th, 2018.
- 11.
https://youtu.be/q-yKLtrTodA, last visited on May 30th, 2018.
- 12.
https://tzuchanchuang.itch.io/gesture-recognition-input-method-for-ar, last visited on June 6th, 2018.
References
Blunsom, P.: Hidden Markov models. Lecture Notes, vol. 15, pp. 18–19, August 2004
Caserman, P., Krabbe, P., Wojtusch, J., von Stryk, O.: Real-time step detection using the integrated sensors of a head-mounted display. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3510–3515, October 2016
Chan, J.C.P., Leung, H., Tang, J.K.T., Komura, T.: A virtual reality dance training system using motion capture technology. IEEE Trans. Learn. Technol. 4(2), 187–195 (2011)
Chen, F.S., Fu, C.M., Huang, C.L.: Hand Gesture recognition using a real-time tracking method and hidden Markov models. Image Vis. Comput. 21(8), 745–758 (2003)
Dittmar, T., Krull, C., Horton, G.: A new approach for touch gesture recognition: conversive hidden non-Markovian models. J. Comput. Sci. 10, 66–76 (2015)
Ferracani, A., Pezzatini, D., Bianchini, J., Biscini, G., Del Bimbo, A.: Locomotion by natural gestures for immersive virtual environments. In: Proceedings of the 1st International Workshop on Multimedia Alternate Realities, AltMM 2016, pp. 21–24. ACM (2016)
Fitzgerald, D., et al.: Development of a wearable motion capture suit and virtual reality biofeedback system for the instruction and analysis of sports rehabilitation exercises. In: 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4870–4874, August 2007
Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans. Cybern. 43(5), 1318–1334 (2013)
Han, P.H., et al.: Moving around in virtual space with spider silk. In: ACM SIGGRAPH 2015 Emerging Technologies, SIGGRAPH 2015, pp. 19:1–19:1. ACM, New York (2015)
Hoang, T.N., Reinoso, M., Vetere, F., Tanin, E.: Onebody: remote posture guidance system using first person view in virtual environment. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction, NordiCHI 2016, pp. 25:1–25:10. ACM, New York (2016)
Jiang, F., Yang, X., Feng, L.: Real-time full-body motion reconstruction and recognition for off-the-shelf VR devices. In: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI 2016, vol. 1, pp. 309–318. ACM, New York (2016)
Just, A., Marcel, S.: A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition. Comput. Vis. Image Underst. 113(4), 532–543 (2009)
Kao, C.Y., Fahn, C.S.: A human-machine interaction technique: hand gesture recognition based on hidden markov models with trajectory of hand motion. Procedia Eng. 15, 3739–3743 (2011). CEIS 2011
Keskin, C., Erkan, A., Akarun, L.: Real time hand tracking and 3D gesture recognition for interactive interfaces using HMM. In: ICANN/ICONIPP 2003, pp. 26–29 (2003)
Koniaris, B., et al.: IRIDiuM: immersive rendered interactive deep media. In: ACM SIGGRAPH 2016 VR Village, SIGGRAPH 2016, pp. 11:1–11:2. ACM, New York (2016)
Lee, B., Cho, M., Min, J., Saakes, D.: Posing and acting as input for personalizing furniture. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction, NordiCHI 2016, pp. 44:1–44:10. ACM, New York (2016)
Nguyen-Duc-Thanh, N., Lee, S., Kim, D.: Two-stage hidden Markov model in gesture recognition for human robot interaction. Int. J. Adv. Robot. Syst. 9(2), 39 (2012)
Pylvänäinen, T.: Accelerometer based gesture recognition using continuous HMMs. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3522, pp. 639–646. Springer, Heidelberg (2005). https://doi.org/10.1007/11492429_77
Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)
Raheja, J., Minhas, M., Prashanth, D., Shah, T., Chaudhary, A.: Robust Gesture Recognition Using Kinect: A Comparison Between DTW and HMM. Optik - International Journal for Light and Electron Optics 126(11), 1098–1104 (2015)
Rhodin, H., et al.: EgoCap: egocentric marker-less motion capture with two fisheye cameras. ACM Trans. Graph. 35(6), 162:1–162:11 (2016)
Tanaka, Y., Hirakawa, M.: Efficient strength training in a virtual world. In: 2016 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), pp. 1–2, May 2016
Yan, S., Ding, G., Guan, Z., Sun, N., Li, H., Zhang, L.: OutsideMe: augmenting dancer’s external self-image by using a mixed reality system. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 2015, pp. 965–970. ACM, New York (2015)
Yang, J., Xu, Y.: Hidden Markov model for gesture recognition. Technical report, Robotics Institute, Carnegie-Mellon University, Pittsburgh, May 1994
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Caserman, P., Tregel, T., Fendrich, M., Kolvenbach, M., Stabel, M., Göbel, S. (2018). Recognition of Full-Body Movements in VR-Based Exergames Using Hidden Markov Models. In: Göbel, S., et al. Serious Games. JCSG 2018. Lecture Notes in Computer Science(), vol 11243. Springer, Cham. https://doi.org/10.1007/978-3-030-02762-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-02762-9_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02761-2
Online ISBN: 978-3-030-02762-9
eBook Packages: Computer ScienceComputer Science (R0)