Abstract
The ability to recognize and classify human gestures bears a huge potential of improving haptic feedback in virtual environments: when it is possible to predict how a user wants to explore or manipulate his environment both the visual and the auditive feedback may be adapted in order to enhance the immersiveness of haptic displays. Within this work a software approach is suggested that allows for such a real-time classification of gestures in a continuous data stream. A visually based tracking system is used to record the hand movements, while Hidden Markov models are applied to analyze the data. After presenting the methodological background and the software implementation, the outcome of an evaluation study is discussed. It reveals satisfying results of gesture classification and, what is particularly important, the recognition is fast enough to enable multi-modal haptic feedback.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Deml, B.: Human factors issues on the design of telepresence systems. Presence: Teleoperators and Virtual Environments 16, 471–487 (2007)
Deml, B., Mihalyi, A., Hannig, G.: Development and experimental evaluation of a thermal display. In: EuroHaptics, pp. 257–262 (2006)
Hannig, G., Deml, B., Mihalyi, A.: Simulating surface roughness in virtual environments by vibro-tactile feedback. In: IFAC Symposium Analysis, Design, and Evaluation of Human-Machine Systems, CD-ROM, pages 6 (2007)
Guest, S., Catmur, C., Lloyd, D., Spence, C.: Audiotactile interactions in roughness perception. Experimental Brain Research 16, 161–171 (2002)
Avanzini, F., Crosato, P.: Integrating physically based sound models in a multimodal rendering architecture. Computer Animation and Virtual Worlds 17, 411–419 (2006)
Dominjon, L., Lecuyer, A., Burkhardt, J., Richard, P., Richir, S.: Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments. In: IEEE Int. Conf. on Virtual Reality, pp. 19–25 (2005)
Cadoz, C.: Die virtuelle Realität. Lübbe, Bergisch Gladbach (2002)
Lederman, S., Klatzky, R.: Hand Movements: A Window into Haptic Object Recognition. Cognitive Psychology 19, 342–368 (1987)
Cutkosky, M.: On grasp choice, grasp models and the design of hands for manufacturing tasks. IEEE Trans. on Robotics and Automation. 5, 269–279 (1989)
Rabiner, L.: A tutorial on hidden Markov models and selected applications in Speech Recognition. Proc. of the IEEE 77, 257–286 (1989)
Xu, Y., Yang, J.: Towards human-robot coordination: skill modeling and transferring via hidden Markov model. In: IEEE Int. Conf. on Robotics and Automation, vol. 2, pp. 1906–1911 (1995)
Bruce, K., Cardelli, L., Pierce, B.: Comparing Object Encodings. In: Ito, T., Abadi, M. (eds.) TACS 1997. LNCS, vol. 1281, pp. 415–438. Springer, Heidelberg (1997)
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution. Springer, Berlin (1996)
Baldonado, M., Chang, C., Gravano, L., Paepcke, A.: The Stanford Digital Library Metadata Architecture. Int. J. Digit. Libr. 1, 108–121 (1997)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Frolov, V., Deml, B., Hannig, G. (2008). Gesture Recognition with Hidden Markov Models to Enable Multi-modal Haptic Feedback. In: Ferre, M. (eds) Haptics: Perception, Devices and Scenarios. EuroHaptics 2008. Lecture Notes in Computer Science, vol 5024. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69057-3_100
Download citation
DOI: https://doi.org/10.1007/978-3-540-69057-3_100
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69056-6
Online ISBN: 978-3-540-69057-3
eBook Packages: Computer ScienceComputer Science (R0)