Abstract
An open source framework for general gesture recognition is presented and tested with isolated signs of sign language. Other than common systems for sign language recognition, this framework makes use of Kinect, a depth camera which makes real-time 3D-reconstruction easily applicable. Recognition is done using hidden Markov models with a continuous observation density. The framework also offers an easy way of initializing and training new gestures or signs by performing them several times in front of the camera. First results with a recognition rate of 97% show that depth cameras are well-suited for sign language recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kelly, D., McDonald, J., Markham, C.: Recognizing Spatiotemporal Gestures and Movement Epenthesis in Sign Language. In: 13th International Machine Vision and Image Processing Conference (IMVIP 2009). IEEE Computer Society, Washington, DC (2009)
Dreuw, P., Rybach, D., Deselaers, T., Zahedi, M., Ney, H.: Speech Recognition Techniques for a Sign Language Recognition System. In: INTERSPEECH 2007, 8th Annual Conference of the International Speech Communication Association (ISCA 2007), pp. 2513–2516 (2007)
Li, X., Parizeau, M., Plamondon, R.: Training Hidden Markov Models with Multiple Observations – A combinatorial Method. IEEE Transactions on PAMI PAMI-22(4), 371–377 (2000)
Vogler, C., Metaxas, D.: ASL Recognition Based on a Coupling Between HMMs and 3D Motion Analysis. In: Proceedings of the Sixth International Conference on Computer Vision, pp. 363–369. Narosa Publishing House (1998) ISBN: 978-8-17319-221-0
Starner, T., Pentland, A.: Real-Time American Sign Language Recognition from Video Using Hidden Markov Models. In: Proceedings of the International Symposium on Computer Vision, ISCV 1995, pp. 265–270. IEEE Publications, U.S (1995) ISBN: 978-0-81867-190-6
Boyes Braem, P.: Einführung in die Gebärdensprache und ihre Erforschung. In: Internationale Arbeiten zur Gebärdensprache und Kommunikation Gehörloser, 1st edn., vol. 11. SIGNUM-Verlag (1990) ISBN: 978-3-92773-110-3
Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1989)
Rahimi, A.: An Erratum for “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition”, website of Ali Rahimi at MIT Media Laboratory, http://xenia.media.mit.edu/~rahimi/rabiner/rabiner-errata/rabiner-errata.html
Official Microsoft Xbox website, introduction of Kinect, http://www.xbox.com/en-US/kinect
Countdown to Kinect: 17 Controller-Free Games Launch in November, Microsoft News Center, https://www.microsoft.com/presspass/press/2010/oct10/10-18mskinectuspr.mspx
Kinect Downgraded To Save Money, Can’t Read Sign Language, News at Kotaku, http://kotaku.com/5609840/kinect-dumbed-down-to-save-money-cant-read-sign-language
CopyCat and Kinect, overview of the CopyCat Kinect demo on the website of the Center for Accessible Technology in Sign (CATS), http://cats.gatech.edu/content/copycat-and-kinect
Integrating Speech and Hearing Challenge Individuals, YouTube channel of Dr. Natheer Khasawneh, http://www.youtube.com/user/knatheer#p/a/u/1/vVL398dUU5Q
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lang, S., Block, M., Rojas, R. (2012). Sign Language Recognition Using Kinect. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2012. Lecture Notes in Computer Science(), vol 7267. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29347-4_46
Download citation
DOI: https://doi.org/10.1007/978-3-642-29347-4_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29346-7
Online ISBN: 978-3-642-29347-4
eBook Packages: Computer ScienceComputer Science (R0)