Abstract
Sign language recognition constitutes a challenging field of research in computer vision. Common problems like overlap, ambiguities, and minimal pairs occur frequently and require robust algorithms for feature extraction and processing. We present a system that performs person-dependent recognition of 232 isolated signs with an accuracy of 99.3% in a controlled environment. Person-independent recognition rates reach 44.1% for 221 signs. An average performance of 87.8% is achieved for six signers in various uncontrolled indoor and outdoor environments, using a reduced vocabulary of 18 signs.
The system uses a background model to remove static areas from the input video on pixel level. In the tracking stage, multiple hypotheses are pursued in parallel to handle ambiguities and facilitate retrospective correction of errors. A winner hypothesis is found by applying high level knowledge of the human body, hand motion, and the signing process. Overlaps are resolved by template matching, exploiting temporally adjacent frames with no or less overlap. The extracted features are normalized for person-independence and robustness, and classified by Hidden Markov Models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Starner, T., Weaver, J., Pentland, A.: Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video. In: IEEE PAMI (1998)
Tanibata, N., Shimada, N., Shirai, Y.: Extraction of Hand Features for Recognition of Sign Language Words. In: Proc. Int. Conf. Vision Interface (2002)
Bauer, B., Kraiss, K.F.: Video-Based Sign Recognition using Self-Organizing Subunits. In: Wachsmuth, I., Sowa, T. (eds.) GW 2001. LNCS (LNAI), vol. 2298, Springer, Heidelberg (2002)
Zieren, J., Kraiss, K.F.: Non-Intrusive Sign Language Recognition for Human-Computer Interaction. In: Proceedings of the IFAC-HMS Symposium (2004)
Yang, M.H., Ahuja, N., Tabb, M.: Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition. In: IEEE TPAMI, vol. 24 (2002)
Akyol, S., Canzler, U.: An Information Terminal using Vision Based Sign Language Recognition. In: Büker, U., Eikerling, H.J., Müller, W. (eds.) ITEA Workshop on Virtual Home Environments, VHE Middleware Consortium, vol. 12 (2002)
Jones, M., Rehg, J.: Statistical Color Models with Application to Skin Detection. Technical Report CRL 98/11, Compaq Cambridge Research Lab (1998)
Imagawa, K., Lu, S., Igi, S.: Color-Based Hand Tracking System for Sign Language Recognition. In: IEEE Int. Conf. on Autom. Face and Gesture Recognition (1998)
Huang, C.L., Huang, W.Y.: Sign language recognition using model-based tracking and a 3D Hopfield neural network. Machine Vision and Applications 10 (1998)
Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Computer Vision and Pattern Recognition 1999, vol. 2 (1999)
KaewTraKulPong, P., Bowden, R.: An Improved Adaptive Background Mixture Model for Realtime Tracking with Shadow Detection. In: AVBS (2001)
Porikli, F., Tuzel, O.: Human Body Tracking by Adaptive Background Models and Mean-Shift Analysis. Technical report, Mitsubishi Electric Research Lab. (2003)
Rabiner, L., Juang, B.H.: An Introduction to Hidden Markov Models. IEEE ASSP Magazine 3 (1986)
Sonka, M., Hlavac, V., Boyle, R.: Image Processing, Analysis and Machine Vision. Brooks Cole (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zieren, J., Kraiss, KF. (2005). Robust Person-Independent Visual Sign Language Recognition. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds) Pattern Recognition and Image Analysis. IbPRIA 2005. Lecture Notes in Computer Science, vol 3522. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11492429_63
Download citation
DOI: https://doi.org/10.1007/11492429_63
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26153-7
Online ISBN: 978-3-540-32237-5
eBook Packages: Computer ScienceComputer Science (R0)