Abstract
In this paper, we present how appearance-based features can be used for the recognition of words in American sign language (ASL) from a video stream. The features are extracted without any segmentation or tracking of the hands or head of the signer, which avoids possible errors in the segmentation step. Experiments are performed on a database that consists of 10 words in ASL with 110 utterances in total. These data are extracted from a publicly available collection of videos and can therefore be used by other research groups. The video streams of two stationary cameras are used for classification, but we observe that one camera alone already leads to sufficient accuracy. Hidden Markov Models and the leaving one out method are employed for training and classification. Using the simple appearance-based features, we achieve an error rate of 7%. About half of the remaining errors are due to words that are visually different from all other utterances.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Triesch, J., von der Malsburg, C.: A System for Person-Independent Hand Posture Recognition against Complex Backgrounds. IEEE Trans. Pattern Analysis and Machine Intelligence 23(12), 1449–1453 (2001)
Birk, H., Moeslund, T.B., Madsen, C.B.: Real-Time Recognition of Hand Alphabet Gestures Using Principal Component Analysis. In: 10th Scandinavian Conference on Image Analysis, Laeenranta, Finland (June 1997)
Malassiottis, S., Aifanti, N., Strintzis, M.G.: A Gesture Recognition System Using 3D Data. In: Proceedings IEEE 1st International Symposium on 3D Data Processing, Visualization, and Transmission, Padova, Italy, June 2002, pp. 190–193 (2002)
Mehdi, S.A., Khan, Y.N.: Sign Language Recognition Using Sensor Gloves. In: Proceedings of the 9th International Conference on Neural Information Processing, Singapore, November 2002, vol. 5, pp. 2204–2206 (2002)
Abe, K., Saito, H., Ozawa, S.: Virtual 3-D Interface System via Hand Motion Recognition From Two Cameras. IEEE Trans. Systems, Man, and Cybernetics 32(4), 536–540 (2002)
Hernandez-Rebollar, J.L., Lindeman, R.W., Kyriakopoulos, N.: A Multi-Class Pattern Recognition System for Practical Finger Spelling Translation. In: Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, Pittsburgh, PA, October 2002, pp. 185–190 (2002)
Nam, Y., Wohn, K.: Recognition of Space-Time Hand-Gestures Using Hidden Markov Model. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, Hong Kong, July 1996, pp. 51–58 (1996)
Bauer, B., Hienz, H., Kraiss, K.F.: Video-Based Continuous Sign Language Recognition Using Statistical Methods. In: Proceedings of the International Conference on Pattern Recognition, Barcelona, Spain, September 2000, pp. 463–466 (2000)
Starner, T., Weaver, J., Pentland, A.: Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video. IEEE Trans. Pattern Analysis and Machine Intelligence 20(12), 1371–1375 (1998)
Vogler, C., Metaxas, D.: Adapting Hidden Markov Models for ASL Recognition by Using Three-dimensional Computer Vision Methods. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Orlando, FL, October 1997, pp. 156–161 (1997)
Neidle, C., Kegl, J., MacLaughlin, D., Bahan, B., Lee, R.G.: The Syntax of American Sign Language: Functional Categories and Hierarchical Structure. MIT Press, Cambridge (2000)
Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 267–296 (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zahedi, M., Keysers, D., Ney, H. (2005). Appearance-Based Recognition of Words in American Sign Language. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds) Pattern Recognition and Image Analysis. IbPRIA 2005. Lecture Notes in Computer Science, vol 3522. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11492429_62
Download citation
DOI: https://doi.org/10.1007/11492429_62
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26153-7
Online ISBN: 978-3-540-32237-5
eBook Packages: Computer ScienceComputer Science (R0)