Abstract
The parallel multistream model is proposed for integration sign language recognition and lip motion. The different time scales existing in sign language and lip motion can be tackled well using this approach. Primary experimental results have shown that this approach is efficient for integration of sign language recognition and lip motion. The promising results indicated that parallel multistream model can be a good solution in the framework of multimodal data fusion. An approach to recognize sign language with scalability with the size of vocabulary and a fast approach to locate lip corners are also proposed in this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ho-Sub Yoon. Jung Soh. Byung-Woo Min. Hyun Seung Yang. Recognition of alphabetical hand gestures using hidden Markov model. IEICE Transactions on Fundamentals of Electronics Communications & Computer Sciences, vol.E82-A, no.7, July 1999, pp.1358–66. Publisher: Inst. Electron. Inf. & Commun. Eng, Japan.
Vogler C. Metaxas D. Parallel hidden Markov models for American Sign Language recognition. Proceedings of the Seventh IEEE International Conference on Computer Vision. IEEE Comput. Soc. Part vol.1, 1999, pp.116–22 vol.1. Los Alamitos, CA, USA
Luettin, J., Thacker, N.A. and Beet, S.W.Speechreading using shape and intensity information,Proc.4th ICSLP Confference,Philadephia,PA,USA,1996.
Michael Vogt.Fast matching of a dynamic lip model to color video sequence under regular illumination conditions.In speechreading by human and machines, NATO ASI series F.,Vol 150,D.G. Stork, M.E. Hennecke (eds), pages199–407,1996.
Jiyong Ma, Wen Gao, Jiangqin Wu and Chunli Wang, A Continuous Chinese Sign Language recognition system, pp428–433, 28-31 March,FG.2000, Grenoble, France.
J. Matas.Color-based Object Recognition. Ph.D thesis, University of Surry,1995.
Karin Sobottka and Ioannis Pitas.Face Localization and facial feature extraction based on color shape and color information. ICIP,1996.
Uwe Meier, Rainer Stifelhagen, Jie Yang and Alex Waibel. Towards unrestricted lip reading.ICMI,1999.
Bourlard, H., Non-stationary multi-channel(multi-stream)processing towards robust and adaptive ASR,Proc.Tampere workshop on robust methods for speech recognition in aderse conditions,1995, pp.1–10.
Mirghafori, N.,A multi-band approach to automatic speech recognition, PhD dissertation, University of California at Berkely, Dec 1998.Reprinted as ICSI Technical report,ICSI TR-99-04.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ma, J., Gao, W., Wang, R. (2000). A Parallel Multistream Model for Integration of Sign Language Recognition and Lip Motion. In: Tan, T., Shi, Y., Gao, W. (eds) Advances in Multimodal Interfaces — ICMI 2000. ICMI 2000. Lecture Notes in Computer Science, vol 1948. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-40063-X_76
Download citation
DOI: https://doi.org/10.1007/3-540-40063-X_76
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41180-2
Online ISBN: 978-3-540-40063-9
eBook Packages: Springer Book Archive