Abstract
We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes of eyebrow, eye, and mouth. A fast method is proposed for locating facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points have been tailored for facial features in TSL. For example, the template matching methods have been simplified for tracking eyebrows and eyes. The mouth was tracked using the optical flow method, taking lips as homogeneous patches. The experiment has been conducted on 70 image sequences covering seven facial expressions. The average recognition rate is 83.3%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ekman, P.: Facial expression and emotion. American Psychologist 48(4), 384–392 (1993)
Ekman, P., Friesen, W.V.: Facial action coding system investigator’s guide. Consulting Psychologist Press, Palo Alto (1978)
Tian, Y., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Machine Intell. 23(2), 97–115 (2001)
Lien, J., Kanade, T., Cohn, J., Li, C.C.: Detection, tracking and classification of action units in facial expression. Journal of Robotics and Autonomous Systems 31(3), 131–146 (2000)
Yin, L., Basu, A.: Generating realistic facial expressions with wrinkles for model-based coding. Computer Vision and Image Understanding 84(2), 201–240 (2001)
Yuille, A., Cohen, D., Hallinan, P.: Feature extraction from faces using deformable templates. In: Proc. IEEE Conf. CVPR, June 1999, pp. 104–109 (1989)
Hara, F., Tanaka, K., Kobayashi, H., Tange, A.: Automatic feature extraction of facial organs and contour. In: Proc. IEEE Int’l Workshop on Robot and Human Communication, September 1, vol. 29, pp. 386–391 (October 1997)
Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)
Yin, L., Basu, A., Yourst, M.T.: Active tracking and cloning of facial expressions using spatio-temporal information. In: Proc. ICTAI 2002 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lin, SH., Hsieh, PF., Wu, CH. (2005). Facial Phoneme Extraction for Taiwanese Sign Language Recognition. In: Tao, J., Tan, T., Picard, R.W. (eds) Affective Computing and Intelligent Interaction. ACII 2005. Lecture Notes in Computer Science, vol 3784. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11573548_24
Download citation
DOI: https://doi.org/10.1007/11573548_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29621-8
Online ISBN: 978-3-540-32273-3
eBook Packages: Computer ScienceComputer Science (R0)