Abstract
In this paper, an emotion recognition from facial temporal sequence has been proposed. Firstly, the temperature difference histogram features and five statistical features are extracted from the facial temperature difference matrix of each difference frame in the data sequences. Then the discrete Hidden Markov Models are used as the classifier for each feature. In which, a feature selection strategy based on the recognition results in the training set is introduced. Finally, the results of the experiments on the samples of the USTC-NVIE database demonstrate the effectiveness of our method. Besides, the experiment results also demonstrate that the temperature information of the forehead is more useful than that of the other regions in emotion recognition and understanding, which is consistent with some related research results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wang, S., Liu, Z., Lv, S., Lv, Y., Wu, G., Peng, P., Chen, F., Wang, X.: A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference. IEEE Transactions on Multimedia, 682–691 (2010)
Cohen, I., Garg, A., Huang, T.S.: Emotion recognition from facial expressions using multilevel HMM. In: NIPS (2000)
Hidden Markov Model Toolbox for Matlab, http://www.cs.ubc.ca/~murphyk/Software/HMM/hmm.html
Nefian, A.V., Liang, L., Pi, X., Xiaoxiang, L., Mao, C., Murphy, K.: A coupled HMM for audio-visual speech recognition. In: International Conference on Acoustics, Speech and Signal Processing CASSP 2002, pp. 2013–2016 (2002)
Puri, C., Olson, L., Pavlidis, I., Levine, J., Starren, J.: StressCam: non-contact measurement of users’ emotional states through thermal imaging. In: CHI Extended Abstracts, pp. 1725–1728 (2005)
Nhan, B.R., Chau, T.: Classifying affective states using thermal infrared imaging of the human face. IEEE Transaction on Biomedical Engineering 57(4), 979–987 (2010)
Gunes, H., Pantic, M.: Automatic, Dimensional and Continuous Emotion Recognition. International Journal of Synthetic Emotions 1(1), 68–99 (2010)
Pavlidis, I., Levine, J.: Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine 21(6), 56–64 (2002)
Yoshitomi, Y.: Facial Expression Recognition for Speaker Using Thermal Image Processing and Speech Recognition System. In: Proc. of 10th WSEAS International Conference on Applied Computer Science, pp. 182–186 (2010)
Khan, M.M.: Cluster-analytic classification of facial expressions using infrared measurements of facial thermal features. Ph.D. Thesis, Department of Computing and Engineering, University of Huddersfield, Huddersfield, UK (2008)
Khan, M.M., Ward, R.D., Ingleby, M.: Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature. Trans. Appl. Percept. 6, 1 (2009)
Merla, A., Romani, G.L.: Thermal signatures of emotional arousal: A functional infrared imaging study. In: IEEE 29th Annu. Int. Conf., pp. 247–249 (2007)
Calvo, R.A., D’Mello, S.: Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications. IEEE Transactions on Affective Computing 1(1), 18–37 (2010)
Hernández, B., Olague, G., Hammoud, R., Trujillo, L., Romero, E.: Visual learning of texture descriptors for facial expression recognition in thermal imagery. Computer Vision and Image Understanding 106(2-3), 258–269 (2007)
Jiang, G., Song, X., Zheng, F., Wang, P., Omer, A.M.: Facial Expression Recognition Using Thermal Image. In: 27th Annual International Conference of the Engineering in Medicine and Biology Society, IEEE-EMBS 2005, pp. 631–633, 17–18 (2006)
Bassili, J.N.: Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. J. Personality Social Psychology 37, 2049–2058 (1979)
Merla, A., Romani, G.L.: Thermal signatures of emotional arousal: A functional infrared imaging study. In: Proc. IEEE 29th Annu. Int. Conf., pp. 247–249 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Z., Wang, S. (2011). Emotion Recognition Using Hidden Markov Models from Facial Temperature Sequence. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6975. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24571-8_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-24571-8_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24570-1
Online ISBN: 978-3-642-24571-8
eBook Packages: Computer ScienceComputer Science (R0)