research-article

Facial expression classification based on local spatiotemporal edge and texture descriptors

Authors:

Yulia Gizatdinova,

Veikko Surakka,

Roope RaisamoAuthors Info & Claims

MB '10: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research

Article No.: 21, Pages 1 - 4

https://doi.org/10.1145/1931344.1931365

Published: 24 August 2010 Publication History

Abstract

Facial expressions are emotionally, socially and otherwise meaningful reflective signals in the face. Facial expressions play a critical role in human life, providing an important channel of nonverbal communication. Automation of the entire process of expression analysis can potentially facilitate human-computer interaction, making it to resemble mechanisms of human-human communication. In this paper, we present an ongoing research that aims at development of a novel spatiotemporal approach to expression classification in video. The novelty comes from a new facial representation that is based on local spatiotemporal feature descriptors. In particular, a combined dynamic edge and texture information is used for reliable description of both appearance and motion of the expression. Support vector machines are utilized to perform a final expression classification. The planned experiments will further systematically evaluate the performance of the developed method with several databases of complex facial expressions.

References

[1]

Aleksic, P. S. and Katsaggelos, A. K. Automatic facial expression recognition using facial animation parameters and multi-stream HMMs. IEEE Trans. Information Forensics and Security 1, 1 (2005), 3.

Digital Library

[2]

Chang, C.-C. and Lin, C.-J. LIBSVM: A library for support vector machines (2001). http://www.csie.ntu.edu.tw/~cjlin/libsvm.

Digital Library

[3]

Ekman, P. and Friesen, W. Facial Action Coding System (FACS): A technique for the measurement of facial action. Consulting Psychologists Press, Palo Alto, California, 1978.

[4]

Fasel, B. and Luettin, J. Automatic facial expression analysis: A survey. J. Pattern Recognition 36, 1 (2003), 259--275.

[5]

Gizatdinova, Y. and Surakka, V. Feature-based detection of facial landmarks from neutral and expressive facial images. IEEE Trans. Pattern Analysis and Machine Intelligence 28, 1 (2006), 135--139.

Digital Library

[6]

Kotsia, I. and Pitas, I. Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Processing 16, 1 (2007), 172--187.

Digital Library

[7]

Lei, Z., Liao, S., He, R., Pietikäinen, M. and Li, SZ. Gabor volume based local binary pattern for face representation and recognition. In Proc. Int. Conf. Automatic Face and Gesture Recognition (2008), 1--6.

[8]

Liao, S., Fan, W., Chung, A. C. S., and Yeung, D.-Y. Facial expression recognition using advanced local binary patterns, tsallis entropies and global appearance features. In Proc. IEEE Int. Conf. Image Processing (2006), 665--668.

[9]

Mattivi, R. and Shao, L. Human action recognition using LBP-TOP as sparse spatio-temporal feature descriptor. Lecture Notes in Computer Science 5702. In Proc. Int. Conf. Computer Analysis of Images and Patterns (2009), 740--747.

Digital Library

[10]

Ojala, T., Pietikäinen, M. and Mäenpää, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Analysis and Machine Intelligence 24, 7 (2002), 971--987.

Digital Library

[11]

Pantic, M., Rothkrantz, J. Automatic analysis of facial expressions: The state of the art. IEEE Trans. Pattern Analysis and Machine Intelligence 22, 12 (2000), 1424--1445.

Digital Library

[12]

Viola, P. and Jones, M. Robust real-time face detection. Int. J. Computer Vision 57, 2 (2004), 137--154.

Digital Library

[13]

Wehrle, Th., Kaiser, S., Schmidt, S., Scherer Kl. R. Studying the dynamics of emotional expression using synthesized facial muscle movements. J. Personality and Social Psychology 78, 1 (2000), 105--119.

[14]

Zeng, Z., Pantic, M., Roisman, G. I. and Huang T. S. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Analysis and Machine Intelligence 31, 1 (2009), 39--58.

Digital Library

[15]

Zhao, G. and Pietikäinen, M. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Analysis and Machine Intelligence 29, 6 (2007), 915--928.

Digital Library

[16]

Zhang, Y. and Ji, Q. Active and dynamic information fusion for facial expression understanding from image sequence. IEEE Trans. Pattern Analysis and Machine Intelligence 27, 5 (2005), 699--714.

Digital Library

Cited By

Gowda SSuresh H(2021)A Survey on Methodologies and Database Used for Facial Emotion RecognitionAdvances in VLSI, Signal Processing, Power Electronics, IoT, Communication and Embedded Systems10.1007/978-981-16-0443-0_30(367-377)Online publication date: 11-Apr-2021
https://doi.org/10.1007/978-981-16-0443-0_30
Sergeeva ASablina V(2020)Eye Landmarks Detection Technology for Facial Micro-Expressions Analysis2020 9th Mediterranean Conference on Embedded Computing (MECO)10.1109/MECO49872.2020.9134338(1-4)Online publication date: Jun-2020
https://doi.org/10.1109/MECO49872.2020.9134338
Yağış EUnel M(2018)Facial Expression Based Emotion Recognition Using Neural NetworksImage Analysis and Recognition10.1007/978-3-319-93000-8_24(210-217)Online publication date: 6-Jun-2018
https://doi.org/10.1007/978-3-319-93000-8_24
Show More Cited By

Index Terms

Facial expression classification based on local spatiotemporal edge and texture descriptors
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Scene understanding
    2. Philosophical/theoretical foundations of artificial intelligence
  2. Computer graphics
    1. Image manipulation
      1. Image processing

Recommendations

Facial expression analysis and expression-invariant face recognition by manifold-based synthesis

Over the last decades, expression classification and face recognition have received substantial attention in computer vision and pattern recognition with more recent efforts focusing on understanding and modelling expression variations. In this paper, ...
Emotional facial expression interface: effects of displayed facial designs
Ergo'IA '10: Proceedings of the Ergonomie et Informatique Avancee Conference

In supporting the visual-mediated emotional recognition, little research has centred on analysing the effect of presenting different combinatorial facial designs. Moreover, in the theoretical field of emotions, research on the recognition of emotional ...
Automatic Facial Expression Recognition Using Boosted Discriminatory Classifiers
Analysis and Modeling of Faces and Gestures
Abstract
Over the last two decades automatic facial expression recognition has become an active research area. Facial expressions are an important channel of non-verbal communication, and can provide cues to emotions and intentions. This paper introduces a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

MB '10: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research

August 2010

183 pages

ISBN:9781605589268

DOI:10.1145/1931344

Editors:
Emilia Barakova
User Experiences Department, Philips Research Europe, The Netherlands
,
Boris de Ruyter
Faculty of Industrial Design, Eindhoven University of Technology, The Netherlands
,
Andrew Spink
Noldus Information Technology, The Netherlands

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Noldus Information Technology

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 August 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Measuring Behavior

Sponsor:

Measuring Behavior: Selected papers from the Proceedings of the 7th International Conference on Methods and Techniques in Behavioral research

August 24 - 27, 2010

Eindhoven, The Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
164
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gowda SSuresh H(2021)A Survey on Methodologies and Database Used for Facial Emotion RecognitionAdvances in VLSI, Signal Processing, Power Electronics, IoT, Communication and Embedded Systems10.1007/978-981-16-0443-0_30(367-377)Online publication date: 11-Apr-2021
https://doi.org/10.1007/978-981-16-0443-0_30
Sergeeva ASablina V(2020)Eye Landmarks Detection Technology for Facial Micro-Expressions Analysis2020 9th Mediterranean Conference on Embedded Computing (MECO)10.1109/MECO49872.2020.9134338(1-4)Online publication date: Jun-2020
https://doi.org/10.1109/MECO49872.2020.9134338
Yağış EUnel M(2018)Facial Expression Based Emotion Recognition Using Neural NetworksImage Analysis and Recognition10.1007/978-3-319-93000-8_24(210-217)Online publication date: 6-Jun-2018
https://doi.org/10.1007/978-3-319-93000-8_24
Kumari JRajesh RPooja K(2015)Facial Expression Recognition: A SurveyProcedia Computer Science10.1016/j.procs.2015.08.01158(486-491)Online publication date: 2015
https://doi.org/10.1016/j.procs.2015.08.011
Gizatdinova YSurakka VHaniff SMäkinen ERaisamo RIso-Tuisku JSand A(2013)Emerging application areas and challenges of automatic face analysisContinuum10.1080/10304312.2013.80330827:4(572-584)Online publication date: 14-Jun-2013
https://doi.org/10.1080/10304312.2013.803308
Gizatdinova YSpakov OSurakka V(2012)Face typingProceedings of the 2012 IEEE Workshop on the Applications of Computer Vision10.1109/WACV.2012.6162997(81-87)Online publication date: 9-Jan-2012
https://dl.acm.org/doi/10.1109/WACV.2012.6162997
Huang XZhao GPietikäinen MZheng W(2011)Expression recognition in videos using a weighted component-based feature descriptorProceedings of the 17th Scandinavian conference on Image analysis10.5555/2009594.2009657(569-578)Online publication date: 1-May-2011
https://dl.acm.org/doi/10.5555/2009594.2009657
Huang XZhao GPietikäinen MZheng W(2011)Expression Recognition in Videos Using a Weighted Component-Based Feature DescriptorImage Analysis10.1007/978-3-642-21227-7_53(569-578)Online publication date: 2011
https://doi.org/10.1007/978-3-642-21227-7_53

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten