research-article

Facial expression recognition using geometric and appearance features

Authors:

Lizhe WangAuthors Info & Claims

ICIMCS '12: Proceedings of the 4th International Conference on Internet Multimedia Computing and Service

Pages 29 - 33

https://doi.org/10.1145/2382336.2382345

Published: 09 September 2012 Publication History

Abstract

A novel method using hybrid geometric and appearance features of the difference between the neutral and fully expressive facial expression images is proposed for facial expression recognition in this paper. The difference tends to emphasize the facial parts that are changed from the neutral to expressive face and eliminate in that way the identity of the facial image. The hybrid features include facial feature point displacements and local texture differences between the normalized neutral and expressive facial expression images. The proposed method achieved an average accuracy of 95% in the extended Cohn-Kanade database with a Support Vector Machine (SVM) classification method.

References

[1]

Anderson, K. and McOwan, P. W. 2006. A real-time automated system for the recognition of human facial expressions. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 36(1), (2006), 96--105.

Digital Library

[2]

Bartlett, M. S.,. Littlewort, G., Fasel, I., and Movellan, R. 2003. Real time face detection and facial expression recognition: development and application to human computer interaction. Proceedings of the CVPR Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction, vol. 5, (2003).

[3]

Vural, E., Cetin, M., Ercil, A., Littlewort, G., Bartlett, M., and Movellan, J. 2008. Automated drowsiness detection for improved driver safety-comprehensive databases for facial expression analysis. Proceedings of the International Conference on Automotive Technologies, (2008).

[4]

Maja Pantic, Z., Glenn, I. R., and Thomas, S. H. 2009. A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), (2009), 39--58.

Digital Library

[5]

Pantic, M., Leon, J., and Rothkrantz, M. 2004. Facial action recognition for facial expression analysis from static face images. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 34(3), (2004), 1449--1460.

Digital Library

[6]

Valstar, M. F., Gunes, H., and Pantic, M. 2007. How to distinguish posed from spontaneous smiles using geometric features. Proceedings of the Ninth ACM Int'l Conf. Multimodal Interfaces (ICMI '07), (2007), 38--45.

Digital Library

[7]

Kotsia, I. and Pitas, I. 2007. Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transactions on Image Processing, 16(1), (2007), 172--187.

Digital Library

[8]

Lu, K. and Zhang, X. 2010. Facial expression recognition from image sequences based on feature points and canonical correlations. Proceedings of the Artificial Intelligence and Computational Intelligence (AICI), 2010 International Conference, vol. 1, (2010), 219--223.

Digital Library

[9]

Lei, G., Li, X., Zhou, J., and Gong, X. 2009. Geometric feature based facial expression recognition using multiclass support vector machines. Proceedings of the IEEE International Conference on Granular Computing, (2009), 318--321.

[10]

Valstar, M. F. and Pantic, M. 2012. Fully automatic recognition of the temporal phases of facial actions. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 42(1), (2012), 28--43.

Digital Library

[11]

Liu, W. and Wang, Z. 2006. Facial expression recognition based on fusion of multiple Gabor features. Proceedings of the 18th International Conference on Pattern Recognition, ICPR 2006, vol. 3, (2006), 536--539.

Digital Library

[12]

Bartlett, M. S., Littlewort, G., Frank, M., Lainscse, C., Fasel, I., and Movellan, J. 2006. Fully automatic facial action recognition in spontaneous behavior. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, FGR 2006, (2006), 223--230.

Digital Library

[13]

Ekman, P. and Friesen, W. V. 1978. Facial action coding system: Manual. Palo Alto: Consulting Psychologist Press, (1978).

[14]

Whitehill. J. and Omlin, C. W. 2006. Haar features for FACS AU recognition. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, (2006), 97--101.

Digital Library

[15]

Chang, Y., Hu, C., and Turk, M. 2004. Probabilistic expression analysis on manifolds. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR '04), vol. 2, (2004),520--527.

Digital Library

[16]

Ying, Z., Wang, Z., and Huang, M. 2010. Advanced intelligent computing theories and applications. With Aspects of Artificial Intelligence, Lecture Notes in Computer Science, 6216, (2010), 457--464.

Digital Library

[17]

Pantic, M. and Patras, I. 2006. Dynamics of facial expression: Recognition of facial actions and their temporal segments form face profile image sequences. IEEE Transactions Systems, Man, and Cybernetics, Part B, 36(2), (2006), 433--449.

Digital Library

[18]

Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. 2010. The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. Proceedings of 2010 IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), (2010), 94--101.

[19]

Mahoor, M. H., Zhou, M., Veon, K. L., Mavadati, S. M., and Cohn, J. F. 2011. Facial action unit recognition with sparse representation. Proceedings of 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), (2011), 336--342.

[20]

Kotsia, I., Zafeiriou, S., Nikolaidis, N., and Pitas, I. 2008. Texture and shape information fusion for facial action unit recognition. Proceedings of 2008 First International Conference on Advances in Computer-Human Interaction, (2008), 77--82.

Digital Library

[21]

Tian, Y. L., Kanade, T., and Cohn, J. F. 2005. Facial expression analysis. Handbook of Face Recognition, S. Z. Li and A. K. Jain, eds., Springer, (2005), 247--276.

[22]

Zhang, Y. and Ji, Q. 2005. Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), (2005), 699--714.

Digital Library

[23]

Viola, P. and Jones, M. 2001. Robust real-time object detection. Proceedings of the 2nd International Workshop on Statistical and Computational Theories of Vision, Modeling, Learning, Computing and Sampling; Vancouver, Canada, (2001).

[24]

Milborrow, S. and Nicolls, F. 2008. Locating facial features with an extended active shape model. Proceedings of the 10th European Conference on Computer Vision: Part IV (2008), 504--513.

Digital Library

[25]

Cootes, T. F., Taylor, C. J., Cooper, D. H., and Graham, J. 1995. Active shape models-their training and application. CVIU 61, (1995),38--59.

Digital Library

[26]

Li, L. and Leung, M. K. H. 2002. Integrating intensity and texture differences for robust change detection. IEEE Transactions on Image Processing, 11(2), (2002, 105--112.

Digital Library

[27]

Chang, C.-C. and Lin, C.-J. 2011. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 27(2), (2011) pp. 1--27.

Digital Library

[28]

Kanade, T., Cohn, J., and Tian, Y. 2000. Comprehensive database for facial expression analysis. Proceedings of the International Conference on Automatic Face and Gesture Recognition, (2000), 46--53.

Digital Library

[29]

Zafeiriou, S. and Petrou, M. 2010. Sparse representations for facial expressions recognition via 11 optimization. Proceedings of 2010 IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), (2010), 32--39.

[30]

Koelstra, S. and Pantic, M. 2008. Non-rigid registration using free-form deformations for recognition of facial actions and their temporal dynamics. Proceedings of 2008 International Conference on Automatic Face and Gesture Recognition (2008), 1--8.

[31]

Jain, S., Hu, C. and Aggrwal, J., Facial expression recognition with temporal modeling of shapes. Proceedings of 2011 IEEE International Conference on Computer Vision workshops, 1642--1649.

Cited By

Chatterjee SSaha DSen SOliva DSarkar R(2024)Moth-flame optimization based deep feature selection for facial expression recognition using thermal imagesMultimedia Tools and Applications10.1007/s11042-023-15861-583:4(11299-11322)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11042-023-15861-5
Gupta SGupta SYadav ATrivedi S(2023)Intelligent Face Emotion Detection Application for Blind People2023 International Conference on Computational Intelligence, Communication Technology and Networking (CICTN)10.1109/CICTN57981.2023.10141200(108-115)Online publication date: 20-Apr-2023
https://doi.org/10.1109/CICTN57981.2023.10141200
Sarvakar KSenkamalavalli RRaghavendra SSantosh Kumar JManjunath RJaiswal S(2023)Facial emotion recognition using convolutional neural networksMaterials Today: Proceedings10.1016/j.matpr.2021.07.29780(3560-3564)Online publication date: 2023
https://doi.org/10.1016/j.matpr.2021.07.297
Show More Cited By

Recommendations

Facial expression recognition based on local region specific features and support vector machines

Facial expressions are one of the most powerful, natural and immediate means for human being to communicate their emotions and intensions. Recognition of facial expression has many applications including human-computer interaction, cognitive science, ...
Expression-invariant face recognition by facial expression transformations

In this paper, we present a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. When a new face image with an arbitrary expression is ...
Automatic Facial Expression Recognition Using Combined Geometric Features

This study presents a geometric feature based automatic facial expression recognition system. The proposed system utilises the facial landmark points to determine the relative distances between the facial features in order to capture deformities caused ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICIMCS '12: Proceedings of the 4th International Conference on Internet Multimedia Computing and Service

September 2012

243 pages

ISBN:9781450316002

DOI:10.1145/2382336

General Chairs:
Xinguo Yu
Institute for Infocomm Research, Singapore
,
Rainer Lienhart
University of Augsburg, Germany
,
Program Chairs:
Zheng-Jun Zha
National University of Singapore
,
Yan Liu
Hong Kong Polytechnic University
,
Shin'ichi Satoh
National Institute of Informatics, Japan

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

National Science Foundation of China
CCNU: Central China Normal University
Daqian Vision: Daqian Vision
Microsoft Research: Microsoft Research
Beijing ACM SIGMM Chapter
NEC: NEC Labs China

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ICIMCS '12

Sponsor:

CCNU
Daqian Vision
Microsoft Research
NEC

ICIMCS '12: The 4th International Conference on Internet Multimedia Computing and Service

September 9 - 11, 2012

Wuhan, China

Acceptance Rates

Overall Acceptance Rate 163 of 456 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
573
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chatterjee SSaha DSen SOliva DSarkar R(2024)Moth-flame optimization based deep feature selection for facial expression recognition using thermal imagesMultimedia Tools and Applications10.1007/s11042-023-15861-583:4(11299-11322)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11042-023-15861-5
Gupta SGupta SYadav ATrivedi S(2023)Intelligent Face Emotion Detection Application for Blind People2023 International Conference on Computational Intelligence, Communication Technology and Networking (CICTN)10.1109/CICTN57981.2023.10141200(108-115)Online publication date: 20-Apr-2023
https://doi.org/10.1109/CICTN57981.2023.10141200
Sarvakar KSenkamalavalli RRaghavendra SSantosh Kumar JManjunath RJaiswal S(2023)Facial emotion recognition using convolutional neural networksMaterials Today: Proceedings10.1016/j.matpr.2021.07.29780(3560-3564)Online publication date: 2023
https://doi.org/10.1016/j.matpr.2021.07.297
Saeed SShah AEhsan MAmirzada MMahmood AMezgebo T(2022)Automated Facial Expression Recognition Framework Using Deep LearningJournal of Healthcare Engineering10.1155/2022/57079302022(1-11)Online publication date: 31-Mar-2022
https://doi.org/10.1155/2022/5707930
Nurrahma Rosanti Paidja ABachtiar F(2022)Engagement Emotion Classification through Facial Landmark Using Convolutional Neural Network2022 2nd International Conference on Information Technology and Education (ICIT&E)10.1109/ICITE54466.2022.9759546(234-239)Online publication date: 22-Jan-2022
https://doi.org/10.1109/ICITE54466.2022.9759546
Sidhom OGhazouani HBarhoumi W(2022)Subject-dependent selection of geometrical features for spontaneous emotion recognitionMultimedia Tools and Applications10.1007/s11042-022-13380-382:2(2635-2661)Online publication date: 1-Jul-2022
https://doi.org/10.1007/s11042-022-13380-3
Hussein HDino HMstafa RHassan M(2022)Person-independent facial expression recognition based on the fusion of HOG descriptor and cuttlefish algorithmMultimedia Tools and Applications10.1007/s11042-022-12438-681:8(11563-11586)Online publication date: 18-Feb-2022
https://doi.org/10.1007/s11042-022-12438-6
Ghazouani H(2021)A genetic programming-based feature selection and fusion for facial expression recognitionApplied Soft Computing10.1016/j.asoc.2021.107173103:COnline publication date: 1-May-2021
https://dl.acm.org/doi/10.1016/j.asoc.2021.107173
Rajan SChenniappan PDevaraj SMadian N(2020)Novel deep learning model for facial expression recognition based on maximum boosted CNN and LSTMIET Image Processing10.1049/iet-ipr.2019.118814:7(1373-1381)Online publication date: 29-Apr-2020
https://doi.org/10.1049/iet-ipr.2019.1188
Pebiana SWidyanto MBasaruddin TLiliana D(2019)Enhancing Facial Component AnalysisProceedings of the 2nd International Conference on Software Engineering and Information Management10.1145/3305160.3305174(175-179)Online publication date: 10-Jan-2019
https://dl.acm.org/doi/10.1145/3305160.3305174
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten