research-article

Facial emotion recognition: A comparative analysis using 22 LBP variants

Authors:

Y. RuichekAuthors Info & Claims

MedPRAI '18: Proceedings of the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence

Pages 88 - 94

https://doi.org/10.1145/3177148.3180092

Published: 27 March 2018 Publication History

Abstract

Facial expression is a significant form of non-verbal communication for human being. It includes much important information about the feeling, the mental and the emotional state of a person which can be useful in several real-world applications and fields like image processing and computer vision. Face can be seen as a composition of micro-patterns of textures. Over the last decades, LBP operator, which shown its robustness in extracting useful features characteristics from an image, has been successfully applied in diverse range of problems including facial expression recognition. Nowadays, many LBP variants have been proposed in the literature. This paper reviews 22 LBP-like descriptors and provides a comparative analysis on facial expression recognition problem using two benchmark databases. the Japanese female facial expression (JAFFE) and Cohn-Kanade (CK) databases. The experiments show that several of the evaluated methods achieve performances that are better than those recorded by the state-of-the-art systems. Recognition rates of 97.14% and 100% have been reached on JAFFE and Cohn-Kanade databases respectively.

References

[1]

E. A. Boyle, A. H. Anderson and A. Newlands, "The effects of visibility on dialogue and performance in a cooperative problem solving task," Language and speech, vol. 37, pp. 1--20, 1994.

[2]

K. Slimani, R. Messoussi, S. Bourekkadi and S. Khoulji, "An intelligent system solution for improving the distance collaborative work," in 2017 Intelligent Systems and Computer Vision (ISCV), 2017.

[3]

A. Mehrabian, "Communication without words," Communication Theory, pp. 193--200, 2008.

[4]

S. Bourekkadi, S. Khoulji, K. Slimani, R. Messoussi and M. L. Kerkeb, "The Design Of A Psychotherapy Remote Intelligent System," Journal of Theoretical and Applied Information Technology, vol. 93, p. 116, 2016.

[5]

P. Ekman, "An argument for basic emotions," Cognition & emotion, vol. 6, pp. 169--200, 1992.

[6]

T. Ojala, M. Pietikainen and T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on pattern analysis and machine intelligence, vol. 24, pp. 971--987, 2002.

Digital Library

[7]

T. Jabid, M. H. Kabir and O. Chae, "Local directional pattern (LDP) for face recognition," in Consumer Electronics (ICCE), 2010 Digest of Technical Papers International Conference on, 2010.

[8]

E. Abusham and H. Bashir, "Face recognition using local graph structure (LGS)," Human-Computer Interaction. Interaction Techniques and Environments, pp. 169--175, 2011.

Digital Library

[9]

X. Tan and B. Triggs, "Enhanced local texture feature sets for face recognition under difficult lighting conditions," IEEE transactions on image processing, vol. 19, pp. 1635--1650, 2010.

Digital Library

[10]

H. K. Bashier, L. S. Hoe, L. T. Hui, M. F. Azli, P. Y. Han, W. K. Kwee and M. S. Sayeed, "Texture classification via extended local graph structure," Optik-International Journal for Light and Electron Optics, vol. 127, pp. 638--643, 2016.

[11]

K. Song, Y. Yan, Y. Zhao and C. Liu, "Adjacent evaluation of local binary patter for texture classification," Journal of Visual Communication and Imag Representation, vol. 33, pp. 323--339, 2015.

Digital Library

[12]

W. Yang, Z. Wang and B. Zhang, "Face recognition using adaptive local ternar patterns method," Neurocomputing, vol. 213, pp. 183--190, 2016.

Digital Library

[13]

A. Fernández, M. X. Álvarez and F. Bianconi, "Image classification with binar gradient contours," Optics and Lasers in Engineering, vol. 49, pp. 1177--118, 2011.

[14]

Y. Kaya, Ö. F. Ertuğrul and R. Tekin, "Two novel local binary pattern descriptor for texture analysis," Applied Soft Computing, vol. 34, pp. 728--735, 2015.

Digital Library

[15]

R. Mehta and K. Egiazarian, "Dominant rotated local binary patterns (DRLBP for texture classification," Pattern Recognition Letters, vol. 71, pp. 16--22, 2016.

Digital Library

[16]

A. Fernández, M. X. Álvarez and F. Bianconi, "Texture description throug histograms of equivalent patterns," Journal of mathematical imaging and visio vol. 45, pp. 76--102, 2013.

Digital Library

[17]

L. Nanni, S. Brahnam and A. Lumini, "A local approach based on a Local Binar Patterns variant texture descriptor for classifying pain states," Expert System with Applications, vol. 37, pp. 7888--7894, 2010.

Digital Library

[18]

A. R. Rivera, J. R. Castillo and O. O. Chae, "Local directional number pattern for face analysis: Face and expression recognition," IEEE transactions on imag processing, vol. 22, pp. 1740--1752, 2013.

Digital Library

[19]

S. K. Vipparthi and S. K. Nagar, "Local extreme complete trio pattern for multimedia image retrieval system," International Journal of Automation and Computing, vol. 13, pp. 457--467, 2016.

Digital Library

[20]

Y. Zhao, D.-S. Huang and W. Jia, "Completed local binary count for rotation invariant texture classification," IEEE transactions on image processing, vol. 21: pp. 4492--4497, 2012.

Digital Library

[21]

Y. Zhao, R.-G. Wang, W.-M. Wang and W. Gao, "Local Quantization Code histogram for texture classification," Neurocomputing, vol. 207, pp. 354--364 2016.

Digital Library

[22]

B. Xu, P. Gong, E. Seto and R. Spear, "Comparison of gray-level reduction an different texture spectrum encoding methods for land-use classification using panchromatic IKONOS image," Photogrammetric Engineering \& Remot Sensing, vol. 69, pp. 529--536, 2003.

[23]

C. Silva, T. Bouwmans and C. Frélicot, "An extended center-symmetric local binary pattern for background modeling and subtraction in videos," in International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISAPP 2015, 2015.

[24]

J. Chen, S. Shan, C. He, G. Zhao, M. Pietikainen, X. Chen and W. Gao, "WLD: A robust local image descriptor," IEEE transactions on pattern analysis an machine intelligence, vol. 32, pp. 1705--1720, 2010.

Digital Library

[25]

M. Lyons, S. Akamatsu, M. Kamachi and J. Gyoba, "Coding facial expression with gabor wavelets," in Automatic Face and Gesture Recognition, 1998 Proceedings. Third IEEE International Conference on, 1998.

Digital Library

[26]

T. Kanade, J. F. Cohn and Y. Tian, "Comprehensive database for facial expression analysis," in Automatic Face and Gesture Recognition, 2000. Proceedings Fourth IEEE International Conference on, 2000.

Digital Library

[27]

Z. Wang and Q. Ruan, "Facial expression recognition based orthogonal local fisher discriminant analysis," in Signal Processing (ICSP), 2010 IEEE 10th International Conference on, 2010.

[28]

S. Zhang, X. Zhao and B. Lei, "Facial expression recognition based on local binary patterns and local fisher discriminant analysis," WSEAS transactions on signal processing, vol. 8, pp. 21--31, 2012.

Digital Library

[29]

S. K. Gupta, S. Agrwal, Y. K. Meena and N. Nain, "A hybrid method of featur extraction for facial expression recognition," in Signal-Image Technology an Internet-Based Systems (SITE), 2011 Seventh International Conference on 2011.

Digital Library

[30]

X. Zhao and S. Zhang, "Facial expression recognition based on local binary patterns and kernel discriminant isomap," Sensors, vol. 11, pp. 9573--9588, 2011

[31]

L. Zhang and D. Tjondronegoro, "Facial expression recognition using facial movement features," IEEE Transactions on Affective Computing, vol. 2, pp. 219--229,2011.

Digital Library

[32]

W. F. Gu, Y. V. Venkatesh and C. Xiang, "A novel application of self-organizing network for facial expression recognition from radial encoded contours," Soft Computing, vol. 14, pp. 113--122, 2010.

Digital Library

[33]

Y. Rahulamathavan, R. C.-W. Phan, J. A. Chambers and D. J. Parish, "Facial expression recognition in the encrypted domain based on local fisher discriminant analysis," IEEE Transactions on Affective Computing, vol. 4, pp. 83--92, 2013.

Digital Library

[34]

V. Gomathi, K. Ramar and A. S. Jeevakumar, "Human facial expression recognition using MANFIS model," World Academy of Science, Engineering and Technology, vol. 50, p. 2009, 2009.

[35]

S. Chatterjee and H. Shi, "A novel neuro fuzzy approach to human emotion determination," in Digital Image Computing: Techniques and Applications (DICTA), 2010 International Conference on, 2010.

Digital Library

[36]

Y. Tong, R. Chen and Y. Cheng, "Facial expression recognition algorithm using LGC based on horizontal and diagonal prior principle," Optik-Internationl Journal for Light and Electron Optics, vol. 125, pp. 4186--4189, 2014.

[37]

C. Shan, S. Gong and P. W. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study," Image and Vision Computing, vol. 27, pp. 803--816, 2009.

Digital Library

[38]

H. Ali, M. Hariharan, S. Yaacob and A. H. Adorn, "Facial emotion recognition using empirical mode decomposition," Expert Systems with Applications, vol. 42, pp. 1261--1277, 2015.

Digital Library

Cited By

Sham ATikka PLamas DAnbarjafari G(2024)Synthesizing facial expressions in dyadic human–robot interactionSignal, Image and Video Processing10.1007/s11760-024-03202-418:S1(909-918)Online publication date: 11-May-2024
https://doi.org/10.1007/s11760-024-03202-4
Jorge Ada Cunha MSoares A(2023)Genetic Algorithms in Machine Learning Applied to Computer Vision: Facial Emotion RecognitionIX Latin American Congress on Biomedical Engineering and XXVIII Brazilian Congress on Biomedical Engineering10.1007/978-3-031-49401-7_12(118-128)Online publication date: 17-Dec-2023
https://doi.org/10.1007/978-3-031-49401-7_12
Mukhopadhyay MDey AGhosh AShaw R(2023)Facial Emotion Recognition Based on Textural Pattern and Histogram of Oriented GradientAdvanced Communication and Intelligent Systems10.1007/978-3-031-25088-0_9(111-119)Online publication date: 15-Feb-2023
https://doi.org/10.1007/978-3-031-25088-0_9
Show More Cited By

Recommendations

Local dominant binary patterns for recognition of multi-view facial expressions
ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing

In this paper, a novel framework is proposed for automatic recognition of facial expressions. However, the face images for the proposed problem are captured at multiple view angle (i.e., multi-view facial expressions). The proposed scheme introduces a ...
Patterns of Weber magnitude and orientation for uncontrolled face representation and recognition

Robust, discriminative and computationally efficient feature extraction is vital for a successful real-world face recognition system. To this end, we present a novel local feature descriptor, named patterns of Weber magnitude and orientation (PWMO), for ...
Feature Representation for Facial Expression Recognition Based on FACS and LBP

In expression recognition, feature representation is critical for successful recognition since it contains distinctive information of expressions. In this paper, a new approach for representing facial expression features is proposed with its objective ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

MedPRAI '18: Proceedings of the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence

March 2018

135 pages

ISBN:9781450352901

DOI:10.1145/3177148

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IAPR: International Association for Pattern Recognition

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

MedPRAI '18

MedPRAI '18: Mediterranean Conference on Pattern Recognition and Artificial Intelligence

March 27 - 28, 2018

Rabat, Morocco

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
206
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sham ATikka PLamas DAnbarjafari G(2024)Synthesizing facial expressions in dyadic human–robot interactionSignal, Image and Video Processing10.1007/s11760-024-03202-418:S1(909-918)Online publication date: 11-May-2024
https://doi.org/10.1007/s11760-024-03202-4
Jorge Ada Cunha MSoares A(2023)Genetic Algorithms in Machine Learning Applied to Computer Vision: Facial Emotion RecognitionIX Latin American Congress on Biomedical Engineering and XXVIII Brazilian Congress on Biomedical Engineering10.1007/978-3-031-49401-7_12(118-128)Online publication date: 17-Dec-2023
https://doi.org/10.1007/978-3-031-49401-7_12
Mukhopadhyay MDey AGhosh AShaw R(2023)Facial Emotion Recognition Based on Textural Pattern and Histogram of Oriented GradientAdvanced Communication and Intelligent Systems10.1007/978-3-031-25088-0_9(111-119)Online publication date: 15-Feb-2023
https://doi.org/10.1007/978-3-031-25088-0_9
Li SGuo LLiu J(2022)Towards East Asian Facial Expression Recognition in the Real World: A New Database and Deep Recognition BaselineSensors10.3390/s2221808922:21(8089)Online publication date: 22-Oct-2022
https://doi.org/10.3390/s22218089
Dresvyanskiy DRyumina EKaya HMarkitantov MKarpov AMinker W(2022)End-to-End Modeling and Transfer Learning for Audiovisual Emotion Recognition in-the-WildMultimodal Technologies and Interaction10.3390/mti60200116:2(11)Online publication date: 27-Jan-2022
https://doi.org/10.3390/mti6020011
Mukhopadhyay MDey AKahali S(2022)A deep-learning-based facial expression recognition method using textural featuresNeural Computing and Applications10.1007/s00521-022-08005-735:9(6499-6514)Online publication date: 22-Nov-2022
https://doi.org/10.1007/s00521-022-08005-7
Boughanem HGhazouani HBarhoumi W(2022)Multichannel convolutional neural network for human emotion recognition from in-the-wild facial expressionsThe Visual Computer10.1007/s00371-022-02690-039:11(5693-5718)Online publication date: 21-Oct-2022
https://doi.org/10.1007/s00371-022-02690-0
Sun JLv YTang CSima HWu X(2020)Face Recognition Based on Local Gradient Number Pattern and Fuzzy Convex-Concave PartitionIEEE Access10.1109/ACCESS.2020.29753128(35777-35791)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2975312
Canedo DNeves A(2019)Facial Expression Recognition Using Computer Vision: A Systematic ReviewApplied Sciences10.3390/app92146789:21(4678)Online publication date: 2-Nov-2019
https://doi.org/10.3390/app9214678
Slimani KLekdioui KMessoussi RTouahni R(2019)Compound Facial Expression Recognition Based on Highway CNNProceedings of the New Challenges in Data Sciences: Acts of the Second Conference of the Moroccan Classification Society10.1145/3314074.3314075(1-7)Online publication date: 28-Mar-2019
https://dl.acm.org/doi/10.1145/3314074.3314075

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten