skip to main content
10.1145/3220511.3220518acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmvaConference Proceedingsconference-collections
research-article

Deep Learning for Real-Time Robust Facial Expression Analysis

Published: 23 April 2018 Publication History

Abstract

The aim of this investigation is to classify real-life facial images into one of six types of emotions. For solving this problem, we propose to use deep machine learning algorithms and convolutional neural network (CNN). CNN is a modern type of neural network, which allows for rapid detection of various objects, as well as to make an effective object classification. For acceleration of CNN learning stage, we use supercomputer NVIDIA DGX-1. This process was implemented in parallel on a large number of independent streams on GPU. Numerical experiments for algorithms were performed on the images of Multi-Pie image database with various lighting of scene and angle rotation of head. For developed models, several metrics of quality were calculated. The designing algorithm was used in real-time video processing in human-computer interaction systems. Moreover, expression recognition can apply in such fields as retail analysis, security, video games, animations, psychiatry, automobile safety, educational software, etc.

References

[1]
Ekman P. and Friesen W.V. 1977. Manual for the Facial Action Coding System, Consulting Psychologists Press.
[2]
Bettadapura V. 2012. Face Expression Recognition and Analysis: The State of the Art, Tech Report, arXiv: 1203.6722.
[3]
Emotion Recognition in the Wild Challenge. https://sites.google.com/site/emotiwchallenge/
[4]
Challenges in Representation Learning: Facial Expression Recognition Challenge.https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/
[5]
Berns K., Hirth J. 2006. Control of Facial expressions of the humanoid robot head roman. In: IEEE International Conference on Intelligent Robots and Systems, pp. 3119--3124.
[6]
Bartlett M.S., Littlewort G., Fasel I., Movellan J.R. 2003. Real time face detection and facial expression recognition: development and applications to human computer interaction. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 53.
[7]
Ucar A. 2017. Deep Convolutional Neural Networks for facial expression recognition. INnovations in Intelligent SysTems and Applications (INISTA), IEEE International Conference on, pp. 371--375.
[8]
Niu Z., Zhou M., Wang L., Gao X., Hua G. 2016. Ordinal Regression with Multiple Output CNN for Age Estimation. IEEE Conference on Computer Vision and Pattern Recognition, pp. 4920--4928.
[9]
Paisitkriangkrai S., Sherrah J., Janney P., Van-Den Hengel A. 2015. Effective Semantic Pixel labeling with Convolutional Networks and Conditional Random Fields. IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 36--43.
[10]
Raghuvanshi A., Choksi V. 2017. Facial Expression Recognition with Convolutional Neural Networks, Tech Report.
[11]
Khalajzadeh H., Mansouri M., Teshnehlab M. 2013. Face Recognition using Convolutional Neural Network and Simple Logistic Classifier, In: Soft Computing in Industrial Applications. Advances in Intelligent Systems and Computing, vol. 223. Springer, Cham, pp. 197--207.
[12]
Wen Z., Huang T. 2003. Capturing subtle facial motions in 3d face tracking. Proceedings of the 9th IEEE International Conference on Computer Vision, pp. 1343--1350.
[13]
Alizadeh S., Fazel A. 2017. Convolutional Neural Networks for Facial Expression Recognition. arXiv: 1704.06756.
[14]
Yu Z. and Zhang C. 2015. Image based static facial expression recognition with multiple deep network learning. Proceedings of the 2015 ACM International Conference Multimodal Interaction, pp. 435--442.
[15]
Bo-Kyeong Kim S.-Y.D., Roh J. and Lee S.-Y. 2015. Hierarchical committee of deep convolutional neural networks for robust facial expression recognition. Journal on Multimodal User Interfaces, vol. 10, pp. 173--189.
[16]
Ali Mollahosseini D.C. and Mahoor M.H. 2016. Going deeper in facial expression recognition using deep neural networks. IEEE Winter Conference on Applications of Computer Vision, pp. 1--10.
[17]
The Binghamton University 3D Facial Expression (BU-3DFE) Database. http://www.cs.binghamton.edu/~lijun/Research/3DFE.
[18]
The Multimedia Understanding Group (MUG) Facial Expression Database. https://mug.ee.auth.gr/fed.
[19]
The CMU Multi-PIE Face Database. http://www.cs.cmu.edu/afs/cs/project/PIE/MultiPie/Multi-Pie/Home.html.
[20]
Ivanovsky L., Khryashchev V., Lebedev A., Kosterin I. 2017. Facial Expression Recognition Algorithm Based on Deep Convolution Neural Network. In Proceedings of the 21th Conference of Open Innovations Association FRUCT'21. Helsinki, Finland.

Cited By

View all
  • (2024)Expressive 3D Facial Animation Generation Based on Local-to-Global Latent DiffusionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345621330:11(7397-7407)Online publication date: 1-Nov-2024

Index Terms

  1. Deep Learning for Real-Time Robust Facial Expression Analysis

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICMVA '18: Proceedings of the International Conference on Machine Vision and Applications
    April 2018
    81 pages
    ISBN:9781450363815
    DOI:10.1145/3220511
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    In-Cooperation

    • University of Canberra: University of Canberra

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 April 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Deep learning
    2. convolutional neural network
    3. facial expression analysis
    4. real-time video analysis

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICMVA 2018

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)4
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Expressive 3D Facial Animation Generation Based on Local-to-Global Latent DiffusionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345621330:11(7397-7407)Online publication date: 1-Nov-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media