Emmanuel Dufourq
ABSTRACT
Humans are generally good at recognising emotions which are portrayed on another person’s face. Can the same be said for machines? In recent years, there has been a tremendous amount of progress in the field of computer vision using deep learning methods, namely by Convolutional Neural Networks (CNNs). How good are these CNNs at recognising facial expressions? With the explosion of research outputs using CNN for Facial Expression Recognition (FER) in recent years, it is an appropriate time to review the state of the art in this field, provide a critical analysis of what has and has not been achieved, and synthesize recommendations for each step of the process needed for FER. This work serves as a guide to those who are new to the field. This survey provides a critique of past work, highlights recommendations and list some open, unanswered questions in FER that deserve further investigation.
- Asad Abbas and Stephan K. Chalup. 2017. Group Emotion Recognition in the Wild by Combining Deep Neural Networks for Facial Expression Classification and Scene-context Analysis. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK). ACM, New York, NY, USA, 561–568. Google Scholar
Digital Library
- F. Abdat, C. Maaoui, and A. Pruski. 2011. Human-Computer Interaction Using Emotion Recognition from Facial Expression. In 2011 UKSim 5th European Symposium on Computer Modeling and Simulation. 196–201. Google Scholar
- N. Aifanti, C. Papachristou, and A. Delopoulos. 2010. The MUG facial expression database. In 11th International Workshop on Image Analysis for Multimedia Interactive Services WIAMIS 10. 1–4.Google Scholar
- Mundher Al-Shabi, Wooi Ping Cheah, and Tee Connie. 2016. Facial Expression Recognition Using a Hybrid CNN-SIFT Aggregator. CoRR abs/1608.02833(2016). arxiv:1608.02833Google Scholar
- Shima Alizadeh and Azar Fazel. 2017. Convolutional Neural Networks for Facial Expression Recognition. arXiv preprint arXiv:1704.06756(2017).Google Scholar
- Octavio Arriaga, Matias Valdenegro-Toro, and Paul Plöger. 2017. Real-time Convolutional Neural Networks for Emotion and Gender Classification. arXiv preprint arXiv:1710.07557(2017).Google Scholar
- Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang. 2016. Training Deep Networks for Facial Expression Recognition with Crowd-sourced Label Distribution. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (Tokyo, Japan) (International Conference on Multimodal Interaction 2016). ACM, New York, NY, USA, 279–283. Google ScholarDigital Library
- Tobi Baumgartner and Jack Culpepper. 2017. Deep Architectures for Face Attributes. Springer International Publishing, Cham, 334–344.Google Scholar
- S. Bazrafkan, T. Nedelcu, P. Filipczuk, and P. Corcoran. 2017. Deep learning for facial expression recognition: A step closer to a smartphone that knows your moods. In 2017 IEEE International Conference on Consumer Electronics (ICCE). 217–220.Google Scholar
- Peter Burkert, Felix Trier, Muhammad Zeshan Afzal, Andreas Dengel, and Marcus Liwicki. 2015. Dexpression: Deep convolutional neural network for expression recognition. arXiv preprint arXiv:1509.05371(2015).Google Scholar
- Junkai Chen, Qihao Ou, Zheru Chi, and Hong Fu. 2017. Smile Detection in the Wild with Deep Convolutional Neural Networks. Mach. Vision Appl. 28, 1-2 (Feb. 2017), 173–183. Google ScholarDigital Library
- M. Chen, L. Zhang, and J. P. Allebach. 2015. Learning deep features for image emotion classification. In 2015 IEEE International Conference on Image Processing (ICIP). 4491–4495.Google ScholarDigital Library
- François Chollet. 2016. Xception: Deep Learning with Depthwise Separable Convolutions. CoRR abs/1610.02357(2016). arxiv:1610.02357Google Scholar
- N. Churamani, M. Kerzel, E. Strahl, P. Barros, and S. Wermter. 2017. Teaching emotion expressions to a human companion robot using deep neural architectures. In 2017 International Joint Conference on Neural Networks (IJCNN). 627–634.Google Scholar
- Jeff Cockburn, Marni Bartlett, James Tanaka, Javier Movellan, Matt Pierce, and Robert Schultz. 2008. Smilemaze: A tutoring system in real-time facial expression perception and production in children with autism spectrum disorder. In ECAG 2008 workshop facial and bodily expressions for control and adaptation of games. Amsterdam, 3.Google Scholar
- C. A. Corneanu, M. O. Simón, J. F. Cohn, and S. E. Guerrero. 2016. Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 8 (Aug 2016), 1548–1568.Google ScholarDigital Library
- C. D. Căleanu. 2013. Face expression recognition: A brief overview of the last decade. In 2013 IEEE 8th International Symposium on Applied Computational Intelligence and Informatics (SACI). 157–161.Google ScholarCross Ref
- Prudhvi Raj Dachapally. 2017. Facial Emotion Detection Using Convolutional Neural Networks and Representational Autoencoder Units. abs/1706.01509 (2017). arxiv:1706.01509Google Scholar
- Charles Darwin. 1872. The Expression of the Emotions in Man and Animals. John Murray.Google Scholar
- Afshin Dehghan, Enrique G. Ortiz, Guang Shu, and Syed Zain Masood. 2017. DAGER: Deep Age, Gender and Emotion Recognition Using Convolutional Neural Network. CoRR abs/1702.04280(2017). arxiv:1702.04280Google Scholar
- T. Devries, K. Biswaranjan, and G. W. Taylor. 2014. Multi-task Learning of Facial Landmarks and Expression. In 2014 Canadian Conference on Computer and Robot Vision. 98–103. Google Scholar
- Abhinav Dhall, Roland Goecke, Shreya Ghosh, Jyoti Joshi, Jesse Hoey, and Tom Gedeon. 2017. From Individual to Group-level Emotion Recognition: EmotiW 5.0. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (International Conference on Multimodal Interaction 2017). ACM, New York, NY, USA, 524–528. Google ScholarDigital Library
- A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. 2011. Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). 2106–2112.Google Scholar
- A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. 2011. Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). 2106–2112.Google Scholar
- A. Dhall, J. Joshi, K. Sikka, R. Goecke, and N. Sebe. 2015. The more the merrier: Analysing the affect of a group of people in images. In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Vol. 1. 1–8.Google Scholar
- A. Dhall, J. Joshi, K. Sikka, R. Goecke, and N. Sebe. 2015. The more the merrier: Analysing the affect of a group of people in images. In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Vol. 1. 1–8.Google Scholar
- Abhinav Dhall, O.V. Ramana Murthy, Roland Goecke, Jyoti Joshi, and Tom Gedeon. 2015. Video and Image Based Emotion Recognition Challenges in the Wild: EmotiW 2015. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (Seattle, Washington, USA) (International Conference on Multimodal Interaction ’15). ACM, New York, NY, USA, 423–426. Google ScholarDigital Library
- H. Ding, S. K. Zhou, and R. Chellappa. 2017. FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition. In 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017). 118–126.Google ScholarDigital Library
- David F. Dinges, Sundara Venkataraman, Eleanor L. McGlinchey, and Dimitris N. Metaxas. 2007. Monitoring of facial stress during space flight: Optical computer recognition combining discriminative and generative methods. Acta Astronautica 60, 4 (2007), 341 – 350. Benefits of human presence in space - historical, scientific, medical, cultural and political aspects. A selection of papers presented at the 15th IAA Humans in Space Symposium, Graz, Austria, 2005.Google ScholarCross Ref
- Dan Duncan, Gautam Shine, and Chris English. [n.d.]. Facial Emotion Recognition in Real Time. ([n. d.]).Google Scholar
- Paul Ekman and Wallace V Friesen. 1971. Constants across cultures in the face and emotion.Journal of personality and social psychology 17, 2(1971), 124.Google Scholar
- Abdallah El Ali, Torben Wallbaum, Merlin Wasmann, Wilko Heuten, and Susanne CJ Boll. 2017. Face2Emoji: Using Facial Emotional Expressions to Filter Emojis. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems(Denver, Colorado, USA) (CHI EA ’17). ACM, New York, NY, USA, 1577–1584. Google ScholarDigital Library
- Mehdi Ghayoumi. 2017. A Quick Review of Deep Learning in Facial Expression. Journal of Communication and Computer 14 (2017), 34–38.Google Scholar
- Panagiotis Giannopoulos, Isidoros Perikos, and Ioannis Hatzilygeroudis. 2018. Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013. Springer International Publishing, Cham, 1–16.Google Scholar
- Ian Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, Yingbo Zhou, Chetan Ramaiah, Fangxiang Feng, Ruifan Li, Xiaojie Wang, Dimitris Athanasakis, John Shawe-Taylor, Maxim Milakov, John Park, Radu Ionescu, Marius Popescu, Cristian Grozea, James Bergstra, Jingjing Xie, Lukasz Romaszko, Bing Xu, Zhang Chuang, and Yoshua Bengio. 2015. Challenges in representation learning: A report on three machine learning contests. Neural Networks 64(2015), 59 – 63. Special Issue on “Deep Learning of Representations”. Google ScholarDigital Library
- Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, Yingbo Zhou, Chetan Ramaiah, Fangxiang Feng, Ruifan Li, Xiaojie Wang, Dimitris Athanasakis, John Shawe-Taylor, Maxim Milakov, John Park, Radu Ionescu, Marius Popescu, Cristian Grozea, James Bergstra, Jingjing Xie, Lukasz Romaszko, Bing Xu, Zhang Chuang, and Yoshua Bengio. 2013. Challenges in Representation Learning: A Report on Three Machine Learning Contests. In Neural Information Processing, Minho Lee, Akira Hirose, Zeng-Guang Hou, and Rhee Man Kil (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 117–124.Google Scholar
- R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker. 2008. Multi-PIE. In 2008 8th IEEE International Conference on Automatic Face Gesture Recognition. 1–8.Google Scholar
- Amogh Gudi. 2015. Recognizing Semantic Features in Faces using Deep Learning. CoRR abs/1512.00743(2015). arxiv:1512.00743Google Scholar
- J. Guo, S. Zhou, J. Wu, J. Wan, X. Zhu, Z. Lei, and S. Z. Li. 2017. Multi-modality Network with Visual and Geometrical Information for Micro Emotion Recognition. In 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017). 814–819.Google Scholar
- Xin Guo, Luisa F. Polanía, and Kenneth E. Barner. 2017. Group-level Emotion Recognition Using Deep Models on Image Scene, Faces, and Skeletons. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (International Conference on Multimodal Interaction 2017). ACM, New York, NY, USA, 603–608. Google ScholarDigital Library
- K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770–778.Google Scholar
- Tobias Hinz, Pablo Barros, and Stefan Wermter. 2016. The Effects of Regularization on Learning Facial Expressions with Convolutional Neural Networks. Springer International Publishing, Cham, 80–87.Google Scholar
- Rachael E. Jack and Philippe G. Schyns. 2015. The Human Face as a Dynamic Tool for Social Communication. Current Biology 25, 14 (2015), R621 – R634.Google Scholar
- Jinwoo Jeon, Jun-Cheol Park, YoungJoo Jo, ChangMo Nam, Kyung-Hoon Bae, Youngkyoo Hwang, and Dae-Shik Kim. 2016. A Real-time Facial Expression Recognizer Using Deep Neural Network. In Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication (Danang, Viet Nam) (IMCOM ’16). ACM, New York, NY, USA, Article 94, 4 pages. Google ScholarDigital Library
- H. Jung, S. Lee, S. Park, B. Kim, J. Kim, I. Lee, and C. Ahn. 2015. Development of deep learning-based facial expression recognition system. In 2015 21st Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV). 1–4.Google Scholar
- Lukasz Kaiser, Aidan N. Gomez, and François Chollet. 2017. Depthwise Separable Convolutions for Neural Machine Translation. CoRR abs/1706.03059(2017). arxiv:1706.03059Google Scholar
- Pooya Khorrami, Tom Le Paine, and Thomas S. Huang. 2015. Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?. In Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW)(ICCVW ’15). IEEE Computer Society, Washington, DC, USA, 19–27. Google ScholarDigital Library
- Pooya Rezvani Khorrami. 2017. How deep learning can help emotion recognition. dissertation. University of Illinois.Google Scholar
- B. K. Kim, S. Y. Dong, J. Roh, G. Kim, and S. Y. Lee. 2016. Fusing Aligned and Non-aligned Face Information for Automatic Affect Recognition in the Wild: A Deep Learning Approach. In 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1499–1508.Google Scholar
- Bo-Kyeong Kim, Hwaran Lee, Jihyeon Roh, and Soo-Young Lee. 2015. Hierarchical Committee of Deep CNNs with Exponentially-Weighted Decision Fusion for Static Facial Expression Recognition. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (Seattle, Washington, USA) (International Conference on Multimodal Interaction ’15). ACM, New York, NY, USA, 427–434. Google ScholarDigital Library
- Byoung Chul Ko. 2018. A Brief Review of Facial Emotion Recognition Based on Visual Information. Sensors 18, 2 (2018), 401.Google ScholarDigital Library
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (Lake Tahoe, Nevada) (NIPS’12). Curran Associates Inc., USA, 1097–1105. Google ScholarDigital Library
- Jyoti Kumari, R. Rajesh, and K.M. Pooja. 2015. Facial Expression Recognition: A Survey. Procedia Computer Science 58 (2015), 486 – 491. Second International Symposium on Computer Vision and the Internet (VisionNet’15).Google Scholar
- Oliver Langner, Ron Dotsch, Gijsbert Bijlstra, Daniel HJ Wigboldus, Skyler T Hawk, and AD Van Knippenberg. 2010. Presentation and validation of the Radboud Faces Database. Cognition and emotion 24, 8 (2010), 1377–1388.Google Scholar
- Charlyn Pushpa Latha and Mohana Priya. 2016. A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition. APTIKOM Journal on Computer Science and Information Technologies 1, 3(2016), 88–104.Google ScholarCross Ref
- S. Lawrence, C. L. Giles, Ah Chung Tsoi, and A. D. Back. 1997. Face recognition: a convolutional neural-network approach. IEEE Transactions on Neural Networks 8, 1 (Jan 1997), 98–113. Google ScholarDigital Library
- Jiaxing Li, Dexiang Zhang, Jingjing Zhang, Jun Zhang, Teng Li, Yi Xia, Qing Yan, and Lina Xun. 2017. Facial Expression Recognition with Faster R-CNN. Procedia Computer Science 107 (2017), 135 – 140. Advances in Information and Communication Technology: Proceedings of 7th International Congress of Information and Communication Technology (ICICT2017). Google Scholar
- Wei Li, Farnaz Abtahi, Christina Tsangouri, and Zhigang Zhu. 2016. Towards an “In-the-Wild” Emotion Dataset Using a Game-Based Framework. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2016 IEEE Conference on. IEEE, 1526–1534.Google Scholar
- W. Li, M. Li, Z. Su, and Z. Zhu. 2015. A deep-learning approach to facial expression recognition with candid images. In 2015 14th IAPR International Conference on Machine Vision Applications (MVA). 279–282.Google Scholar
- K. Liu, M. Zhang, and Z. Pan. 2016. Facial Expression Recognition with CNN Ensemble. In 2016 International Conference on Cyberworlds (CW). 163–166.Google Scholar
- André Teixeira Lopes, Edilson de Aguiar, Alberto F. De Souza, and Thiago Oliveira-Santos. 2017. Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order. Pattern Recognition 61(2017), 610 – 628. Google ScholarDigital Library
- P. Lucey, J. Cohn, S. Lucey, I. Matthews, S. Sridharan, and K. M. Prkachin. 2009. Automatically detecting pain using facial actions. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. 1–8.Google ScholarCross Ref
- P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews. 2010. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. 94–101.Google Scholar
- D. Lundqvist, Flykt A., and A. Ohman. 1998. The Karolinska Directed Emotional Faces – KDEF.Google Scholar
- Zhaojie Luo, J. Chen, T. Takiguchi, and Y. Ariki. 2017. Facial Expression Recognition with deep age. In 2017 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 657–662.Google Scholar
- I. Lusi, J. C. S. J. Junior, J. Gorbova, X. Baró, S. Escalera, H. Demirel, J. Allik, C. Ozcinar, and G. Anbarjafari. 2017. Joint Challenge on Dominant and Complementary Emotion Recognition Using Micro Emotion Features and Head-Pose Estimation: Databases. In 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017). 809–813.Google Scholar
- M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba. 1998. Coding facial expressions with Gabor wavelets. In Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition. 200–205. Google Scholar
- Brais Martinez and Michel F. Valstar. 2016. Advances, Challenges, and Opportunities in Automatic Facial Expression Recognition. Springer International Publishing, Cham, 63–100.Google Scholar
- Masakazu Matsugu, Katsuhiko Mori, Yusuke Mitari, and Yuji Kaneda. 2003. Subject Independent Facial Expression Recognition with Robust Face Detection Using a Convolutional Neural Network. Neural Networks 16, 5-6 (June 2003), 555–559. Google ScholarDigital Library
- Viraj Mavani, Shanmuganathan Raman, and Krishna P Miyapuram. 2017. Facial expression recognition using visual saliency and deep learning. arXiv preprint arXiv:1708.08016(2017).Google Scholar
- Veena Mayya, Radhika M. Pai, and M.M. Manohara Pai. 2016. Automatic Facial Expression Recognition Using DCNN. Procedia Computer Science 93 (2016), 453 – 461. Proceedings of the 6th International Conference on Advances in Computing and Communications.Google Scholar
- Z. Meng, P. Liu, J. Cai, S. Han, and Y. Tong. 2017. Identity-Aware Convolutional Neural Network for Facial Expression Recognition. In 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017). 558–565.Google Scholar
- A. Mollahosseini, D. Chan, and M. H. Mahoor. 2016. Going deeper in facial expression recognition using deep neural networks. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). 1–10.Google ScholarCross Ref
- Ali Mollahosseini, Behzad Hassani, Michelle J Salvador, Hojjat Abdollahi, David Chan, and Mohammad H Mahoor. 2016. Facial expression recognition from world wild web. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2016 IEEE Conference on. IEEE, 1509–1516.Google ScholarCross Ref
- N. Mousavi, H. Siqueira, P. Barros, B. Fernandes, and S. Wermter. 2016. Understanding how deep neural networks learn face expressions. In 2016 International Joint Conference on Neural Networks (IJCNN). 227–234.Google Scholar
- Hong-Wei Ng, Viet Dung Nguyen, Vassilios Vonikakis, and Stefan Winkler. 2015. Deep Learning for Emotion Recognition on Small Datasets Using Transfer Learning. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction(Seattle, Washington, USA) (International Conference on Multimodal Interaction ’15). ACM, New York, NY, USA, 443–449. Google ScholarDigital Library
- Sébastien Ouellet. 2014. Real-time emotion recognition for gaming using deep convolutional network features. arXiv preprint arXiv:1408.3750(2014).Google Scholar
- M. Pantic and L. J. M. Rothkrantz. 2000. Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 12 (Dec 2000), 1424–1445. Google Scholar
- M. Pantic, M. Valstar, R. Rademaker, and L. Maat. 2005. Web-based database for facial expression analysis. In 2005 IEEE International Conference on Multimedia and Expo. 5 pp.–.Google Scholar
- O. M. Parkhi, A. Vedaldi, and A. Zisserman. 2015. Deep Face Recognition. In British Machine Vision Conference.Google Scholar
- X. Peng, L. Li, X. Feng, and J. Fan. 2017. Spontaneous facial expression recognition by heterogeneous convolutional networks. In 2017 International Conference on the Frontiers and Advances in Data Science (FADS). 70–73.Google Scholar
- X. Peng, Z. Xia, L. Li, and X. Feng. 2016. Towards Facial Expression Recognition in the Wild: A New Database and Deep Recognition System. In 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1544–1550.Google Scholar
- Valfredo Pilla Jr, André Zanellato, Cristian Bortolini, Humberto R Gamba, Gustavo Benvenutti Borba, and Henry Medeiros. 2016. Facial Expression Classification Using Convolutional Neural Network and Support Vector Machine. (2016).Google Scholar
- Diah Anggraeni Pitaloka, Ajeng Wulandari, T. Basaruddin, and Dewi Yanti Liliana. 2017. Enhancing CNN with Preprocessing Stage in Automatic Emotion Recognition. Procedia Computer Science 116 (2017), 523 – 529. Discovery and innovation of computer science technology in artificial intelligence era: The 2nd International Conference on Computer Science and Computational Intelligence (ICCSCI 2017). Google ScholarDigital Library
- G. Pons and D. Masip. 2017. Supervised Committee of Convolutional Neural Networks in Automated Facial Expression Analysis. IEEE Transactions on Affective Computing PP, 99 (2017), 1–1. Google Scholar
- Gerard Pons and David Masip. 2018. Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition. arXiv preprint arXiv:1802.06664(2018).Google Scholar
- Christopher Pramerdorfer and Martin Kampel. 2016. Facial Expression Recognition using Convolutional Neural Networks: State of the Art. arXiv preprint arXiv:1612.02903(2016).Google Scholar
- Arushi Raghuvanshi and Vivek Choksi. 2016. Facial Expression Recognition with Convolutional Neural Networks. (2016).Google Scholar
- Tarik A. Rashid. 2016. Convolutional Neural Networks based Method for Improving Facial Expression Recognition. Springer International Publishing, Cham, 73–84.Google Scholar
- Alexandr Rassadin, Alexey Gruzdev, and Andrey Savchenko. 2017. Group-level Emotion Recognition Using Transfer Learning from Face Identification. In Proceedings of the 19th ACM International Conference on Multimodal Interaction(Glasgow, UK) (International Conference on Multimodal Interaction 2017). ACM, New York, NY, USA, 544–548. Google ScholarDigital Library
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems. 91–99. Google Scholar
- Sohini Roychowdhury and Michelle Emmons. 2015. A Survey of the Trends in Facial and Expression Recognition Databases and Methods. International Journal of Computer Science and Engineering Survey (IJCSES) 6, 5(2015).Google Scholar
- Ariel Ruiz-Garcia, Mark Elshaw, Abdulrahman Altahhan, and Vasile Palade. 2016. Deep Learning for Emotion Recognition in Faces. Springer International Publishing, Cham, 38–46.Google Scholar
- Dinh Viet Sang, Nguyen Van Dat, and Do Phan Thuan. 2017. Facial Expression Recognition Using Deep Convolutional Neural Networks. International Conference on Knowledge and Systems Engineering (KSE) (2017).Google Scholar
- K. A. Sannikov, A. A. Bashlikov, and A. A. Druki. 2017. Two-level algorithm of facial expressions classification on complex background. In 2017 International Siberian Conference on Control and Communications (SIBCON). 1–5.Google ScholarCross Ref
- E. Sariyanidi, H. Gunes, and A. Cavallaro. 2015. Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 6 (June 2015), 1113–1133.Google ScholarDigital Library
- Arman Savran, Neşe Alyüz, Hamdi Dibeklioğlu, Oya Çeliktutan, Berk Gökberk, Bülent Sankur, and Lale Akarun. 2008. Biometrics and Identity Management. Springer-Verlag, Berlin, Heidelberg, Chapter Bosphorus Database for 3D Face Analysis, 47–56.Google Scholar
- J. Schwan, E. Ghaleb, E. Hortal, and S. Asteriadis. 2017. High-performance and lightweight real-time deep face emotion recognition. In 2017 12th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP). 76–79.Google Scholar
- K. Shan, J. Guo, W. You, D. Lu, and R. Bie. 2017. Automatic facial expression recognition based on a deep convolutional-neural-network structure. In 2017 IEEE 15th International Conference on Software Engineering Research, Management and Applications (SERA). 123–128.Google Scholar
- M. Shin, M. Kim, and D. S. Kwon. 2016. Baseline CNN structure analysis for facial expression recognition. In 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). 724–729.Google ScholarDigital Library
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556(2014).Google Scholar
- I. Song, H. J. Kim, and P. B. Jeon. 2014. Deep learning for real-time robust facial expression recognition on a smartphone. In 2014 IEEE International Conference on Consumer Electronics (ICCE). 564–567.Google ScholarCross Ref
- Daniel Spiers. 2016. Facial emotion detection using deep learning.Google Scholar
- Luca Surace, Massimiliano Patacchiola, Elena Battini Sönmez, William Spataro, and Angelo Cangelosi. 2017. Emotion Recognition in the Wild Using Deep Neural Networks and Bayesian Classifiers. In Proceedings of the 19th ACM International Conference on Multimodal Interaction(Glasgow, UK) (International Conference on Multimodal Interaction 2017). ACM, New York, NY, USA, 593–597. Google ScholarDigital Library
- Josh M Susskind, Adam K Anderson, and Geoffrey E Hinton. 2010. The toronto face database. Department of Computer Science, University of Toronto, Toronto, ON, Canada, Tech. Rep 3(2010).Google Scholar
- C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2015. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1–9.Google Scholar
- Elena Battini Sönmez and Angelo Cangelosi. 2017. Convolutional neural networks with balanced batches for facial expressions recognition. , 10341 - 10341 - 6 pages.Google Scholar
- M. Taini, G. Zhao, S. Z. Li, and M. Pietikainen. 2008. Facial expression recognition from near-infrared video sequences. In 2008 19th International Conference on Pattern Recognition. 1–4.Google Scholar
- Lianzhi Tan, Kaipeng Zhang, Kai Wang, Xiaoxing Zeng, Xiaojiang Peng, and Yu Qiao. 2017. Group Emotion Recognition with Individual Facial Emotion CNNs and Global Image Based CNNs. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (International Conference on Multimodal Interaction 2017). ACM, New York, NY, USA, 549–552. Google ScholarDigital Library
- Yingli Tian, Takeo Kanade, and Jeffrey F Cohn. 2011. Facial expression recognition. In Handbook of face recognition. Springer, 487–519.Google Scholar
- http://mplab.ucsd.edu. 2009. The MPLab GENKI Database.Google Scholar
- Hugo Gamboa Valero. 2016. Automatic Facial Expression Recognition.Google Scholar
- M. F. Valstar, B. Jiang, M. Mehu, M. Pantic, and K. Scherer. 2011. The first facial expression recognition and analysis challenge. In Face and Gesture 2011. 921–926.Google Scholar
- P. Viola and M. Jones. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Vol. 1. I–511–I–518 vol.1.Google Scholar
- S. Wang, Z. Liu, S. Lv, Y. Lv, G. Wu, P. Peng, F. Chen, and X. Wang. 2010. A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference. IEEE Transactions on Multimedia 12, 7 (Nov 2010), 682–691. Google ScholarDigital Library
- Qinglan Wei, Yijia Zhao, Qihua Xu, Liandong Li, Jun He, Lejun Yu, and Bo Sun. 2017. A New Deep-learning Framework for Group Emotion Recognition. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (International Conference on Multimodal Interaction 2017). ACM, New York, NY, USA, 587–592. Google ScholarDigital Library
- G. Wen, H. Li, and D. Li. 2015. An ensemble convolutional echo state networks for facial expression recognition. In 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). 873–878. Google Scholar
- Ting Wu, Siyao Fu, and Guosheng Yang. 2012. Survey of the Facial Expression Recognition Research. In Advances in Brain Inspired Cognitive Systems, Huaguang Zhang, Amir Hussain, Derong Liu, and Zhanshan Wang (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 392–402.Google Scholar
- Xiao-Ling Xia, Cui Xu, and Bing Nan. 2017. Facial Expression Recognition Based on TensorFlow Platform. In ITM Web of Conferences, Vol. 12. EDP Sciences, 01005.Google Scholar
- S. Xie and H. Hu. 2017. Facial expression recognition with FRR-CNN. Electronics Letters 53, 4 (2017), 235–237.Google ScholarCross Ref
- X. Xiong and F. De la Torre. 2013. Supervised Descent Method and Its Applications to Face Alignment. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 532–539. Google Scholar
- Mao Xu, Wei Cheng, Qian Zhao, Li Ma, and Fang Xu. 2015. Facial expression recognition based on transfer learning from deep convolutional networks. In 2015 11th International Conference on Natural Computation (ICNC). 702–708.Google Scholar
- Huei-Fang Yang, Bo-Yao Lin, Kuang-Yu Chang, and Chu-Song Chen. 2018. Joint Estimation of Age and Expression by Combining Scattering and Convolutional Networks. ACM Trans. Multimedia Comput. Commun. Appl. 14, 1, Article 9 (Jan. 2018), 18 pages. Google ScholarDigital Library
- Lijun Yin, Xiaozhou Wei, Yi Sun, Jun Wang, and M. J. Rosato. 2006. A 3D facial expression database for facial behavior research. In 7th International Conference on Automatic Face and Gesture Recognition (FGR06). 211–216. Google ScholarDigital Library
- G. Yolcu, I. Oztel, S. Kazan, C. Oz, K. Palaniappan, T. E. Lever, and F. Bunyak. 2017. Deep learning-based facial expression recognition for monitoring neurological disorders. In 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Vol. 00. 1652–1657.Google Scholar
- Zhiding Yu and Cha Zhang. 2015. Image Based Static Facial Expression Recognition with Multiple Deep Network Learning. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (Seattle, Washington, USA) (International Conference on Multimodal Interaction ’15). ACM, New York, NY, USA, 435–442. Google ScholarDigital Library
- M. V. Zavarez, R. F. Berriel, and T. Oliveira-Santos. 2017. Cross-Database Facial Expression Recognition Based on Fine-Tuned Deep Convolutional Network. In 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). 405–412.Google Scholar
- Ting Zhang. 2018. Facial Expression Recognition Based on Deep Learning: A Survey. Springer International Publishing, Cham, 345–352.Google Scholar
- T. Zhang, W. Zheng, Z. Cui, Y. Zong, J. Yan, and K. Yan. 2016. A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition. IEEE Transactions on Multimedia 18, 12 (Dec 2016), 2528–2536. Google ScholarDigital Library
- Y. Zhou and B. E. Shi. 2017. Action unit selective feature maps in deep networks for facial expression recognition. In 2017 International Joint Conference on Neural Networks (IJCNN). 2031–2038.Google ScholarCross Ref
- A Survey on Factors Affecting Facial Expression Recognition based on Convolutional Neural Networks
Recommendations
Facial expression recognition with Convolutional Neural Networks
Facial expression recognition has been an active research area in the past 10 years, with growing application areas including avatar animation, neuromarketing and sociable robots. The recognition of facial expressions is not an easy problem for machine ...
Expression-invariant face recognition by facial expression transformations
In this paper, we present a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. When a new face image with an arbitrary expression is ...
Pose-robust feature learning for facial expression recognition
Automatic facial expression recognition (FER) from non-frontal views is a challenging research topic which has recently started to attract the attention of the research community. Pose variations are difficult to tackle and many face analysis methods ...
Comments