Abstract
Automatic facial emotion recognition plays an important role in human-computer interaction. Although humans can recognize emotions with little or no effort, reliable emotion recognition by machines is always a challenge. To explore the doubt that whether machines can discriminate better than humans or not, we proposed two different ideas about facial emotion recognition. One is based on image visual features, the other is based on EEG signals which were recorded when the subject is watching facial emotion pictures. Correspondingly, the Deep Convolutional Neural Network (DCNN) model is adopted to enable the machine to learn visual features from facial emotion pictures automatically. The Gated Recurrent Unit (GRU) model is used to extract specific emotional EEG features from EEG signals. These two methods were verified on the Chinese Facial Affective Picture System (CFAPS) and our developed Emotion EEG data (EMOT), and the recognition performance based on EEG features was found to be significantly better than the image visual.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tian, Y.L., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)
Cohen, I., Sebe, N., Sun, Y., Lew, M.S., Huang, T.S.: Evaluation of expression recognition techniques. In: Bakker, E.M., Lew, M.S., Huang, T.S., Sebe, N., Zhou, X.S. (eds.) CIVR 2003. LNCS, vol. 2728, pp. 184–195. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-45113-7_19
Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Fully automatic facial action recognition in spontaneous behavior. In: International Conference on Automatic Face and Gesture Recognition, pp. 223–230 (2006)
Pantic, M., Rothkrantz, L.J.: Facial action recognition for facial expression analysis from static face images. IEEE Trans. Syst. Man Cybern. Part B Cybern. 34(3), 1449–1461 (2004)
Alizadeh, S., Fazel, A.: Convolutional neural networks for facial expression recognition. arXiv preprint arXiv:1704.06756 (2017)
https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data. Accessed 2013
Ng, H.W., Nguyen, V.D., Vonikakis, V., Winkler, S.: Deep learning for emotion recognition on small datasets using transfer learning. In: ACM on International Conference on Multimodal Interaction, pp. 443–449 (2015)
Giannopoulos, P., Perikos, I., Hatzilygeroudis, I.: Deep learning approaches for facial emotion recognition: a case study on FER-2013. In: Hatzilygeroudis, I., Palade, V. (eds.) Advances in Hybridization of Intelligent Methods. SIST, vol. 85, pp. 1–16. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-66790-4_1
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Bajaj, V., Pachori, R.B.: Detection of human emotions using features based on the multiwavelet transform of EEG signals. In: Hassanien, A.E., Azar, A.T. (eds.) Brain-Computer Interfaces. ISRL, vol. 74, pp. 215–240. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-10978-7_8
Li, X., Li, X., et al.: Effects of emotion on cognitive processing: series of event-related potentials study. Adv. Psychol. Sci. 14(04), 505–510 (2006)
Niedermeyer, E., da Silva, F.L.: Electroencephalography: Basic Principles, Clinical Applications, and Related Fields. Lippincott Williams & Wilkins (2005)
Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Deng, J.: A large-scale hierarchical image database. In: 2009 Proceedings of IEEE Computer Vision and Pattern Recognition (2009)
https://github.com/keras-team/keras. Accessed 2015
Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7(3), 162–175 (2017)
Stewart, A.X., Nuthmann, A., Sanguinetti, G.: Single-trial classification of EEG in a visual object task using ICA and machine learning. J. Neurosci. Methods 228, 1–14 (2014)
Acknowledgements
This work was supported by National Key R&D Program of China for Intergovernmental International Science and Technology Innovation Cooperation Project (2017YFE0116800), National Natural Science Foundation of China (61671193), Science and Technology Program of Zhejiang Province (2018C04012).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Long, Y., Kong, W., Ling, W., Yang, C., Zhu, J. (2019). Comparison of Facial Emotion Recognition Based on Image Visual Features and EEG Features. In: Sun, F., Liu, H., Hu, D. (eds) Cognitive Systems and Signal Processing. ICCSIP 2018. Communications in Computer and Information Science, vol 1006. Springer, Singapore. https://doi.org/10.1007/978-981-13-7986-4_15
Download citation
DOI: https://doi.org/10.1007/978-981-13-7986-4_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-7985-7
Online ISBN: 978-981-13-7986-4
eBook Packages: Computer ScienceComputer Science (R0)