Abstract
Facial Emotion Recognition (FER) is the automatic processing of human emotions by means of facial expression analysis [1]. The most common approach exploits 3D Face Descriptors (3D-FD) [2], which derive from depth maps [3] by using mathematical operators. In recent years, Convolutional Neural Networks (CNNs) have been successfully employed in a wide range of tasks including large-scale image classification systems and to overcome the hurdles in facial expression classification. Based on previous studies, the purpose of the present work is to analyze and compare the abstraction level of 3D face descriptors with abstraction in deep CNNs. Experimental results suggest that 3D face descriptors have an abstraction level comparable with the features extracted in the fourth layer of CNN, the layer of the network having the highest correlations with emotions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Eureqa Formulize is developed by Nutonian, Inc. https://www.nutonian.com/products/eureqa/.
References
Ekman, P.: Facial expression and emotion. Am. Psychol. 48(4), 384 (1993)
Vezzetti, E., Marcolin, F.: Geometrical descriptors for human face morphological analysis and recognition. Robot. Auton. Syst. 60, 928–939, 06 (2012)
Nonis, F., Dagnes, N., Marcolin, F., Vezzetti, E.: 3d approaches and challenges in facial expression recognition algorithmsa literature review. Appl. Sci. 9(18), 3904 (2019)
Ieracitano, C., Mammone, N., Bramanti, A., Hussain, A., Morabito, F.C.: A convolutional neural network approach for classification of dementia stages based on 2d-spectral representation of EEG recordings. Neurocomputing 323, 96–107 (2019)
Ieracitano, C., Adeel, A., Morabito, F.C., Hussain, A.: A novel statistical analysis and autoencoder driven intelligent intrusion detection approach. Neurocomputing (2019)
Huynh, P., Tran T.-D., Kim, Y.-G.: Convolutional Neural Network Models for Facial Expression Recognition Using 3D-BUFE Database, pp. 441–450 (2016)
Chen, Z., Huang, D., Wang, Y., Chen, L.: Fast and light manifold CNN based 3d facial expression recognition across pose variations, pp. 229–238 (2018)
Li, H., Sun, J., Zongben, X., Chen, L.: Multimodal 2d+ 3d facial expression recognition with deep fusion convolutional neural network. IEEE Trans. Multimed. 19(12), 2816–2831 (2017)
Yang, H., Yin, L.: CNN based 3d facial expression recognition using masking and landmark features. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 556–560. IEEE (2017)
Oyedotun, O.K., Demisse, G., Rahman Shabayek, A.E., Aouada, D., Ottersten, B.: Facial expression recognition via joint deep learning of RGB-depth map latent representations. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3161–3168 (2017)
Russell, B.: Principles of Mathematics. Routledge (2009)
Ferrari, P.L.: Abstraction in mathematics. Philos. Trans. R. Soc. London. Ser. B Biol. Sci. 358(1435), 1225–1230 (2003)
Bracewell, R.N., Bracewell, R.N.: The Fourier Transform and Its Applications, vol. 31999. McGraw-Hill, New York (1986)
Papoulis, A.: The Fourier Integral and Its Applications. McGraw-Hill (1962)
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3d face analysis. In: European Workshop on Biometrics and Identity Management, pp. 47–56. Springer, Berlin (2008)
Friesen, E., Ekman, P.: Facial action coding system: a technique for the measurement of facial movement. Palo Alto 3 (1978)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Marcolin, F., Vezzetti, E.: Novel descriptors for geometrical 3d face analysis. Multimed. Tools Appl. 76, 07 (2016)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Pearson, K.: Vii. note on regression and inheritance in the case of two parents. Proc. R. Soc. Lond. 58(347–352), 240–242 (1895)
Koza, J.R., Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1. MIT Press (1992)
Schmidt, M., Lipson, M.: Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2009)
Shorten. C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 60 (2019)
Stone, M.: Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc. Ser. B (Methodological) 36(2), 111–133 (1974)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Nonis, F., Barbiero, P., Cirrincione, G., Olivetti, E.C., Marcolin, F., Vezzetti, E. (2021). Understanding Abstraction in Deep CNN: An Application on Facial Emotion Recognition. In: Esposito, A., Faundez-Zanuy, M., Morabito, F., Pasero, E. (eds) Progresses in Artificial Intelligence and Neural Systems. Smart Innovation, Systems and Technologies, vol 184. Springer, Singapore. https://doi.org/10.1007/978-981-15-5093-5_26
Download citation
DOI: https://doi.org/10.1007/978-981-15-5093-5_26
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-5092-8
Online ISBN: 978-981-15-5093-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)