Abstract
Facial expressions convey the vast majority of the emotional information contained in social utterances. From the point of view of affective intelligent systems, it is therefore important to develop appropriate emotion recognition models based on facial images. As a result of the high interest of the research and industrial community in this problem, many ready-to-use tools are being developed, which can be used via suitable web APIs. In this paper, two of the most popular APIs were tested: Microsoft Face API and Kairos Emotion Analysis API. The evaluation was performed on images representing 8 emotions—anger, contempt, disgust, fear, joy, sadness, surprise and neutral—distributed in 4 benchmark datasets: Cohn-Kanade (CK), Extended Cohn-Kanade (CK+), Amsterdam Dynamic Facial Expression Set (ADFES) and Radboud Faces Database (RaFD). The results indicated a significant advantage of the Microsoft API in the accuracy of emotion recognition both in photos taken en face and at a 45\(^\circ \) angle. Microsoft’s API also has an advantage in the larger number of recognised emotions: contempt and neutral are also included.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cohn, J.F., De La Torre, F.: Automated face analysis for affective computing. In: Calvo, R.A., D’Mello, S., Gratch, J., Kappas, A. (eds.) The Oxford Handbook of Affective Computing, pp. 131–150. Oxford University Press (2015). https://doi.org/10.1093/oxfordhb/9780199942237.013.020
Deshmukh, R.S., Jagtap, V.: A survey: software API and database for emotion recognition. In: 2017 International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 284–289 (2017). https://doi.org/10.1109/ICCONS.2017.8250727
Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)
Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, Oxford (2005)
Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124–129 (1971)
Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS): Manual. A Human Face, Salt Lake City (2002)
Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 757–763 (1997)
Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG 2000), Grenoble, France, pp. 46–53 (2000)
Khanal, S.R., Barroso, J., Lopes, N., Sampaio, J., Filipe, V.: Performance analysis of Microsoft’s and Google’s emotion recognition API using pose-invariant faces. In: Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, DSAI 2019, Thessaloniki, Greece, 20–22 June 2018, pp. 172–178. ACM (2018). https://doi.org/10.1145/3218585.3224223
Konar, A., Chakraborty, A. (eds.): Emotion Recognition: A Pattern Analysis Approach. Wiley, New Jersey (2015)
Kutt, K., Drążyk, D., Bobek, S., Nalepa, G.J.: Personality-based affective adaptation methods for intelligent systems. Sensors 21(1), 163 (2021). https://doi.org/10.3390/s21010163
Kutt, K., et al.: BIRAFFE: bio-reactions and faces for emotion-based personalization. In: AFCAI 2019: Workshop on Affective Computing and Context Awareness in Ambient Intelligence. CEUR Workshop Proceedings, vol. 2609. CEUR-WS.org (2020)
Kutt, K., Drążyk, D., Szelążek, M., Bobek, S., Nalepa, G.J.: The BIRAFFE2 experiment. Study in bio-reactions and faces for emotion-based personalization for AI systems. CoRR abs/2007.15048 (2020). https://arxiv.org/abs/2007.15048
Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D., Hawk, S., van Knippenberg, A.: Presentation and validation of the radboud face database. Cogn. Emotion 24(8), 1377–1388 (2010)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: Proceedings of the Third International Workshop on CVPR for Human Communicative Behavior Analysis (CVPR4HB 2010), San Francisco, USA, pp. 94–101, (2010)
Mehrabian, A.: Communication without words. Psychol. Today 2(4), 53–56 (1968)
Nalepa, G.J., Kutt, K., Bobek, S.: Mobile platform for affective context-aware systems. Future Generar. Comput. Syst. 92, 490–503 (2019). https://doi.org/10.1016/j.future.2018.02.033
Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)
Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178 (1980)
van der Schalk, J., Hawk, S., Fischer, A., Doosje, B.: Moving faces, looking places: validation of the Amsterdam dynamic facial expression set (ADFES). Emotion 11(4), 907–920 (2011)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004). https://doi.org/10.1023/B:VISI.0000013087.49260.fb
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Kutt, K., Sobczyk, P., Nalepa, G.J. (2022). Evaluation of Selected APIs for Emotion Recognition from Facial Expressions. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence. IWINAC 2022. Lecture Notes in Computer Science, vol 13259. Springer, Cham. https://doi.org/10.1007/978-3-031-06527-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-06527-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06526-2
Online ISBN: 978-3-031-06527-9
eBook Packages: Computer ScienceComputer Science (R0)