Ioannis Kansizoglou
ABSTRACT
Emotions convey concise information regarding an individual’s internal state, while in the long-term they can be used to form an opinion about his/her overall personality. The latter can be proved particularly vital in many human-robot interaction tasks, like in the case of an assisted living robotic agent, where the human’s mood may in turn require the adaptation of the robot’s behavior. As a result, the paper at hand proposes a novel approach enabling an artificial agent to conceive and gradually learn the personality of a human, by tracking his emotional variations throughout their interaction time. To achieve that, the facial landmarks of the subject are extracted and fed into a Deep Neural Network architecture that estimates the two coefficients of human emotions, viz., arousal and valence, as introduced by the broadly known Russell’s model. Finally, by creating a dashboard for user-friendly display of our results, we present both momentarily and in the long-term the monitored fluctuations of a person’s emotional state.
- Sevegni Odilon Clement Allognon, Alceu de S Britto, and Alessandro L Koerich. 2020. Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor. In 2020 International Joint Conference on Neural Networks. 1–8.Google Scholar
Cross Ref
- Adriana Breaban, Gijs van de Kuilen, and Charles N. Noussair. 2016. Prudence, Emotional State, Personality, and Cognitive Ability. Frontiers in Psychology 7 (2016), 1688.Google ScholarCross Ref
- Konstantinos Charalampous, Ioannis Kostavelis, and Antonios Gasteratos. 2017. Recent trends in social aware robot navigation: A survey. Robotics and Autonomous Systems 93 (2017), 85–104.Google ScholarDigital Library
- Paul Ekman, Wallace V Friesen, Maureen O’sullivan, Anthony Chan, Irene Diacoyanni-Tarlatzis, Karl Heider, Rainer Krause, William Ayhan LeCompte, Tom Pitcairn, Pio E Ricci-Bitti, 1987. Universals and cultural differences in the judgments of facial expressions of emotion.Journal of personality and social psychology 53 (1987), 712.Google Scholar
- Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. 448–456.Google ScholarDigital Library
- Elmer Jacobs, Joost Broekens, and Catholijn Jonker. 2014. Emergent dynamics of joy, distress, hope and fear in reinforcement learning agents. In Adaptive learning agents workshop at AAMAS2014.Google Scholar
- Ioannis Kansizoglou, Loukas Bampis, and Antonios Gasteratos. 2019. An active learning paradigm for online audio-visual emotion recognition. IEEE Transactions on Affective Computing(2019).Google ScholarCross Ref
- Ioannis Kansizoglou, Loukas Bampis, and Antonios Gasteratos. 2020. Deep Feature Space: A Geometrical Perspective. arXiv preprint arXiv:2007.00062(2020).Google Scholar
- Vahid Kazemi and Josephine Sullivan. 2014. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1867–1874.Google ScholarDigital Library
- Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.Google Scholar
- Hyun-Soon Lee and Bo-Yeong Kang. 2020. Continuous emotion estimation of facial expressions on JAFFE and CK+ datasets for human–robot interaction. Intelligent Service Robotics 13 (2020), 15–27.Google ScholarDigital Library
- Ali Mollahosseini, Behzad Hasani, and Mohammad H Mahoor. 2017. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing 10 (2017), 18–31.Google ScholarDigital Library
- Olivia Nocentini, Laura Fiorini, Giorgia Acerbi, Alessandra Sorrentino, Gianmaria Mancioppi, and Filippo Cavallo. 2019. A survey of behavioral models for social robots. Robotics 8(2019), 54.Google ScholarCross Ref
- Fabien Ringeval, Andreas Sonderegger, Juergen Sauer, and Denis Lalanne. 2013. Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. In 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). 1–8.Google ScholarCross Ref
- Silvia Rossi, Francois Ferland, and Adriana Tapus. 2017. User profiling and behavioral adaptation for HRI: A survey. Pattern Recognition Letters 99 (2017), 3–12.Google ScholarDigital Library
- James A Russell and Lisa Feldman Barrett. 1999. Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant.Journal of personality and social psychology 76 (1999), 805.Google Scholar
- Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15 (2014), 1929–1958.Google Scholar
- Konstantinos A Tsintotas, Loukas Bampis, and Antonios Gasteratos. 2019. Probabilistic appearance-based place recognition through bag of tracked words. IEEE Robotics and Automation Letters 4 (2019), 1737–1744.Google ScholarCross Ref
- Panagiotis Tzirakis, George Trigeorgis, Mihalis A Nicolaou, Björn W Schuller, and Stefanos Zafeiriou. 2017. End-to-end multimodal emotion recognition using deep neural networks. IEEE Journal of Selected Topics in Signal Processing 11 (2017), 1301–1309.Google ScholarCross Ref
- Paul Viola and Michael Jones. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001. I–I.Google ScholarCross Ref
Recommendations
The Role of Empathic Traits in Emotion Recognition and Emotion Contagion of Cozmo Robots
HRI '22: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot InteractionIn this online study, we investigated how well people could recognize emotions displayed by video recordings of a Cozmo robot, and the extent to which emotion recognition is shaped by individuals' empathic traits. We also explored whether participants ...
Emotion Recognition Using Physiological Signals
MIDI '15: Proceedings of the Mulitimedia, Interaction, Design and InnnovationIn this paper the problem of emotion recognition using physiological signals is presented. Firstly the problems with acquisition of physiological signals related to specific human emotions are described. It is not a trivial problem to elicit real ...
Physiological emotion analysis using support vector regression
Physical and mental diseases were deeply affected by stress and negative emotions. In general, emotions can be roughly recognized by facial expressions. Since facial expressions may be controlled and expressed differently by different people ...
Comments