Abstract
We present a study on the multimodal analysis of emotions. It will be used to create an emotion detector that can be implemented in a social robot. First, we specify the nature of the emotions studied. Secondly, we detail the main stages of the research: constitution of the dataset; extraction of the dataset from an image stream, a sound stream, and a text stream; classification of facial expressions and prosody in image and sound streams; labeling of the emotions of the text flow with a predefined typology; labeling the emotions of the image and sound stream classes with the tagged text stream. Thirdly, we discuss the protocol used to evaluate the research work. Finally, we present the perspectives of the research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lacheret, A.: Le corps en voix ou l’expression prosodique des émotions. Evolutions Psychomotrices, Fédération Européenne des Psychomotriciens 23(90), 25–37 (2011)
Bassil, J.: Facial motion in the perception of faces and of emotional expression. J. Experimental Psychology 4(3), 373–379 (1978)
Buvet, P.-A., Fache, B., Rouam, A.: How does a robot speak? about the man-machine verbal interaction. In: Proceedings of The 3rd International Workshop on the Applications of Knowledge Representation and Semantic Technologies in Robotics ((AnSWeR19), CEUR. http://ceur-ws.org/Vol-2487/(2019)
Buvet, P.-A., Fache, B., Rouam, A.: Interview with a robot: How to Equip the Elderly Companion Robots with Speech?. In: Proceedings of the Future Technologies Conference (FTC 2020, 2, pp. 310–326, Springer, Heidelberg (2020). https://doi.org/10.1007/978-3-030-63089-8_20
Buvet, P.-A., Fache, B., Rouam. A.: Which intelligence for human-machine dialogue systems?. In: Proceedings of the Future Technologies Conference (FTC 2021), 1, pp. 121–133, Springer, Heidelberg (2021). https://doi.org/10.1007/978-3-030-89906-6_10
Buvet, P.-A.: Linguistique et intelligence, Etudes de linguistique appliquée, Klincksieck (in press)
Buvet, P.-A.: Prédication et relation: l’exemple de la détermination, in Le prédicat en questions, Champion, Paris (in press)
Tisseron, S. : Le jour où mon robot m’aimera, Albin Michel, Paris (2015)
Devillers, L.: Les Robots émotionnels. Santé, surveillance, sexualité… : et l’éthique dans tout ça ? Editions de l’Observatoire, Paris
Bensoussan, J., Bensoussan A.: IA, robots et droit, Liège (2019)
Pelachaud, C. (ed.): Systèmes d’interaction Émotionnelle. Lavoisier, Paris (2010)
Stock-Homburg, R.: Survey of emotions in human-robot interactions perspectives from robotic psychology on 20 years of research. Int. J. Sociologic Robotics 14, 389–411 (2022)
Kansizoglou, I., Bampis, L., Gasteratos, A.: An active learning paradigm for online audio-visual emotion recognition. IEEE Transactions on Affective Computing (2019). https://doi.org/10.1109/TAFFC.2019.2961089
Guanghui, C., Xiaoping, Z.: Multi-modal emotion recognition by fusing correlation features of speech-visual. IEEE Signal Process. Lett. 28, 533–537 (2021)
Amatayakul, S., AlbertNicole, G.: Surmonter ses émotions, conquérir son destin. Réflexions sur l’éthique de Descartes. Diogène 237(1), 109–120 (2012)
Coppin, G., Sander, D.: Théories et Concepts Contemporains en Psychologie de l’émotion in Systèmes d’interaction Émotionnelle, pp. 25–56. Lavoisier, Paris (2010)
Damasio, R.A.: Looking for Spinoza: Joy, Sorrow and the Feeling Brain. Harcourt, Boston (2003)
Rizzolatti, G.S., C.: Les Neurones Miroirs. Odile Jacob, Paris (2008)
Buvet, P.-A., Girardin, C., Gross, G., Groud, C.: Les prédicats d’affect, LIDIL, 32, Presses Universitaires de Grenoble, pp. 125–143 (2005)
Gross, M.: Les bases empiriques de la notion de prédicat sémantique, Langages, 63, Larousse, pp. 7–52 (1981)
Damasio, R.A.: Self Comes to Mind: Constructing the Conscious Brain. Knopf Doubleday Publishing Group, New York (2002)
VOSK Offline Speech Recognition API: https://alphacephei.com/vosk/ Accessed 28 Mar 2022
Mary, L.: Extraction and Representation of Prosody for Speaker, Speech and Language Recognition. Springer, Heidelberg (2015). https://doi.org/10.1007/978-1-4614-1159-8
Chapelle, O., Scholkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Buvet, PA., Fache, B., Fadel, W., Rouam, A. (2023). How Does a Social Robot Analyze Emotions?. In: Arai, K. (eds) Proceedings of the Future Technologies Conference (FTC) 2022, Volume 3. FTC 2022 2022. Lecture Notes in Networks and Systems, vol 561. Springer, Cham. https://doi.org/10.1007/978-3-031-18344-7_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-18344-7_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18343-0
Online ISBN: 978-3-031-18344-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)