Abstract
Learning effective models from emotion-elicited physiological responses for the classification and description of emotions is increasingly required to derive accurate analysis from affective interactions. Despite the relevance of this task, there is still lacking an integrative view of existing contributions. Additionally, there is no agreement on how to deal with the differences of physiological responses across individuals, and on how to learn from flexible sequential behavior and subtle but meaningful spontaneous variations of the signals. In this work, we rely on empirical evidence to define seven principles for a robust mining physiological signals to recognize and characterize affective states. These principles compose a coherent and complete roadmap for the development of new methods for the analysis of physiological signals. In particular, these principles address the current over-emphasis on feature-based models by including critical generative views derived from different streams of research, including multivariate data analysis and temporal data mining. Additionally, we explore how to use background knowledge related with the experimental setting and psychophysiological profiles from users to shape the learning of emotion-centered models. A methodology that integrates these principles is proposed and validated using signals collected during human-to-human and human-to-robot affective interactions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Illustrative applications include: measuring human interaction with artificial agents, assisting clinical research (emotion-centered understanding of addiction, affect dysregulation, alcoholism, anxiety, autism, attention deficit, depression, drug reaction, epilepsy, menopause, locked-in syndrome, pain management, phobias and desensitization therapy, psychiatric counseling, schizophrenia, sleep disorders, and sociopathy), studying the effect of body posture and exercises in well-being, disclosing responses to marketing and suggestive interfaces, reducing conflict in schools and prisons through the early detection of hampering behavior, fostering education by relying on emotion-centered feedback to escalate behavior and increase motivation, development of (pedagogic) games, and self-awareness enhancement.
- 2.
Learning descriptive models of emotions from labeled signals should satisfy four major requirements: flexibility (descriptive models cope with the complex and variable physiological expression of emotions within and among individuals), discriminative power (descriptive models capture and enhance the different physiological responses among emotions at an individual and group level), completeness (descriptive models contain all of the discriminative properties and, when the reconstitution of the signal behavior is relevant, of flexible sequential abstractions), and usability (descriptive models are compact and the abstractions of physiological responses are easily interpretable).
- 3.
- 4.
Software available in http://web.ist.utl.pt/rmch/research/software/eda.
- 5.
scripts, data and statistical sheets available in http://web.ist.utl.pt/rmch/research/software/eda.
References
Andreassi, J.: Psychophysiology: Human Behavior and Physiological Response. Lawrence Erlbaum, Mahwah (2007)
Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, New York (2006)
Bos, D.O.: EEG-based emotion recognition the influence of visual and auditory stimuli. Emotion 57(7), 1798–1806 (2006)
Cacioppo, J., Tassinary, L., Berntson, G.: Handbook of Psychophysiology. Cambridge University Press, New York (2007)
Chang, C., Zheng, J., Wang, C.: Based on support vector regression for emotion recognition using physiological signals. In: IJCNN, pp. 1–7 (2010)
Ekman, P., Friesen, W.V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W.A., Pitcairn, T., Ricci-Bitti, P.E., Scherer, K.R., Tomita, M., Tzavaras, A.: Universals and cultural differences in the judgments of facial expressions of emotion. J. Pers. Soc. Psychol. 53, 712–717 (1988)
Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition using bio-sensors: first steps towards an automatic system. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 36–48. Springer, Heidelberg (2004)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009)
Henriques, R., Paiva, A.: Descriptive models of emotion: learning useful abstractions from physiological responses during affective interactions. In: PhyCS Special Session on Recognition of Affect Signals from PhysiologIcal Data for Social Robots (OASIS’14). SCITEPRESS, Lisbon (2014)
Henriques, R., Paiva, A.: Seven principles to mine flexible behavior from physiological signals for effective emotion recognition and description in affective interactions. In: Physiological Computing Systems (PhyCS’14). SCITEPRESS, Lisbon (2014)
Henriques, R., Paiva, A., Antunes, C.: On the need of new methods to mine electrodermal activity in emotion-centered studies. In: Cao, L., Zeng, Y., Symeonidis, A.L., Gorodetsky, V.I., Yu, P.S., Singh, M.P. (eds.) ADMI. LNCS (LNAI), vol. 7607, pp. 203–215. Springer, Heidelberg (2013)
Jerritta, S., Murugappan, M., Nagarajan, R., Wan, K.: Physiological signals based human emotion recognition: a review. In: IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA) 2011, pp. 410–415 (2011)
Katsis, C., Katertsidis, N., Ganiatsas, G., Fotiadis, D.: Toward emotion recognition in car-racing drivers: a biosignal processing approach. IEEE Trans. Syst. Man Cybern. Syst. Hum. 38(3), 502–512 (2008)
Kulic, D., Croft, E.A.: Affective state estimation for human-robot interaction. Trans. Rob. 23(5), 991–1000 (2007)
Lang, P.: The emotion probe: studies of motivation and attention. Am. Psychol. 50, 372–372 (1995)
Leite, I., Henriques, R., Martinho, C., Paiva, A.: Sensors in the wild: exploring electrodermal activity in child-robot interaction. In: HRI, pp. 41–48. ACM/IEEE (2013)
Lessard, C.S.: Signal Processing of Random Physiological Signals. Synthesis Lectures on Biomedical Engineering. Morgan and Claypool Publishers, San Rafael (2006)
Lin, J., Keogh, E., Lonardi, S., Chiu, B.: A symbolic representation of time series, with implications for streaming algorithms. In: ACM SIGMOD Workshop on DMKD, pp. 2–11. ACM, New York (2003)
Lin, J., Keogh, E.J., Lonardi, S., Chiu, B.Y.: A symbolic representation of time series, with implications for streaming algorithms. In: Zaki, M.J., Aggarwal, C.C. (eds.) DMKD, pp. 2–11. ACM (2003)
Maaoui, C., Pruski, A., Abdat, F.: Emotion recognition for human-machine communication. In: IROS, pp. 1210–1215. IEEE/RSJ (2008)
Mitsa, T.: Temporal data mining. In: DMKD. Chapman & Hall/CRC (2009)
Murphy, K.: Dynamic Bayesian networks: representation, inference and learning. Ph.D. thesis, UC Berkeley, CS Division (2002)
Oatley, K., Keltner, D., Jenkins, J.M.: Understanding Emotions. Blackwell, Cambridge (2006)
Petrantonakis, P., Hadjileontiadis, L.: Emotion recognition from EEG using higher order crossings. TITB 14(2), 186–197 (2010)
Picard, R.W.: Affective computing: challenges. Int. J. Hum. Comput. Stud. 59(1–2), 55–64 (2003)
Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)
Rani, P., Liu, C., Sarkar, N., Vanman, E.: An empirical study of machine learning techniques for affect recognition in human-robot interaction. Pattern Anal. Appl. 9(1), 58–69 (2006)
Rigas, G., Katsis, C.D., Ganiatsas, G., Fotiadis, D.I.: A user independent, biosignal based, emotion recognition method. In: Conati, C., McCoy, K., Paliouras, G. (eds.) UM 2007. LNCS (LNAI), vol. 4511, pp. 314–318. Springer, Heidelberg (2007)
Villon, O., Lisetti, C.: Toward recognizing individual’s subjective emotion from physiological signals in practical application. In: Computer-Based Medical Systems, pp. 357–362 (2007)
Vyzas, E.: Recognition of emotional and cognitive states using physiological data. Master’s thesis, MIT (1999)
Wagner, J., Kim, J., Andre, E.: From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. In: ICME, pp. 940–943. IEEE (2005)
Wu, C.K., Chung, P.C., Wang, C.J.: Extracting coherent emotion elicited segments from physiological signals. In: WACI, pp. 1–6. IEEE (2011)
Acknowledgments
This article is an extended version of our previous work [10]. This work is supported by Fundação para a Ciência e Tecnologia under the project PEst-OE/EEI/LA0021/2013 and PhD grant SFRH/BD/ 75924/2011, and by the project EMOTE from the EU 7thFramework Program (FP7/2007–2013). The authors would like to thank: Tiago Ribeiro for implementing the robots’ behavior with sharp expressiveness, Iolanda Leite and Ivo Capelo for their support during the preparation and execution of the experiments, and Arvid Kappas for his contributions on the design of the experiments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Henriques, R., Paiva, A. (2014). Learning Effective Models of Emotions from Physiological Signals: The Seven Principles. In: da Silva, H., Holzinger, A., Fairclough, S., Majoe, D. (eds) Physiological Computing Systems. PhyCS 2014. Lecture Notes in Computer Science(), vol 8908. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-45686-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-662-45686-6_9
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-45685-9
Online ISBN: 978-3-662-45686-6
eBook Packages: Computer ScienceComputer Science (R0)