Abstract
In this article we describe a new approach to enhance presence technologies. First, we discuss the strong relationship between cognitive processes and emotions and how human physiology is uniquely affected when experiencing each emotion. Secondly, we introduce our prototype multimodal affective user interface. In the remainder of the paper we describe the emotion elicitation experiment we designed and conducted and the algorithms we implemented to analyse the physiological signals associated with emotions. These algorithms can then be used to recognise the affective states of users from physiological data collected via non-invasive technologies. The affective intelligent user interfaces we plan to create will adapt to user affect dynamically in the current context, thus providing enhanced social presence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allen, A. Roman, L. Cox, R. and Cardwell, B. (1996) Home health visits using a cable television network: user satisfaction. J Telemed Telecare 2:92–94
Baylor AL (2000) Beyond butlers: Intelligent agents as mentors. J Educ Comput Res 22:373–382
Bianchi N, Lisetti CL (2002) Modeling multimodal expression of user’s affective subjective experience. Int J User Model User-adapt Interact 12(1):49–84
Birdwhistle R (1970) Kinesics and context: essays on body motion and communication. University of Pennsylvania Press
Bower G (1981) Mood and memory. Am Psychol 36(2)
Briggs P, Burford B, Dracup C (1998) Modeling self-confidence in users of a computer system showing unrepresentative design. Int J Hum Comput Stud 49:717–742
Casanueva JS, Blake EH (2001) The effects of avatars on co-presence in a collaborative virtual environment. Technical Report CS01–02–00, Department of Computer Science, University of Cape Town, South Africa
Chovil N (1991) Discourse-oriented facial displays in conversation. Res Lang Soc Interact 25:163–194
Collet C, Vernet-Maury E, Delhomme G, Dittmar A (1997) Autonomic nervous system response patterns specificity to basic emotions. J Auton Nerv Syst 62(1–2):45–57
Colquitt JA, LePine JA, Noe RA (2000) Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. J Appl Psychol 85:678–707
Crist TM, Kaufman SB, Crampton KR (1996) Home telemedicine: a home health care agency strategy for maximizing resources. Home Health Care Manage Pract 8:1-9
Damasio A (1994) Descartes’ error. Avon, New York
Darkins AW, Carey MA (2000) Telemedicine and telehealth: principles, policies performance and pitfalls. Springer, Berlin Heidelberg New York
Derryberry D, Tucker D (1992) Neural mechanisms of emotion. J Consult Clin Psychol 60(3):329–337
Dillon C, Keogh E, Freeman J, Davidoff J (2000) Aroused and immersed: the psychophysiology of presence. In: Proceedings of 3rd International Workshop on Presence, Delft University of Technology, Delft, The Netherlands, March 2000, pp 27–28
Ekman P (1989) Handbook of social psychophysiology, pp 143–146. Wiley, Chichester
Ekman P, Levenson RW, Friesen WV (1983) Autonomic nervous system activity distinguishes between emotions. Science 221:1208–1210
Ekman P, Friesen WV (1975) Unmasking the face: a guide to recognizing emotions from facial expressions. Prentice Hall, Englewood Cliffs, NJ
Feldman Barrett L, Gross JJ, Conner Christensen T, Benvenuto M (2001) Knowing what you’re feeling and knowing what to do about it: mapping the relation between emotion differentiation and emotion regulation. Cognit Emotion 15:713–724
Frijda N (1986) The emotions. New York: Cambridge University Press. MIT Press
Goleman D (1995) Emotional intelligence. Bantam, New York
Gross JJ, Levenson RW (1997) Hiding feelings: the acute effects of inhibiting negative and positive emotions. J Abnorm Psychol 10(1):95–103
Gross JJ, Levenson RW (1995) Emotion elicitation using films. Cognit Emotion 9:87–108
Guinn C, Hubal H (2003) Extracting emotional information from the text of spoken dialog. In: Proceedings of user modeling (UM) 03 Workshop “assessing and adapting to user attitudes and affect: why, when and how?”, Pittsburgh, PA
Hagan MT, Menhaj MB (1994) Training feedforward networks with the marquardt algorithm. IEEE Trans Neural Netw 5(6): 989–993
IJsselsteijn WA (2002) Elements of a multi-level theory of presence: phenomenology, mental processing and neural correlates. In: Proceedings of PRESENCE 2002, pp 245–259 Universidade Fernando Pessoa, Porto, Portugal, 9–11 October 2002
IJsselsteijn WA, de Ridder H, Freeman J, Avons SE (2000) Presence: concept, determinants and measurement. In: Proceedings of the SPIE, Human Vision and Electronic Imaging V, 3959–76
James L (2000) Road rage and aggressive driving. Prometheus, Amherst, NY
Kalawsky RS (2000) The validity of presence as a reliable human performance metric in immersive environments. In: Proceedings of 3rd international workshop on presence, Delft University of Technology, Delft, The Netherlands
Larson J, Rodriguez C (1999) Road rage to road-wise. Tom Doherty Associates, New York
Lewis VE, Williams RN (1989) Mood-congruent vs mood-state-dependent learning: implications for a view of emotion. J Soc Behav Pers 4:157–171
Ledoux J (1992) Brain mechanisms of emotion and emotional learning. Curr Opin Neurobiol 2:191–197
Lisetti CL (1999) A user model of emotion-cognition. In Proceedings of the UM’99 Workshop on Attitude, Personality, and Emotions in User-Adapted Interaction (Banff, Canada, June 1999)
Lisetti CL, Nasoz F (2002) MAUI: a multimodal affective user interface. In: Proceedings of ACM Multimedia International Conference, Juan les Pins, France, December 2002
Lisetti CL, Nasoz F, Lerouge C, Ozyer O, Alvarez K (2003) Developing multimodal intelligent affective interfaces for tele-home health care. Int J Hum Comput Stud 59(1–2):245–255
Lombard M, Ditton T (1997) At the heart of it all: the concept of presence. J Comput Mediated Commun 3(2)
Lorenz R, Gregory RP, Davis DL (2000) Utility of a brief self-efficacy scale in clinical training program evaluation. Eval Health Prof 23:182–193
Martocchio JJ, Dulebohn J (1994) Performance feedback effects in training: the role of perceived controllability. Pers Psychol 47:357–373
Martocchio JJ, Judge TA (1997) Relationship between conscientiousness and learning in employee training: mediating influences of self-deception and self-efficacy. J Appl Psychol 82:764–773
Martocchio JJ (1994) Effects of conceptions of ability on anxiety, self-efficacy, and learning in training. J Appl Psychol 79:819–825
Minsky M (1980) Telepresence. Omni, June 1980:45–51
Mitchell TM (1997) Machine learning. McGraw-Hill
Nicol AA (1999) Presenting your findings: a practical guide for creating tables. American Physiological Association, Washington, DC
Picard RW, Healey J, Vyzas E (2001) Toward machine emotional intelligence analysis of affective physiological state. IEEE Trans Pattern Anal 23(10):1175–1191
Rozell EJ, Gardner WL (2000) Cognitive, motivation, and affective processes associated with computer-related performance: a path analysis. Comput Hum Behav 16:199–222
Takeuchi A, Nagao K (1993) Communicative facial displays as a new conversational modality. In: Proceedings of the INTERCHI’93 conference on human factors in computing systems, Amsterdam pp 187–193
Warner I (1997) Telemedicine applications for home health care. J Telemed Telecare 3:65–66
Warr P, Bunce D (1995) Trainee characteristics and the outcomes of open learning. Pers Psychol 48:347–375
Zajonc R (1984) On the primacy of affect. Am Psychol 39:117–124
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nasoz, F., Alvarez, K., Lisetti, C.L. et al. Emotion recognition from physiological signals using wireless sensors for presence technologies. Cogn Tech Work 6, 4–14 (2004). https://doi.org/10.1007/s10111-003-0143-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10111-003-0143-x