Skip to main content
SpringerLink
Log in

The influence of facial interface design on dynamic emotional recognition

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

The use of facial interfaces in distant communications highlights the relevance of emotional recognition. However researches on emotional facial expression (EFE) recognition are mainly based on static and posed stimuli and their results are not much transferable to daily interactions. The purpose of the present study is to compare emotional recognition of authentic EFEs with 11 different interface designs. A widget allowing participants both to recognize an emotion and to assess it on-line was used. Divided-face and compound-face interfaces are compared with a common full frontal interface. Analytic and descriptive on-line results reveal that some interfaces facilitate emotional recognition whereas others would decrease it. This study suggests that relevant interfaces could improve emotional recognition and thus facilitate distant communications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. O’Malley C, Langton S, Anderson A, Doherty-Sneddon G, Bruce V (1996) Comparison of face-to-face and video-mediated interaction. Interact Comput 8:177–192

    Article  Google Scholar 

  2. Tajariol F, Dubois M, Adam JM (2002) Mediated-communication in tutoring: When the face isn’t enough. Presented at international conference on computer and advanced technology in education, Cancun

  3. Tajariol F, Adam J-M, Dubois M (2008) Seeing the face and observing the actions: the effects of nonverbal cues on mediated tutoring dialogue. In: Woolf B, Aïmeur E, Nkambou R, Lajoie S (eds) Intelligent tutoring systems, vol. 5091. Lecture Notes in Computer Science. Springer, Berlin, pp 480–489

  4. Anderson A, Garrod SC, Clark A, Boyle E, Mullin J (1992) The human communication research centre dialogue database. J Child Lang. 19:711–711

    Article  Google Scholar 

  5. Doherty-Sneddon G, Kent G (1996) Visual signals and the communication abilities of children. J Child Psychol Psychiatry 37:949–959

    Article  Google Scholar 

  6. Doherty-Sneddon G, Anderson A, O’Malley C, Langton S, Garrod S, Bruce V (1997) Face-to-face and video-mediated communication: a comparison of dialogue structure and task performance. J Exp Psychol Appl 3:105–125

    Article  Google Scholar 

  7. Nardi BA, Kuchinsky A, Whittaker S, Leichner R, Schwarz H (1995) Video-as-data: Technical and social aspects of a collaborative multimedia application. Comput Support Coop Work (CSCW) 4:73–100

    Article  Google Scholar 

  8. Gaver WW, Sellen A, Heath C, Luff P (1993) One is not enough: Multiple views in a media space. Presented at conference on human factors in computing systems, Amsterdam

  9. Carletta J, McEwan R, Anderson A (1998) Communication in virtual supply chain teams. Globalization of manufacturing in the digital communications era of the 21st century: innovation, agility, and the virtual enterprise, pp 55–67

  10. Carles L (2001) Benefits and limits of video channel on mediated interactions. Technical report 1, Geneva Interaction Lab

  11. Ekman P (1992) An argument for basic emotions. Cogn Emotion 6:169–200

    Google Scholar 

  12. Izard CE (1994) Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychol Bull 115:288–299

    Article  Google Scholar 

  13. Hess U, Thibault P (2009) Darwin and emotion expression. Am Psychol 64:120

    Article  Google Scholar 

  14. Ekman P, Rosenberg EL (1997) What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS). Oxford University Press, New York

    Google Scholar 

  15. Wagner HL, MacDonald CJ, Manstead AS (1986) Communication of individual emotions by spontaneous facial expressions. J Pers Soc Psychol 50:737

    Article  Google Scholar 

  16. Hess U, Banse R, Kappas A (1995) The intensity of facial expression is determined by underlying affective state and social situation. J Pers Soc Psychol 69:280

    Article  Google Scholar 

  17. Fernandez-Dols JM, Sanchez F, Carrera P, Ruiz-Belda MA (1997) Are spontaneous expressions and emotions linked? An experimental test of coherence. J Nonverbal Behav 21:163–177

    Article  Google Scholar 

  18. Nummenmaa T (1992) Pure and blended emotion in the human face: psychometric experiments. Suomalainen tiedeakatemia, Helsinki

    Google Scholar 

  19. Wallbott HG, Scherer KR (1986) Cues and channels in emotion recognition. J Pers Soc Psychol 51:690–699

    Article  Google Scholar 

  20. Russell JA (1997) Reading emotions from and into faces: Resurrecting a dimensional–contextual perspective. In: Russell JA, Fernández-Dols JM (eds) The psychology of facial expression. Cambridge University Press, New York, p 320

    Chapter  Google Scholar 

  21. Tcherkassof A, Bollon T, Dubois M, Pansu P, Adam JM (2007) Facial expressions of emotions: a methodological contribution to the study of spontaneous and dynamic emotional faces. Eur J Soc Psychol 37:1325–1345

    Article  Google Scholar 

  22. Bassili JN (1978) Facial motion in the perception of faces and of emotional expression. J Exp Psychol Hum Percept Perform 4:373–379

    Article  Google Scholar 

  23. Bassili JN (1979) Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. J Pers Soc Psychol 37:2049–2058

    Article  Google Scholar 

  24. Hess U, Kleck RE (1990) Differentiating emotion elicited and deliberate emotional facial expressions. Eur J Soc Psychol 20:369–385

    Article  Google Scholar 

  25. Kamachi M, Bruce V, Mukaida S, Gyoba J, Yoshikawa S, Akamatsu S (2001) Dynamic properties influence the perception of facial expressions. Perception 30:875–887

    Article  Google Scholar 

  26. Manstead ASR, Fischer AH, Jakobs EB (1999) The social and emotional functions of facial displays. In: Philippot P, Feldman RS, Coats EJ (eds) The social context of nonverbal behavior. Cambridge University Press, New York, p 313

    Google Scholar 

  27. Ekman P, Friesen WV, Press CP (1975) Pictures of facial affect. Consulting Psychologists Press, Palo Alto

    Google Scholar 

  28. Matsumoto D, Ekman P (1995) Japanese and Caucasian facial expressions of emotion (JACFEE) and Japanese and Caucasian neutral faces (JACNeuF), intercultural and emotion research laboratory, department of psychology. San Francisco State University, San Francisco

    Google Scholar 

  29. Bould E, Morris N (2008) Role of motion signals in recognizing subtle facial expressions of emotion. Br J Psychol 99:167–189

    Article  Google Scholar 

  30. Motley MT, Camden CT (1988) Facial expression of emotion: a comparison of posed expressions versus spontaneous expressions in an interpersonal communication setting. West J Commun (includes communication reports) 52:1–22

    Google Scholar 

  31. Heath C, Luff P (1991) Disembodied conduct: communication through video in a multi-media office environment. Presented at conference on human factors in computing systems: reaching through technology. New Orleans

  32. Lemay G, Kirouac G, Lacouture Y (1995) Expressions faciales émotionnelles spontanées dynamiques et statiques: comparaison d’études de jugement catégoriel et dimensionnel. Dynamic and static spontaneous facial expressions of emotion: A comparison of categorical and dimensional judgments. Can J Behav Sci (Revue canadienne des sciences du comportement) 27:125–139

    Article  Google Scholar 

  33. Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. Presented at international conference on automatic face and gesture recognition, Grenoble

  34. Ambadar Z, Schooler JW, Cohn JF (2005) Deciphering the enigmatic face. Psychol Sci 16:403–410

    Article  Google Scholar 

  35. Back DE, Jordan TR, Thomas SM (2009) The recognition of mental states from dynamic and static facial expressions. Visual Cogn 17:1271–1286

    Article  Google Scholar 

  36. Short J, Williams E, Christie B (1976) The social psychology of telecommunications. Wiley, London

    Google Scholar 

  37. McKeown G, Valstar MF, Cowie R, Pantic M (2010) The SEMAINE corpus of emotionally coloured character interactions. Presented at international conference on multimedia and expo (ICME), Suntec

  38. Sneddon I, McRorie M, McKeown G, Hanratty J (2012) The Belfast induced natural emotion database. IEEE Trans Affect Comput 3:32–41

    Article  Google Scholar 

  39. Tcherkassof A, Dupré D, Meillon B, Mandran N, Dubois M, Adam JM (2012) DynEmo: a video database of natural facial expressions of emotions. IEEE Trans Affect Comput (submitted)

  40. Sweller J, Chandler P (1994) Why some material is difficult to learn. Cogn Instr 12:185–233

    Article  Google Scholar 

  41. Anderson AH, Smallwood L, MacDonald R, Mullin J, Fleming AM, O’Malley C (2000) Video data and video links in mediated communication: what do users value? Int J Hum Comput Stud 52:165–187

    Article  Google Scholar 

  42. Roy C, Blais C, Fiset D, Gosselin F (2010) Visual information extraction for static and dynamic facial expression of emotions: an eye-tracking experiment. J Vision 10:531–531

    Article  Google Scholar 

  43. Dupré D, Dubois M, Mandran N, Tcherkassof A (2010) Emotional facial expression interface: effects of displayed facial designs. Presented at proceedings of ErgoIA 2010 conference, Biarritz

  44. Douglas-Cowie E, Cowie R, Sneddon I, Cox C, Lowry O, McRorie M, Martin JC, Devillers L, Abrilian S, Batliner A (2007) The humaine database: addressing the collection and annotation of naturalistic and induced emotional data. Presented at affective computing and intelligent interaction, Berlin

  45. Mayer RE, Anderson RB (1992) The instructive animation: helping students build connections between words and pictures in multimedia learning. J Educ Psychol 84:444–452

    Article  Google Scholar 

  46. Meillon B, Tcherkassov A, Mandran N, Adam JM, Dubois M, Dupré D, Benoit AM, Guérin-Dugué A, Caplier A (2010) Dynemo: a corpus of dynamic and spontaneous emotional facial expressions. Presented at international workshop series on multimodal corpora, tools and resources: advances in capturing. Coding and analyzing multimodality, Malte

Download references

Author information

Authors and Affiliations

  1. Laboratoire Inter-Universitaire de Psychologie (LIP), University of Grenoble, Grenoble, France

    Michel Dubois, Damien Dupré & Anna Tcherkassof

  2. Laboratoire d’Informatique de Grenoble (LIG), University of Grenoble, Grenoble, France

    Jean-Michel Adam, Nadine Mandran & Brigitte Meillon

Authors
  1. Michel Dubois
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Damien Dupré
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Michel Adam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anna Tcherkassof
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nadine Mandran
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Brigitte Meillon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Damien Dupré.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dubois, M., Dupré, D., Adam, JM. et al. The influence of facial interface design on dynamic emotional recognition. J Multimodal User Interfaces 7, 111–119 (2013). https://doi.org/10.1007/s12193-012-0103-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-012-0103-y

Keywords

Search

Navigation