Skip to main content
SpringerLink
Log in

Emotional responses to watching and touching 3d emotional face in a virtual environment

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Facial expressions play a crucial role in modulating the emotional responses in the viewers. Touch is an important factor in shaping human emotions and social communication. The objective of this study is to investigate the effects of viewing and touching a virtual emotional face on the emotional responses of a viewer/toucher. In the case of touching the model, the effects of physical properties, namely stiffness and texture, are examined. Emotional facial expressions for neutrality, anger, fear, disgust, happiness, surprise, and sadness are developed and experimentally validated for the visual stimuli whereas four combinations of stiffness/texture properties are examined for the physical properties (low/high stiffness and smooth/rough texture). 25 participants viewed and touched the virtual emotional face and reported their respective emotional responses. The results showed that watching angry, happy, surprised, and sad faces significantly increased their anger, happiness, surprise, and sadness levels, respectively (p < 0.05). Watching a scared or a sad face significantly modulated the participants’ surprise levels (p < 0.05). On the other hand, viewing and touching an angry face significantly reduced the surprise level in the toucher (p < 0.05). As for differences based on physical properties, our results suggested that viewing and touching the disgusted face significantly modulated sadness. In particular, high stiffness/rough texture condition resulted in a significant increase in sadness while viewing and touching the disgusted face, compared to the high stiffness/smooth texture condition (p < 0.01). These conclusions suggest that viewing and touching an emotional face in a virtual environment modulates the emotional responses in the viewer/toucher. Findings of this study help the field of virtual reality to expand to a greater understanding of building emotionally compelling interpersonal interactions in the virtual environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://unity.com/.

  2. https://www.turbosquid.com/3d-models/3d-model-male-head-morph-targets/261694.

  3. https://www.oculus.com/rift/.

  4. https://www.leapmotion.com/.

  5. https://www.autodesk.com/products/3ds-max/overview.

References

  • Abdi H (2010) Holm’s sequential Bonferroni procedure. Encycl Res Des 1(8):1–8

    Google Scholar 

  • Aviezer H, Hassin RR, Ryan J, Grady C, Susskind J, Anderson A, Moscovitch M, Bentin S (2008) Angry, disgusted, or afraid? studies on the malleability of emotion perception. Psychol Sci 19(7):724–732

    Google Scholar 

  • Barnes CJ, Childs THC, Henson B, Southee CH (2007) Surface finish and touch—a case study in a new human factors tribology. Wear 257:740–750

    Google Scholar 

  • Baumeister RF, Bratslavsky E, Finkenauer C, Vohs KD (2001) Bad is stronger than good. Rev Gen Psychol 5(4):323

    Google Scholar 

  • Bernal G, Maes P (2017) Emotional beasts: visually expressing emotions through avatars in vr. In: Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems, pp 2395–2402

  • Bhatta S, Tiippana K, Vahtikari K, Hughes M, Kyttä M (2017) Exploring relationships between touch perception and surface physical properties. Front Psychol 8:367

    Google Scholar 

  • Calbi M, Heimann K, Barratt D, Siri F, Umiltà MA, Gallese V (2017) How context influences our perception of emotional faces: a behavioral study on the kuleshov effect. Front Psychol 8:1684

    Google Scholar 

  • Cardello AV, Winterhalter C, Schutz HG (2003) Predicting the handle and comfort of military clothing fabrics from sensory and instrumental data: development and application of new psychophysical methods. Text Res J 73(3):221–237

    Google Scholar 

  • Chen X, Shao F, Barnes C, Childs T, Henson B (2009) Exploring relationships between touch perception and surface physical properties. Int J Des 3(2):67–76

    Google Scholar 

  • Childs THC, Henson B (2007) Human tactile perception of screen-printed surfaces: self-report and contact mechanics experiments. J Eng Tribol 221(3):421–441

    Google Scholar 

  • Dailey MN, Cottrell GW, Padgett C, Adolphs R (2002) Empath: a neural network that categorizes facial expressions. J Cogn Neurosci 14(8):1158–1173

    Google Scholar 

  • Deng H, Hu P (2017) Matching your face or appraising the situation: two paths to emotional contagion. Front Psychol 8:2278

    Google Scholar 

  • Dimberg U, Söderkvist S (2011) The voluntary facial action technique: a method to test the facial feedback hypothesis. J Nonverbal Behav 35(1):17–33

    Google Scholar 

  • Dyck M, Winbeck M, Leiberg S, Chen Y, Gur RC, Mathiak K (2008) Recognition profile of emotions in natural and virtual faces. PLoS ONE 3(11):e3628

    Google Scholar 

  • Eid MA, Osman HA (2016) Affective haptics: current research and future directions. IEEE Access 4:26–40

    Google Scholar 

  • Ekman P (1984) Expression and the nature of emotion. Approaches Emot 3:19–344

    Google Scholar 

  • Ekman P (1992) Are there basic emotions? Psychol Rev 99(3):550–553

    Google Scholar 

  • Ekman P, Friesen WV, O’sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, Krause R, LeCompte WA, Pitcairn T, Ricci-Bitti PE, et al (1987) Universals and cultural differences in the judgments of facial expressions of emotion. J Personal Soc Psychol 53(4):712

  • Essick GK, McGlone F, Dancer C, Fabricant D, Ragin Y, Phillips N, Jones T, Guest S (2010) Quantitative assessment of pleasant touch. Neurosci Biobehav Rev 34(2):192–203

    Google Scholar 

  • Etcoff NL, Magee JJ (1992) Categorical perception of facial expressions. Cognition 44(3):227–240

    Google Scholar 

  • Flack WF Jr, Laird JD, Cavallaro LA (1999) Separate and combined effects of facial expressions and bodily postures on emotional feelings. Eur J Soc Psychol 29(2–3):203–217

    Google Scholar 

  • Friesen E, Ekman P (1978) Facial action coding system: a technique for the measurement of facial movement. Palo Alto 3,

  • Gescheider GA, Hughson BA (1991) Stimulus context and absolute magnitude estimation: a study of individual differences. Percept Psychophys 50(1):45–57

    Google Scholar 

  • Hatfield E, Cacioppo JT, Rapson RL (1993) Emotional contagion. Curr Dir Psychol Sci 2(3):96–100

    Google Scholar 

  • Hess U, Fischer A (2013) Emotional mimicry as social regulation. Personal Soc Psychol Rev 17(2):142–157

    Google Scholar 

  • Holliins M, Faldowski R, Rao S, Young F (1993) Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis. Percept Psychophys 54(6):697–705

    Google Scholar 

  • Israr A, Abnousi F (2018) Towards pleasant touch: vibrotactile grids for social touch interactions. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems, pp 1–6

  • Klatzky RL, Lederman SJ (2002) Touch. In Healy AF, Proctor RW (Eds) Experimental psychology, pp 147–176

  • Klinger E, Bouchard S, Légeron P, Roy S, Lauer F, Chemin I, Nugues P (2005) Virtual reality therapy versus cognitive behavior therapy for social phobia: A preliminary controlled study. Cyberpsychol Behav 8(1):76–88

    Google Scholar 

  • Ko BC (2018) A brief review of facial emotion recognition based on visual information. Sensors 18(2):401

    Google Scholar 

  • Kret ME, Roelofs K, Stekelenburg JJ, de Gelder B (2013) Emotional signals from faces, bodies and scenes influence observers’ face expressions, fixations and pupil-size. Front Hum Neurosci 7(1):810

    Google Scholar 

  • Lane RD, Chua PM, Dolan RJ (1999) Common effects of emotional valence, arousal and attention on neural activation during visual processing of pictures. Neuropsychologia 37(9):989–997

    Google Scholar 

  • Lawrence-Wood ER (2011) Trust me, this is (n’t) scary!: How trust affects social emotional influence in threatening situations. Flinders University of South Australia, School of Psychology

  • Lederman SJ, Klatzky RL (2007) Special issue: new directions in touch. Can J Exp Psychol 61(3):169–275

    Google Scholar 

  • Lederman SJ, Klatzky RL, Abramowicz A, Salsman K, Kitada R, Hamilton C (2007) Haptic recognition of static and dynamic expressions of emotion in the live face. Psychol Sci 18(2):158–164

    Google Scholar 

  • Lederman SJ, Klatzky RL, Rennert-May E, Lee J, Ng K, Hamilton C (2008) Haptic processing of facial expressions of emotion in 2d raised-line drawings. IEEE Trans Haptics 1(1):27–38

    Google Scholar 

  • Lewis MB (2012) Exploring the positive and negative implications of facial feedback. Emotion 12(4):852

    Google Scholar 

  • Löken LS, Wessberg J, McGlone F, Olausson H (2009) Coding of pleasant touch by unmyelinated afferents in humans. Nat Neurosci 12(5):547

    Google Scholar 

  • Lu Y, Luo Y, Lei Y, Jaquess KJ, Zhou C, Li H (2016) Decomposing valence intensity effects in disgusting and fearful stimuli: an event-related potential study. Soc Neurosci 11(6):618–626

    Google Scholar 

  • Marcus G, Neuman W, MacKuen M (2015) Measuring emotional response: comparing alternative approaches to measurement. Political Sci Res Methods 5(4):733–754

    Google Scholar 

  • Min YK, Chung SC, Min BC (2005) Physiological evaluation on emotional change induced by imagination. Appl Psychophysiol Biofeedback 30(2):137–150

    Google Scholar 

  • Mumenthaler C, Sander D (2015) Automatic integration of social information in emotion recognition. J Exp Psychol Gen 144(2):392

    Google Scholar 

  • Nagano H, Okamoto S, Yamada Y (2013) Visual and sensory properties of textures that appeal to human touch. Int J Affect Eng 12(3):375–384

    Google Scholar 

  • Oatley K, Johnson-Laird PN (1987) Towards a cognitive theory of emotions. Cogn Emotion 1(1):29–50

    Google Scholar 

  • Okamoto S, Nagano H, Yamada Y (2012) Psychophysical dimensions of tactile perception of textures. IEEE Trans Haptics 6(1):81–93

    Google Scholar 

  • Ollivier R, Goupil L, Liuni M, Aucouturier JJ (2019) Enjoy the violence: Is appreciation for extreme music the result of cognitive control over the threat response system? Music Percept Interdiscip J 37(2):95–110

    Google Scholar 

  • Parkinson B, Simons G (2009) Affecting others: social appraisal and emotion contagion in everyday decision making. Personal Soc Psychol Bull 35(8):1071–1084

    Google Scholar 

  • Pell PJ, Richards A (2011) Cross-emotion facial expression aftereffects. Vis Res 51(17):1889–1896

    Google Scholar 

  • Peters K, Kashima Y (2015) A multimodal theory of affect diffusion. Psychol Bull 141(5):966

    Google Scholar 

  • Posner J, Russell JA, Peterson BS (2005) The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Develop Psychopathol 17(3):715

    Google Scholar 

  • Ravaja N, Harjunen V, Ahmed I, Jacucci G, Spapé MM (2017) Feeling touched: emotional modulation of somatosensory potentials to interpersonal touch. Sci Rep 7(1):1–11

    Google Scholar 

  • Riva G, Mantovani F, Capideville CS, Preziosa A, Morganti F, Villani D, Gaggioli A, Botella C, Alcañiz M (2007) Affective interactions using virtual reality: the link between presence and emotions. CyberPsychol Behav 10(1):45–56

    Google Scholar 

  • Russell JA (1980) A circumplex model of affect. J Personal Soc Psychol 39(6):1161

    Google Scholar 

  • Schroeder U, Hennenlotter A, Erhard P, Haslinger B, Stahl R, Lange KW, Ceballos-Baumann AO (2004) Functional neuroanatomy of perceiving surprised faces. Human Brain Mapp 23(4):181–187

    Google Scholar 

  • Seeley SH, Garcia E, Mennin DS (2015) Recent advances in laboratory assessment of emotion regulation. Curr Opin Psychol 3:58–63

    Google Scholar 

  • Singh H, Bauer M, Chowanski W, Sui Y, Atkinson D, Baurley S, Fry M, Evans J, Bianchi-Berthouze N (2014) The brain’s response to pleasant touch: an eeg investigation of tactile caressing. Front Hum Neurosci 8:893

    Google Scholar 

  • Smith C, Scott H (1997) A componential approach to the meaning of facial expressions. The psychology of facial expression 229,

  • Susskind J, Littlewort G, Bartlett M, Movellan J, Anderson A (2007) Human and computer recognition of facial expressions of emotion. Neuropsychologia 45(1):152–162

    Google Scholar 

  • Vrticka P, Lordier L, Bediou B, Sander D (2014) Human amygdala response to dynamic facial expressions of positive and negative surprise. Emotion 14(1):161

    Google Scholar 

  • Yohanan S, MacLean KE (2012) The role of affective touch in human-robot interaction: Human intent and expectations in touching the haptic creature. Int J Soc Robot 4(2):163–180

    Google Scholar 

  • Zhao K, Yan WJ, Chen YH, Zuo XN, Fu X (2013) Amygdala volume predicts inter-individual differences in fearful face recognition. PLoS ONE 8(8):e74096. https://doi.org/10.1371/journal.pone.0074096

    Article  Google Scholar 

Download references

Funding

This research was supported by New York University Abu Dhabi.

Author information

Authors and Affiliations

  1. New York University Abu Dhabi, 129188, Abu Dhabi, Saadiyat Island, United Arab Emirates

    Muhammad Hassan Jamil, Wanjoo Park & Mohamad Eid

Authors
  1. Muhammad Hassan Jamil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wanjoo Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamad Eid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamad Eid.

Ethics declarations

Conflicts of interest

The authors declare to have no conflicts of interest.

Ethical standards

The current research was approved by New York University Abu Dhabi’s local Ethics Committee (IRB # 062-2018).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A: Perceived emotions

A: Perceived emotions

(See Table 5)

Table 5 Relative rating scores for perceived emotions when participant’s watch six basic emotions from the case when watching neutral face. Wilcoxon signed rank test corrected by Bonferroni, **p < 0.01, ***p < 0.001
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jamil, M.H., Park, W. & Eid, M. Emotional responses to watching and touching 3d emotional face in a virtual environment. Virtual Reality 25, 553–564 (2021). https://doi.org/10.1007/s10055-020-00473-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-020-00473-3

Keywords

Search

Navigation