Skip to main content
SpringerLink
Log in

Influence of the Level of Immersion in Emotion Recognition Using Virtual Humans

  • Conference paper
  • First Online:
Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications (IWINAC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13258))

  • 1371 Accesses

Abstract

Facial affect recognition is a skill that involves the ability to perceive and distinguish between different affective facial expressions. This skill is crucial in day-to-day human relationships, and it is negatively affected by various psychiatric disorders. This has led to the development of tests that attempt to measure or train this skill. Virtual Reality (VR) is a technology that has been used for this purpose, as it can immerse the user in a virtual world that can be more similar to the real world than a traditional desktop computer. Therefore, the objective of this paper is twofold. First, a potential improvement in emotion recognition using VR instead of a traditional desktop screen condition is studied. Secondly, the impact that the presentation angle of the virtual faces has on emotion recognition is investigated. To this end, the same application was adapted to be shown in a desktop screen and in a VR environment to 36 mentally healthy participants recruited for each condition. A set of dynamic virtual faces (DVFs) were used as stimuli to show the 6 basic emotions plus the neutral expression. The DVFs were also rotated to be presented to the participants in front and side views. The results show a slight improvement in the VR environment. In addition, the accuracy is better in the front views than in the side views for both conditions, which is consistent with previous studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baceviciute, S., Terkildsen, T., Makransky, G.: Remediating learning from non-immersive to immersive media: using EEG to investigate the effects of environmental embeddedness on reading in virtual reality. Comput. Educ. 164, 104122 (2021)

    Google Scholar 

  2. Barfield, W., Zeltzer, D., Sheridan, T., Slater, M.: Presence and performance within virtual environments. In: Virtual Environments and Advanced Interface Design, pp. 473–513 (1995)

    Google Scholar 

  3. Burdea, G.C., Coiffet, P.: Virtual Reality Technology. Wiley (2003)

    Google Scholar 

  4. Busin, Y., Lukasova, K., Asthana, M.K., Macedo, E.C.: Hemiface differences in visual exploration patterns when judging the authenticity of facial expressions. Front. Psychol. 8, 2332 (2018)

    Article  Google Scholar 

  5. Castillo, J.C., Fernández-Caballero, A., Castro-González, Á., Salichs, M.A., López, M.T.: A framework for recognizing and regulating emotions in the elderly. In: Pecchia, L., Chen, L.L., Nugent, C., Bravo, J. (eds.) IWAAL 2014. LNCS, vol. 8868, pp. 320–327. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13105-4_46

  6. Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  7. Fernández-Sotos, P., García, A.S., Vicente-Querol, M.A., Lahera, G., Rodriguez-Jimenez, R., Fernández-Caballero, A.: Validation of dynamic virtual faces for facial affect recognition. PLoS ONE 16(1 1), 1–15 (2021)

    Google Scholar 

  8. García, A.S., Fernández-Sotos, P., Vicente-Querol, M.A., Lahera, G., Rodriguez-Jimenez, R., Fernandez-Caballero, A.: Design of reliable virtual human facial expressions and validation by healthy people. Integr. Comput.-Aided Eng. 27(3), 287–299, 104122 (2020)

    Google Scholar 

  9. García, A.S., Martínez, D., Molina, J.P., González, P.: Collaborative virtual environments: you can’t do it alone, can you? In: International Conference on Virtual Reality, pp. 224–233. Springer (2007)

    Google Scholar 

  10. García, A.S., et al.: A collaborative workspace architecture for strengthening collaboration among space scientists. In: 2015 IEEE Aerospace Conference, pp. 1–12. IEEE (2015)

    Google Scholar 

  11. García, A.S., et al.: Acceptance and use of a multi-modal avatar-based tool for remediation of social cognition deficits. J. Ambient. Intell. Humaniz. Comput. 11(11), 4513–4524, 104122 (2019). https://doi.org/10.1007/s12652-019-01418-8

  12. Guo, K., Shaw, H.: Face in profile view reduces perceived facial expression intensity: an eye-tracking study. Acta Physiol. (Oxf) 155, 19–28 (2015)

    Google Scholar 

  13. Gutiérrez-Maldonado, J., Rus-Calafell, M., González-Conde, J.: Creation of a new set of dynamic virtual reality faces for the assessment and training of facial emotion recognition ability. Virtual Reality 18(1), 61–71 (2013). https://doi.org/10.1007/s10055-013-0236-7

    Article  Google Scholar 

  14. Heldal, I., Steed, A., Schroeder, R.: Evaluating collaboration in distributed virtual environments for a puzzle-solving task. In: HCI International (2005)

    Google Scholar 

  15. Kohler, C.G., et al.: Facial emotion recognition in schizophrenia: intensity effects and error pattern. Am. J. Psychiatry 160(10), 1768–1774, 104122 (2003)

    Google Scholar 

  16. Lozano-Monasor, E., López, M.T., Vigo-Bustos, F., Fernández-Caballero, A.: Facial expression recognition in ageing adults: from lab to ambient assisted living. J. Ambient. Intell. Humaniz. Comput. 8(4), 567–578 (2017). https://doi.org/10.1007/s12652-017-0464-x

    Article  Google Scholar 

  17. Lozano-Monasor, E., López, M.T., Fernández-Caballero, A., Vigo-Bustos, F.: Facial expression recognition from webcam based on active shape models and support vector machines. In: Pecchia, L., Chen, L.L., Nugent, C., Bravo, J. (eds.) IWAAL 2014. LNCS, vol. 8868, pp. 147–154. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13105-4_23

  18. Marín-Morales, J., Llinares, C., Guixeres, J., Alcañiz, M.: Emotion recognition in immersive virtual reality: From statistics to affective computing. Sensors 20(18), 5163 (2020)

    Article  Google Scholar 

  19. Marwick, K., Hall, J.: Social cognition in schizophrenia: a review of face processing. Br. Med. Bull. 88(1), 43–58 (2008)

    Article  Google Scholar 

  20. Muros, N.I., et al.: Facial affect recognition by patients with schizophrenia using human avatars. J. Clin. Med. 10(9), 1904, 104122 (2021)

    Google Scholar 

  21. Rapuano, M., Sbordone, F.L., Borrelli, L.O., Ruggiero, G., Iachini, T.: The effect of facial expressions on interpersonal space: a gender study in immersive virtual reality. In: Esposito, A., Faundez-Zanuy, M., Morabito, F.C., Pasero, E. (eds.) Progresses in Artificial Intelligence and Neural Systems. SIST, vol. 184, pp. 477–486. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-5093-5_40

  22. Roberts, D., Wolff, R., Otto, O., Steed, A.: Constructing a gazebo: supporting teamwork in a tightly coupled, distributed task in virtual reality. Presence 12(6), 644–657 (2003)

    Article  Google Scholar 

  23. Russell, J.A.: Is there universal recognition of emotion from facial expression? a review of the cross-cultural studies. Psychol. Bull. 115(1), 102 (1994)

    Article  CAS  Google Scholar 

  24. Sandín, B., Chorot, P., Lostao, L., Joiner, T.E., Santed, M.A., Valiente, R.M.: Escalas PANAS de afecto positivo y negativo: Validacion factorial y convergencia transcultural. Psicothema 11(1), 37–51 (1999)

    Google Scholar 

  25. Slater, M., Linakis, V., Usoh, M., Kooper, R.: Immersion, presence and performance in virtual environments: an experiment with tri-dimensional chess. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 163–172 (1996)

    Google Scholar 

  26. Stevens, J.A., Kincaid, J.P., et al.: The relationship between presence and performance in virtual simulation training. Open J. Modelling Simul. 3(02), 41, 104122 (2015)

    Google Scholar 

  27. Surcinelli, P., Andrei, F., Montebarocci, O., Grandi, S.: Emotion recognition of facial expressions presented in profile. Psychological Reports (2021)

    Google Scholar 

  28. Watson, D., Clark, L.A., Tellegen, A.: Development and validation of brief measures of positive and negative affect: the PANAS scales. J. Pers. Soc. Psychol. 54(6), 1063–1070, 104122 (1988)

    Google Scholar 

  29. del Águila, J., González-Gualda, L.M., Játiva, M.A., Fernández-Sotos, P., Fernández-Caballero, A., García, A.S.: How interpersonal distance between avatar and human influences facial affect recognition in immersive virtual reality. Front. Psychol. 12, 675515 (2021)

    Google Scholar 

Download references

Acknowledgements

Grants PID2020-115220RB-C21 and EQC2019-006063-P funded by MCIN/AEI/ 10.13039/ 501100011033 and by “ERDF A way to make Europe”. This work was also partially supported by CIBERSAM of the Instituto de Salud Carlos III.

Author information

Authors and Affiliations

  1. Instituto de Investigación en Informática, Unidad de Neurocognición y Emoción, 02071, Albacete, Spain

    Miguel A. Vicente-Querol, Antonio Fernández-Caballero, José P. Molina, Pascual González & Arturo S. García

  2. Universidad de Castilla-La Mancha, Departamento de Sistemas Informáticos, 02071, Albacete, Spain

    Antonio Fernández-Caballero, José P. Molina, Pascual González & Arturo S. García

  3. Biomedical Research Networking Centre in Mental Health (CIBERSAM), 28029, Madrid, Spain

    Antonio Fernández-Caballero, Pascual González & Patricia Fernández-Sotos

  4. Complejo Hospitalario Universitario de Albacete, Servicio de Salud Mental, 02006, Albacete, Spain

    Luz M. González-Gualda & Patricia Fernández-Sotos

Authors
  1. Miguel A. Vicente-Querol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Fernández-Caballero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José P. Molina
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pascual González
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Luz M. González-Gualda
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Patricia Fernández-Sotos
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Arturo S. García
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arturo S. García .

Editor information

Editors and Affiliations

  1. Universidad Politécnica de Cartagena, Cartagena, Spain

    José Manuel Ferrández Vicente

  2. Universidad Nacional de Educación a Distancia, Madrid, Spain

    José Ramón Álvarez-Sánchez

  3. Universidad Nacional de Educación a Distancia, Madrid, Spain

    Félix de la Paz López

  4. Ohio State University, Columbus, OH, USA

    Hojjat Adeli

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vicente-Querol, M.A. et al. (2022). Influence of the Level of Immersion in Emotion Recognition Using Virtual Humans. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications. IWINAC 2022. Lecture Notes in Computer Science, vol 13258. Springer, Cham. https://doi.org/10.1007/978-3-031-06242-1_46

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06242-1_46

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06241-4

  • Online ISBN: 978-3-031-06242-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation