Skip to main content
SpringerLink
Log in

Development and Validation of Basic Virtual Human Facial Emotion Expressions

  • Conference paper
  • First Online:
Understanding the Brain Function and Emotions (IWINAC 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11486))

Abstract

This paper introduces the design process of facial expressions on virtual humans to play basic emotions. The design of the emotions is grounded on the Facial Action Coding System that enables describing facial expressions based on Action Units. All the tools employed to attain the final human avatar expressions are detailed. Then, an experiment with healthy volunteers is carried out to validate the designed virtual human facial emotions. As result, we obtained that the faces are correctly interpreted by healthy people with an accuracy of \(83.56\%\). Thus, as recognition works quite well with this small sample of healthy people, this paper is a first step towards validating and enhancing the avatar characters generated, experimenting with a sufficient number of healthy persons, and, then, designing therapies based on human avatars to enhance facial affect recognition in patients with deficits in facial affect recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adobe: Fuse (2019). https://www.adobe.com/es/products/fuse.html

  2. Adobe: Mixamo (2019). https://www.mixamo.com

  3. Amini, R., Lisetti, C., Ruiz, G.: HapFACS 3.0: FACS-based facial expression generator for 3D speaking virtual characters. IEEE Trans. Affect. Comput. 6(4), 348–360 (2015)

    Article  Google Scholar 

  4. Autodesk: 3D Studio Max (2019). https://www.autodesk.es/products/3ds-max/overview

  5. Barkhof, E., de Sonneville, L.M., Meijer, C.J., de Haan, L.: Specificity of facial emotion recognition impairments in patients with multi-episode schizophrenia. Schizophr. Res.: Cogn. 2(1), 12–19 (2015)

    Google Scholar 

  6. Bordon, N., O’Rourke, S., Hutton, P.: The feasibility and clinical benefits of improving facial affect recognition impairments in schizophrenia: systematic review and meta-analysis. Schizophr. Res. 188, 3–12 (2017)

    Article  Google Scholar 

  7. Castillo, J.C., Fernández-Caballero, A., Castro-González, Á., Salichs, M.A., López, M.T.: A framework for recognizing and regulating emotions in the elderly. In: Ambient Assisted Living and Daily Activities, pp. 320–327 (2014)

    Google Scholar 

  8. Ekman, P.: Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life. Times Books, New York (2003)

    Google Scholar 

  9. Ekman, P., Friesen, W., Hager, J.: Facial action coding system investigator’s guide. Technical report, Research Nexus (2002)

    Google Scholar 

  10. Ekman, P., Friesen, W., Hager, J.: Facial action coding system: the manual on CD rom. Technical report, A Human Face (2002)

    Google Scholar 

  11. Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)

    Article  Google Scholar 

  12. Ekman, P., Friesen, W.V.: Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. ISHK (2003)

    Google Scholar 

  13. Fernández-Caballero, A., et al.: Human-avatar symbiosis in cognitive cybertherapies: proof of concept for auditory verbal hallucinations. In: Ubiquitous Computing and Ambient Intelligence, pp. 742–753 (2017)

    Google Scholar 

  14. Fernández-Caballero, A., Latorre, J.M., Pastor, J.M., Fernández-Sotos, A.: Improvement of the elderly quality of life and care through smart emotion regulation. In: Ambient Assisted Living and Daily Activities, pp. 348–355 (2014)

    Google Scholar 

  15. Fernández-Caballero, A., et al.: Human-avatar symbiosis for the treatment of auditory verbal hallucinations in schizophrenia through virtual/augmented reality and brain-computer interfaces. Front. Neuroinformatics 11, 64 (2017)

    Article  Google Scholar 

  16. Fernández-Sotos, P., et al.: Digital technology for Internet access by patients with early stage schizophrenia in Spain: a multicenter research study. J. Med. Internet Res. 21, e11824 (2019)

    Article  Google Scholar 

  17. Friesen, E., Ekman, P.: Facial Action Coding System: a Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  18. García, A.S., Navarro, E., Fernández-Caballero, A., González, P.: Towards the design of avatar-based therapies for enhancing facial affect recognition. In: Ambient Intelligence - Software and Applications - 9th International Symposium on Ambient Intelligence, pp. 306–313 (2019)

    Google Scholar 

  19. García-Sánchez, M., Teruel, M.A., Navarro, E., González, P., Fernández-Caballero, A.: A distributed tool to perform dynamic therapies for social cognitive deficit through avatars. In: Ubiquitous Computing and Ambient Intelligence, pp. 731–741 (2017)

    Google Scholar 

  20. Guadagno, R.E., Swinth, K.R., Blascovich, J.: Social evaluations of embodied agents and avatars. Comput. Hum. Behav. 27(6), 2380–2385 (2011)

    Article  Google Scholar 

  21. Gutiérrez-Maldonado, J., Rus-Calafell, M., González-Conde, J.: Creation of a new set of dynamic virtual reality faces for the assessment and training of facial emotion recognition ability. Virtual Reality 18(1), 61–71 (2014)

    Article  Google Scholar 

  22. Lozano-Monasor, E., López, M.T., Fernández-Caballero, A., Vigo-Bustos, F.: Facial expression recognition from webcam based on active shape models and support vector machines. In: Ambient Assisted Living and Daily Activities, pp. 147–154 (2014)

    Google Scholar 

  23. Marwick, K., Hall, J.: Social cognition in schizophrenia: a review of face processing. Br. Med. Bull. 88(1), 43–58 (2008)

    Article  Google Scholar 

  24. NVIDIA: Texture Tools for Adobe Photoshop (2019). https://developer.nvidia.com/nvidia-texture-tools-adobe-photoshop

  25. Parke, F.I.: Computer generated animation of faces. In: Proceedings of the ACM Annual Conference. vol. 1, pp. 451–457. ACM (1972)

    Google Scholar 

  26. Russell, J.A.: Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychol. Bull. 115(1), 102 (1994)

    Article  Google Scholar 

  27. Sachs, G., et al.: Training of affect recognition (TAR) in schizophrenia-impact on functional outcome. Schizophr. Res. 138(2–3), 262–267 (2012)

    Article  Google Scholar 

  28. Slater, M., Steed, A.: Meeting people virtually: experiments in shared virtual environments. In: Schroeder R. (eds.) The Social Life of Avatars, pp. 146–171 (2002)

    Google Scholar 

  29. Sokolova, M.V., Fernández-Caballero, A.: A review on the role of color and light in affective computing. Appl. Sci. 5(3), 275–293 (2015)

    Article  Google Scholar 

  30. Tessier, M.H., Gingras, C., Robitaille, N., Jackson, P.L.: Toward dynamic pain expressions in avatars: perceived realism and pain level of different action unit orders. Comput. Hum. Behav. 96, 95–109 (2019)

    Article  Google Scholar 

  31. Tinwell, A., Grimshaw, M., Williams, A.: Uncanny behaviour in survival horror games. J. Gaming Virtual Worlds 2(1), 3–25 (2010)

    Article  Google Scholar 

  32. Unity3d: Unity 3D (2019). https://unity3d.com

  33. Wölwer, W., et al.: Neurophysiological correlates of impaired facial affect recognition in individuals at risk for schizophrenia. Schizophr. Bull. 38(5), 1021–1029 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially supported by Spanish Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación (AEI)/European Regional Development Fund (FEDER, UE) under DPI2016-80894-R and TIN2015-72931-EXP grants, and by Biomedical Research Networking Centre in Mental Health (CIBERSAM) of the Instituto de Salud Carlos III.

Author information

Authors and Affiliations

  1. Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    Miguel Á. Vicente-Querol, Arturo S. García & Antonio Fernández-Caballero

  2. Departamento de Sistemas Informáticos, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    Arturo S. García & Antonio Fernández-Caballero

  3. Department of Psychiatry, Instituto de Investigación Sanitaria Hospital 12 de Octubre (imas12), Madrid, Spain

    Patricia Fernández-Sotos & Roberto Rodriguez-Jimenez

  4. CIBERSAM (Biomedical Research Networking Centre in Mental Health), Madrid, Spain

    Patricia Fernández-Sotos, Roberto Rodriguez-Jimenez & Antonio Fernández-Caballero

  5. CogPsy-Group, Universidad Complutense de Madrid, Madrid, Spain

    Roberto Rodriguez-Jimenez

Authors
  1. Miguel Á. Vicente-Querol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arturo S. García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patricia Fernández-Sotos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roberto Rodriguez-Jimenez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antonio Fernández-Caballero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Fernández-Caballero .

Editor information

Editors and Affiliations

  1. Universidad Politécnica de Cartagena, Cartagena, Spain

    José Manuel Ferrández Vicente

  2. Universidad Nacional de Educación a Distancia, Madrid, Spain

    José Ramón Álvarez-Sánchez

  3. Universidad Nacional de Educación a Distancia, Madrid, Madrid, Spain

    Félix de la Paz López

  4. Universidad Politécnica de Cartagena, Cartagena, Spain

    Javier Toledo Moreo

  5. The Ohio State University, Columbus, OH, USA

    Hojjat Adeli

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vicente-Querol, M.Á., García, A.S., Fernández-Sotos, P., Rodriguez-Jimenez, R., Fernández-Caballero, A. (2019). Development and Validation of Basic Virtual Human Facial Emotion Expressions. In: Ferrández Vicente, J., Álvarez-Sánchez, J., de la Paz López, F., Toledo Moreo, J., Adeli, H. (eds) Understanding the Brain Function and Emotions. IWINAC 2019. Lecture Notes in Computer Science(), vol 11486. Springer, Cham. https://doi.org/10.1007/978-3-030-19591-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-19591-5_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-19590-8

  • Online ISBN: 978-3-030-19591-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation