Skip to main content
SpringerLink
Log in

Acceptance and use of a multi-modal avatar-based tool for remediation of social cognition deficits

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

This paper focuses on the validation of a tool designed to improve affect recognition, a fundamental aspect of social cognition as it greatly affects the functionality and quality of life of patients with mental disorders. The presented tool facilitates the generation of multi-modal avatar-based therapies by mental health professionals in this important clinical domain. Moreover, the tool for remediation of social cognitive deficits may be customised to each patient’s impairment. This paper describes how the tool was assessed by therapists after viewing a video explaining its most relevant aspects. The participants were asked to fill in a questionnaire based on UTAUT2 for the study of the acceptance and use of this technology. In light of the results obtained from 41 therapists about their intention of use, the most important statement is that their interest for this kind of tools is high. Nonetheless, there are some factors that negatively affect their behavioural intention.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abirached B, Zhang Y, Aggarwal JK, Fernandes T, Carlos J, Orvalho V (2011) Improving communication skills of children with ASDs through interaction with virtual characters. In: IEEE 1st international conference on serious games and applications for health, IEEE, pp 3493–3502

  • Adery LH, Ichinose M, Torregrossa LJ, Wade J, Nichols H, Bekele E, Bian D, Gizdic A, Granholm E, Sarkar N, Park S (2018) The acceptability and feasibility of a novel virtual reality based social skills training game for schizophrenia: preliminary findings. Psychiatry Res 270:496–502. https://doi.org/10.1016/j.psychres.2018.10.014

    Article  Google Scholar 

  • Akalin N, Kiselev A, Kristoffersson A, Loutfi A (2017) An evaluation tool of the effect of robots in eldercare on the sense of safety and security. In: Kheddar A, Yoshida E, Ge SS, Suzuki K, Cabibihan J-J, Eyssel F, He H (eds) International conference on social robotics. Springer, Cham, pp 628–637

    Chapter  Google Scholar 

  • Barclay D, Higgins C, Thompson R (1995) The partial least squares (PLS) approach to casual modeling: personal computer adoption ans use as an illustration. Technol Stud 2:285–309

    Google Scholar 

  • Baron-Cohen S, Wheelwright S, Skinner R, Martin J, Clubley E (2001) The autism-spectrum quotient (AQ): evidence from asperger syndrome/high-functioning autism, malesand females, scientists and mathematicians. J Autism Dev Disord 31(1):5–17. https://doi.org/10.1023/A:1005653411471

    Article  Google Scholar 

  • Beer JM, Fisk AD, Rogers WA (2009) Emotion recognition of virtual agents facial expressions: the effects of age and emotion intensity. Proc Hum Factors Ergon Soc 53(2):131–135. https://doi.org/10.1177/154193120905300205

    Article  Google Scholar 

  • Brothers L (1990) The neural basis of primate social communication. Motiv Emot 14(2):81–91. https://doi.org/10.1007/BF00991637

    Article  Google Scholar 

  • Castillo JC, Castro-González Á, Fernández-Caballero A, Latorre JM, Pastor JM, Fernández-Sotos A, Salichs MA (2016) Software architecture for smart emotion recognition and regulation of the ageing adult. Cogn Comput 8(2):357–367. https://doi.org/10.1007/s12559-016-9383-y

    Article  Google Scholar 

  • Cerezo E, Hupont I, Baldassarri S, Ballano S (2012) Emotional facial sensing and multimodal fusion in a continuous 2d affective space. J Ambient Intell Hum Comput 3(1):31–46. https://doi.org/10.1007/s12652-011-0087-6

    Article  Google Scholar 

  • Fernández-Caballero A, Latorre JM, Pastor JM, Fernández-Sotos A (2014) Improvement of the elderly quality of life and care through smart emotion regulation. In: Pecchia L, Chen LL, Nugent C, Bravo J (eds) Ambient assisted living and daily activities. Springer, Cham, pp 348–355. https://doi.org/10.1007/978-3-319-13105-4_50

    Chapter  Google Scholar 

  • Fernández-Caballero A, Martínez-Rodrigo A, Pastor JM, Castillo JC, Lozano-Monasor E, López MT, Zangr’oniz R, Latorre JM, Fernández-Sotos A (2016) Smart environment architecture for emotion detection and regulation. J Biomed Inform 64:55–73. https://doi.org/10.1016/j.jbi.2016.09.015

    Article  Google Scholar 

  • Fernández-Caballero A, Navarro E, Fernández-Sotos P, González P, Ricarte JJ, Latorre JM, Rodriguez-Jimenez R (2017) Human-avatar symbiosis for the treatment of auditory verbal hallucinations in schizophrenia through virtual/augmented reality and brain–computer interfaces. Front Neuroinform 11:64. https://doi.org/10.3389/fninf.2017.00064

    Article  Google Scholar 

  • Fernández-Sotos P, Navarro E, Torio I, Dompablo M, Fernández-Caballero A, Rodriguez-Jimenez R (2018) Pharmacological interventions in social cognition deficits: a systematic mapping review. Psychiatry Res 270:57–67. https://doi.org/10.1016/j.psychres.2018.09.012

    Article  Google Scholar 

  • Fernández-Sotos P, Torio I, Fernández-Caballero A, Navarro E, González P, Dompablo M, Rodriguez-Jimenez R (2019) Social cognition remediation interventions: a systematic mapping review. PLoS One 14:e0218720

    Article  Google Scholar 

  • García AS, Molina JP, Martínez D, González P (2008) Enhancing collaborative manipulation through the use of feedback and awareness in CVEs. In: Proceedings of the 7th ACM SIGGRAPH international conference on virtual-reality continuum and its applications in industry. ACM, New York, p 32. https://doi.org/10.1145/1477862.1477904

    Chapter  Google Scholar 

  • García AS, Navarro E, Fernández-Caballero A, González P (2018) Towards the design of avatar-based therapies for enhancing facial affect recognition. In: Novais P, Jung JJ, Villarrubia G, Fernández-Caballero A, Navarro E, González P, Carneiro D, Pinto A, Campbell AT, Durães D (eds) Ambient intelligence—software and applications, 9th international symposium on ambient intelligence. Springer, Cham, pp 306–313

    Google Scholar 

  • Garrido MV, Lopes D, Prada M, Rodrigues D, Jerónimo R, Mourão RP (2016) The many faces of a face: comparing stills and videos of facial expressions in eight dimensions (SAVE database). Behav Res Methods 49(4):1343–1360. https://doi.org/10.3758/s13428-016-0790-5

    Article  Google Scholar 

  • Gottesman II (1991) Schizophrenia genesis: the origins of madness. WH Freeman/Times Books/Henry Holt & Co, New York

    Google Scholar 

  • Green MF, Penn DL, Bentall R, Carpenter WT, Gaebel W, Gur RC, Kring AM, Park S, Silverstein SM, Heinssen R (2008) Social cognition in schizophrenia: an NIMH workshop on definitions, assessment, and research opportunities. Schizophr Bull 34(6):1211–1220. https://doi.org/10.1093/schbul/sbm145

    Article  Google Scholar 

  • Hair JF Jr, Hult GTM, Ringle C, Sarstedt M (2014) A primer on partial least squares structural equation modeling (PLS-SEM). Sage Publications, Thousand Oaks

    MATH  Google Scholar 

  • Henseler J, Ringle CM, Sarstedt M (2015) A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci 43(1):115–135

    Article  Google Scholar 

  • Kandalaft MR, Didehbani N, Krawczyk DC, Allen TT, Chapman SB (2013) Virtual reality social cognition training for young adults with high-functioning autism. J Autism Dev Disord 43(1):34–44. https://doi.org/10.1007/s10803-012-1544-6

    Article  Google Scholar 

  • Khalilzadeh J, Ozturk AB, Bilgihan A (2017) Security-related factors in extended UTAUT model for NFC based mobile payment in the restaurant industry. Comput Human Behav 70:460–474

    Article  Google Scholar 

  • Krishnappa Babu PR, Lahiri U (2019) Classification approach for understanding implications of emotions using eye-gaze. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-019-01329-8

    Article  Google Scholar 

  • Lahera G, Ruiz-Murugarren S, Iglesias P, Ruiz-Bennasar C, Herrería E, Montes JM, Fernández-Liria A (2012) Social cognition and global functioning in bipolar disorder. J Nerv Menta Dis 200(2):135–141. https://doi.org/10.1097/NMD.0b013e3182438eae

    Article  Google Scholar 

  • Laumer S, Gubler F, Maier C, Weitzel T (2018) Job seekers’ acceptance of job recommender systems: results of an empirical study. In: Proceedings of the 51st Hawaii international conference on system sciences. Curran Associates, Inc., pp 3172–3181

  • Lozano-Monasor E, López MT, Vigo-Bustos F, Fernández-Caballero A (2017) Facial expression recognition in ageing adults: from lab to ambient assisted living. J Ambient Intell Hum Comput 8(4):567–578. https://doi.org/10.1007/s12652-017-0464-x

    Article  Google Scholar 

  • Manis KT, Choi D (2018) The virtual reality hardware acceptance model (vr-ham): Extending and individuating the technology acceptance model (TAM) for virtual reality hardware. J Bus Res. https://doi.org/10.1016/j.jbusres.2018.10.021

    Article  Google Scholar 

  • Marcoulides GA, Saunders C (2006) PLS: a silver bullet? Manag Inf Syst Q 30(2):1

    Article  Google Scholar 

  • McDuff D, Mahmoud A, Mavadati M, Amr M, Turcot J, Kaliouby Re (2016) Affdex sdk: a cross-platform real-time multi-face expression recognition toolkit. In: Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems. ACM, pp 3723–3726

  • Mendoza-Palechor F, Menezes ML, Sant’Anna A, Ortiz-Barrios M, Samara A, Galway L (2018) Affective recognition from EEG signals: an integrated data-mining approach. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-018-1065-z

    Article  Google Scholar 

  • Oechslein O, Fleischmann M, Hess T (2014) An application of utaut2 on social recommender systems: incorporating social information for performance expectancy. In: 2014 47th Hawaii international conference on system sciences. IEEE, pp 3297–3306

  • Oliver M, Teruel M, Molina J, Romero-Ayuso D, González P (2018) Ambient intelligence environment for home cognitive telerehabilitation. Sensors 18(11):3671

    Article  Google Scholar 

  • Pinkham AE, Penn DL, Green MF, Buck B, Healey K, Harvey PD (2014) The social cognition psychometric evaluation study: results of the expert survey and RAND panel. Schizophr Bull 40(4):813–823. https://doi.org/10.1093/schbul/sbt081

    Article  Google Scholar 

  • Pinkham AE, Penn DL, Green MF, Harvey PD (2016) Social cognition psychometric evaluation: results of the initial psychometric study. Schizophr Bull 42(2):494–504. https://doi.org/10.1093/schbul/sbv056

    Article  Google Scholar 

  • Roark DA, Barrett SE, Spence MJ, Abdi H, O’Toole AJ (2003) Psychological and neural perspectives on the role of motion in face recognition. Behav Cogn Neurosci Rev 2(1):15–46. https://doi.org/10.1177/1534582303002001002

    Article  Google Scholar 

  • Rosenberg H, McDonald S, Dethier M, Kessels RP, Westbrook RF (2014) Facial emotion recognition deficits following moderate-severe traumatic brain injury (TBI): re-examining the valence effect and the role of emotion intensity. J Int Neuropsychol Soc 20(10):994–1003. https://doi.org/10.1017/S1355617714000940

    Article  Google Scholar 

  • Rus-Calafell M, Gutiérrez-Maldonado J, Ribas-Sabaté J (2014) A virtual reality-integrated program for improving social skills in patients with schizophrenia: a pilot study. J Behav Ther Exp Psychiatry 45(1):81–89. https://doi.org/10.1016/j.jbtep.2013.09.002

    Article  Google Scholar 

  • Samara A, Galway L, Bond R, Wang H (2019) Affective state detection via facial expression analysis within a human–computer interaction context. J Ambient Intell Hum Comput 10(6):2175–2184. https://doi.org/10.1007/s12652-017-0636-8

    Article  Google Scholar 

  • Souto YM, Campo MV, Llenderrozas FD, Álvarez MR, Mateos R, Caballero AAG (2018) Randomized clinical trial with e-MotionalTraining® 1.0 for social cognition rehabilitation in schizophrenia. Front Psychiatry 26(9):40. https://doi.org/10.3389/fpsyt.2018.00040

    Article  Google Scholar 

  • Teruel MA, Navarro E, González P (2017) Exploiting awareness for the development of collaborative rehabilitation systems. Mobile Inf Syst. https://doi.org/10.1155/2017/4714328

    Article  Google Scholar 

  • Tracy JL, Randles D, Steckler CM (2015) The nonverbal communication of emotions. Curr Opin Behav Sci 3:25–30. https://doi.org/10.1016/j.cobeha.2015.01.001

    Article  Google Scholar 

  • Venkatesh V, Morris MG, Davis GB, Davis FD (2003) User acceptance of information technology: toward a unified view. MIS Q 27:425–478

    Article  Google Scholar 

  • Venkatesh V, Thong JY, Xu X (2012) Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q 36:157–178

    Article  Google Scholar 

  • Woods S, Walters M, Koay KL, Dautenhahn K (2006) Comparing human robot interaction scenarios using live and video based methods: towards a novel methodological approach. In: 9th ieee international workshop on advanced motion control, 2006. IEEE, pp 750–755

  • Yang YJD, Allen T, Abdullahi SM, Pelphrey K, Volkmar F, Chapman S (2017) Brain responses to biological motion predict treatment outcome in young adults with autism receiving virtual reality social cognition training: preliminary findings. Behav Res Ther 93:55–66. https://doi.org/10.1016/j.brat.2017.03.014

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by Spanish Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación (AEI)/European Regional Development Fund (FEDER, UE) under EmoBioFeedback (DPI2016-80894-R), Vi-SMARt (TIN2016-79100-R) and HA-SYMBIOSIS (TIN2015-72931-EXP) Grants, and by Biomedical Research Networking Centre in Mental Health (CIBERSAM) of the Instituto de Salud Carlos III.

Author information

Authors and Affiliations

  1. Instituto de Investigación en Informática de Albacete, 02071, Albacete, Spain

    Arturo S. García, Antonio Fernández-Caballero, Elena Navarro & Pascual González

  2. Complejo Hospitalario Universitario de Albacete, Servicio de Psiquiatría, Albacete, Spain

    Patricia Fernández-Sotos

  3. Departamento de Sistemas Informáticos, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    Antonio Fernández-Caballero, Elena Navarro & Pascual González

  4. Departamento de Psicología, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    José M. Latorre

  5. Instituto de Investigación Sanitaria Hospital 12 de Octubre (imas12), 28041, Madrid, Spain

    Roberto Rodriguez-Jimenez

  6. Biomedical Research Networking Centre in Mental Health (CIBERSAM), Madrid, Spain

    Patricia Fernández-Sotos, Antonio Fernández-Caballero, Elena Navarro, Roberto Rodriguez-Jimenez & Pascual González

  7. CogPsy-Group, Universidad Complutense de Madrid, Madrid, Spain

    Roberto Rodriguez-Jimenez

Authors
  1. Arturo S. García
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patricia Fernández-Sotos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Fernández-Caballero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elena Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. José M. Latorre
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Roberto Rodriguez-Jimenez
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pascual González
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Fernández-Caballero.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 119430 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

García, A.S., Fernández-Sotos, P., Fernández-Caballero, A. et al. Acceptance and use of a multi-modal avatar-based tool for remediation of social cognition deficits. J Ambient Intell Human Comput 11, 4513–4524 (2020). https://doi.org/10.1007/s12652-019-01418-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-019-01418-8

Keywords

Search

Navigation