Abstract
Recent studies have been investigating behavior (of approach and avoidance) toward the interaction with agents in virtual reality. This paper aims to explore the cognitive construction of a complementary relationship with agents in virtual reality through a brief interaction in a virtual task. A sample of 53 adult participants with or attending higher education was studied, divided into a control group of 20, and two experimental groups, one of 20 and another of 13 participants. The experimental groups interacted with a virtual agent, one group knowing that it was a non-human character (NPC) that communicated verbally, and the second group being informed that it was an avatar controlled in real-time by another person. The results confirmed an effect of cognitive flexibility on the anthropomorphic and complementary perception of NPCs and the application of relational models toward NPCs. An effect of the nature of the agents (NPC vs. Avatar) in the application of relational models was mediated by the complementary anthropomorphic projection of the agents. The previous interaction with the virtual agent, perceived as an NPC, provided faster skin conductance responses with images of the NPC. These findings contribute to: (1) understanding the impact of cognitive flexibility, a neuropsychological construct, in the construction of projections and relationships with agents in virtual reality; (2) creation and use of virtual reality tools based on communication in a function-led approach; (3) critical view of social models, for the development of a scale that allows assessing anthropomorphism and complementarity in virtual reality.
Similar content being viewed by others
References
Bailenson J (2018) Protecting nonverbal data tracked in virtual reality. JAMA Pediatr 172(10):905–906. https://doi.org/10.1001/jamapediatrics.2018.1909
Bartneck C, Kulic D, Croft E, Zoghbi S (2009) Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int J Soc Robot 1:71–81. https://doi.org/10.1007/S12369-008-0001-3
Bender, H. A. & Spat-Lemus, J. (2019). Cognitive training and rehabilitation in aging and dementia. In: Ravdin L., Katzen H. (eds) Handbook on the Neuropsychology of aging and dementia. Clinical handbooks in neuropsychology. Springer, Cham. https://doi.org/10.1007/978-3-319-93497-6_24
Brito R, Waldzus S, Sekerdej M, Schubert T (2011) The contexts and structures of relating to others: How memberships in different types of groups shape the construction of interpersonal relationships. J Soc Pers Relat 28(3):406–431. https://doi.org/10.1177/0265407510384420
Bush KA, Privratsky A, Gardner J, Zielinski MJ, Kilts CD (2018) Common functional brain states encode both perceived emotion and the psychophysiological response to affective stimuli. Sci Rep 8:15444. https://doi.org/10.1038/s41598-018-33621-6
Costa, N. (2014, July). Does relational complementarity fulfill the needs to belong, trust and control in social interactions? Communication presented in European Association for Social Psychology, Amsterdam, Netherlands. https://doi.org/10.13140/2.1.3667.7125
Daher, S., Kim, K., Lee, M., Bruder, G., Schubert, R., Bailenson, J., & Welch, G. (2017a, March). Can social presence be contagious? effects of social presence priming on interaction with virtual humans. Communication presented in 2017a IEEE Symposium on 3D User Interfaces (3DUI), Los Angeles, USA. https://doi.org/10.1109/3DUI.2017.7893341
Daher, S., Kim, K., Lee, M., Schubert, R., Bruder, G., Bailensom, J., & Welch, G. (2017b). Effects of social priming on social presence with intelligent virtual agents. In J. Beskow, C. Peters, G. Castellano, C. O'Sullivan, I. Leite, S. Kopp (Eds.), Intelligent virtual agents. IVA 2017b. Lecture notes in computer science (pp.87–100). Springer, Cham. https://doi.org/10.1007/978-3-319-67401-8_1
Dotsch R, Wigboldus DH (2008) Virtual prejudice. J Exp Soc Psychol 44(4):1194–1198. https://doi.org/10.1016/j.jesp.2008.03.003
Emmelkamp PMG, Meyerbröker K, Morina N (2020) Virtual Reality Therapy in Social Anxiety Disorder. Curr Psychiatry Rep 22(7):32. https://doi.org/10.1007/s11920-020-01156-1
Epley N, Waytz A, Cacioppo JT (2007) On seeing human: A three-factor theory of anthropomorphism. Psychol Rev 114(4):864–886. https://doi.org/10.1037/0033-295X.114.4.864
Favre M, Sornette D (2013) Categorization of exchange fluxes explains the four relational models. PLoS ONE 10(3):e0120882. https://doi.org/10.1371/journal.pone.0120882
Fiske AP (2000) Complementarity theory: Why human social capacities evolved to require cultural complements. Pers Soc Psychol Rev 4(1):76–94. https://doi.org/10.1207/S15327957PSPR0401_7
Fox J, Ahn SJ, Janssen JH, Yeykelis L, Segovia KY, Bailenson JN (2015) Avatars versus agents: A meta-analysis quantifying the effect of agency on social influence. Human-Computer Interaction 30(5):401–432. https://doi.org/10.1080/07370024.2014.921494
Gamito P, Oliveira O, Morais D, Coelho C, Santos N, Alves, … Brito, R. (2019) Cognitive stimulation of elderly individuals with instrumental virtual reality-based activities of daily life: Pre-post treatment study. Cyberpsychol Behav Soc Netw 22(1):69–75. https://doi.org/10.1089/cyber.2017.0679
Gatti E, Calzolari E, Maggioni E, Obrist M (2018) Emotional ratings and skin conductance response to visual, auditory and haptic stimuli. Scientific Data 5:180120. https://doi.org/10.1038/sdata.2018.120
Hall-McMaster S, Muhle-Karbe PS, Myers NE, Stokes MG (2019) Reward boosts neural coding of task rules to optimize cognitive flexibility. J Neurosci 39(43):8549–8561. https://doi.org/10.1523/JNEUROSCI.0631-19.2019
Harvey, P. D. (2019). Domains of cognition and their assessment. Dialogues in Clinical Neuroscience, 21(3), 227–237. https://doi.org/10.31887/DCNS.2019.21.3/pharvey
Herrera F, Bailenson J, Weisz E, Ogle E, Zaki J (2018) Building long-term empathy: A largescale comparison of traditional and virtual reality perspective-taking. PLoS ONE 13(10):e0204494. https://doi.org/10.1371/journal.pone.0204494
Ho A, Hancock J, Miner AS (2018) Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. J Commun 68(4):712–733. https://doi.org/10.1093/joc/jqy026
Iacoboni M, Lieberman MD, Knowlton BJ, Molnar-Szakacs I, Moritz M, Throop CJ, Fiske AP (2004) Watching social interactions produces dorsomedial prefrontal and medial parietal BOLD fMRI signal increases compared to a resting baseline. Neuroimage 21:1167–1173. https://doi.org/10.1016/j.neuroimage.2003.11.013
Imhoff R, Dotsch R (2013) Effects do we look like me or like us? Visual projection as self- or ingroup-projection. Soc Cogn 31(6):806–816. https://doi.org/10.1521/soco.2013.31.6.806
Kim K, Maloney D, Bruder G, Bailenson JN, Welch GF (2017) The effects of virtual human’s spatial and behavioral coherence with physical objects on social presence in AR. Computer Animation and Virtual Worlds 28:e1771. https://doi.org/10.1002/cav.1771
Kim, K., Boelling, L., Haesler, S., Bailenson, J. N., Bruder, G., & Welch, G. F. (2018, October). Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in ar. Communication presented in Proceedings of the 17th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2018), Munich, Germany.
Loon A, Bailenson J, Zaki J, Bostick J, Willer R (2018) Virtual reality perspective-taking increases cognitive empathy for specific others. PLoS ONE 13(8):e0202442. https://doi.org/10.1371/journal.pone.0202442
McCall C, Singer T (2015) Facing off with unfair others: introducing proxemic imaging as an implicit measure of approach and avoidance during social interaction. PLoS ONE 10(2):e0117532. https://doi.org/10.1371/journal.pone.0117532
McGraw AP, Tetlock PE (2005) Taboo trade-offs, relational framing, and the acceptability of exchanges. J Consum Psychol 15(1):2–15. https://doi.org/10.1207/s15327663jcp1501_2
Mepham KD, Martinovic B (2018) Multilingualism and out-group acceptance: The mediating roles of cognitive flexibility and deprovincialization. J Lang Soc Psychol 37(1):51–73. https://doi.org/10.1177/0261927X17706944
Oh CS, Bailenson JN, Welch GF (2018) A systematic review of social presence: definition, antecedents, and implications. Frontiers in Robotics and AI 5:114. https://doi.org/10.3389/frobt.2018.00114
Olfers, K. J. F., & Band, G. P. H. (2018). Game-based training of flexibility and attention improves task-switch performance: near and far transfer of cognitive training in an EEG study. Psychological Research, 82, 186–202. https://doi.org/10.1007/s00426-017-0933-z
Oliveira, J., Gamito, P., Alghazzawi, D. M., Fardoun, H. M., Rosa, P. J., Sousa, T., ... Lopes, P. (2017). Performance on naturalistic virtual reality tasks depends on global cognitive functioning as assessed via traditional neurocognitive tests. Applied Neuropsychology: Adult. https://doi.org/10.1080/23279095.2017.1349661
Parsons TD (2015) Virtual Reality for Enhanced Ecological Validity and Experimental Control in the Clinical, Affective and Social Neurosciences. Front Hum Neurosci 9:660. https://doi.org/10.3389/fnhum.2015.00660
Rokeach M (1948) Generalized mental rigidity as a factor in ethnocentrism. J Abnorm Soc Psychol 43:259–278. https://doi.org/10.1037/h0056134
Rosenfield N, Lamkin K, Re J, Day K, Boyd L, Linstead E (2019) A virtual reality system for practicing conversation skills for children with autism. Multimodal Technologies Interact 3:28. https://doi.org/10.3390/mti3020028
Schul Y, Vinokur AD (2000) Projection in person perception among spouses as a function of the similarity in their shared experiences. Pers Soc Psychol Bull 26(8):987–1001. https://doi.org/10.1177/01461672002610008
Shriram, K., Oh, S.Y., & Bailenson, J. N. (2017). Virtual reality and prosocial behavior, in Burgoon, J.K., Magnenat-Thalmann, N., Pantic, M. & Vinciarelli, A. (Eds.), Social signal processing (304–316). Cambridge University Press, Cambridge. https://doi.org/10.1017/9781316676202.022
Sundararajan L (2015) Indigenous psychology: Grounding science in culture, why and how? Journal of the Theory of Social Behaviour 45(1):64–81. https://doi.org/10.1111/jtsb.12054
Verweij M, Senior TJ, Domínguez JF, Turner R (2015) Emotion, rationality, and decision-making: how to link affective and social neuroscience with social theory. Front Neurosci 9:332. https://doi.org/10.3389/fnins.2015.00332
Waytz A, Cacioppo J, Epley N (2014) Who sees human? The stability and importance of individual differences in anthropomorphism. Perspect Psychol Sci 5(3):219–232. https://doi.org/10.1177/1745691610369336
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix
Appendix A
2.1 Images used in the skin conductance evaluation task
Appendix B
3.1 Experimental procedure phases
Control group | Experimental group with an NPC in VR | Experimental group with an NPC in VR as an avatar | |
---|---|---|---|
Phase 1 | Wisconsin Card Sorting Test | ||
Phase 2 | – | Cooperative task with a VA | |
Phase 3 | Task of visualizing images while measuring skin conductance | ||
Phase 4 | Social psychology assessment battery | ||
Phase 5 | Debriefing |
Appendix C
4.1 Script used in the virtual reality task
Agent says: "Good morning, I'm Ana. Are you also from Lusófona?".
0.5 s after a window of 3 options appears:
"Yes, I am a Lusófona student";
"No, but I've been";
"No, I've never been";
0.5 s after reply:
Agent says, “Okay. So… we were asked to translate, format and work with images of a paper in Castilian. What part do you prefer to do?”.
0.5 s after a window of 3 options appears:
"Okay, I'll take care of formatting and image";
"Okay, I'll handle the translation";
"Okay, I'll take care of the formatting and image, but as you translate the text, you could format the text boxes";
0.5 s after reply:
Agent says: “Okay, I'll do the rest. But look… the paper has no title and as it is about recycling and health, I thought of calling it… recycling behaviors. Any ideas to improve this title?”.
0.5 s after the answer in the options window:
“Impact of recycling behaviors on health”;
“Behaviors of recycling and public health”;
“Public health: Recycling behaviors”;
0.5 s after reply:
Agent says: “Good idea! The paper will be excellent. It was a pleasure to work with you!”.
0.5 s after the answer in the options window:
"So do I.";
“It was also a pleasure from me”;
"Ok, everything is in order";
End of task, indicating the end of the task;
Rights and permissions
About this article
Cite this article
Abril, T., Oliveira, J. & Gamito, P. Construction and effect of relationships with agents in a virtual reality environment. Virtual Reality 27, 3665–3678 (2023). https://doi.org/10.1007/s10055-022-00669-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-022-00669-9