skip to main content
10.1145/3570945.3607330acmconferencesArticle/Chapter ViewAbstractPublication PagesivaConference Proceedingsconference-collections
extended-abstract

Effect of Avatar Facial Expressiveness on Team Collaboration in Virtual Reality

Published: 22 December 2023 Publication History

Abstract

This paper investigates the impact of displaying the facial expressions of a user in real time on the performance of the task, the social presence and the behavioral changes of the users interacting in a Virtual Reality environment. To evaluate this approach, we conducted a study where the users collaborated to build a TV stand in dyad including a novice and an expert assistant. The collaborative task was divided into two independent phases: a face-to-face discussion phase without object manipulation, and a furniture assembly phase with object manipulation. Our results indicate that the proposed approach can increase social presence and lead to gaze behavior changes in multi-user environments during the face-to-face phase.

References

[1]
Jeremy N Bailenson, Andrew C Beall, Jack Loomis, Jim Blascovich, and Matthew Turk. 2004. Transformed social interaction: Decoupling representation from behavior and form in collaborative virtual environments. Presence: Teleoperators & Virtual Environments 13, 4 (2004), 428--441.
[2]
Jeremy N Bailenson, Jim Blascovich, Andrew C Beall, and Jack M Loomis. 2001. Equilibrium theory revisited: Mutual gaze and personal space in virtual environments. Presence: Teleoperators & Virtual Environments 10, 6 (2001), 583--598.
[3]
Gary Bente, Felix Eschenburg, and Lisa Aelker. 2007. Effects of simulated gaze on social presence, person perception and personality attribution in avatar-mediated communication. In Presence 2007: Proceedings of the 10th Annual International Workshop on Presence, October 25-27, 2007, Barcelona, Spain. Citeseer, 207--14.
[4]
Frank Biocca, Chad Harms, and Judee K Burgoon. 2003. Toward a more robust theory and measure of social presence: Review and suggested criteria. Presence: Teleoperators & virtual environments 12, 5 (2003), 456--480.
[5]
Frank Biocca, Chad Harms, and Jenn Gregg. 2001. The networked minds measure of social presence: Pilot test of the factor structure and concurrent validity. In 4th annual international workshop on presence, Philadelphia, PA. 1--9.
[6]
Jim Blascovich. 2000. Using physiological indexes of psychological processes in social psychological research. Handbook of research methods in social and personality psychology (2000), 117--137.
[7]
Juan Casanueva and Edwin Blake. 2001. The Effects of Avatars on Co-presence in a Collaborative Virtual Environment. In Annual Conference of the South African Institute of Computer Scientists and Information Technologists (SAICSIT), September 25-28, 2001, Pretoria, South Africa. 19--28.
[8]
Deborah A Coker and Judeek Burgoon. 1987. The nature of conversational involvement and nonverbal encoding patterns. Human Communication Research 13, 4 (1987), 463--494.
[9]
Sara Dalzel-Job. 2015. Social interaction in virtual environments: the relationship between mutual gaze, task performance and social presence. (2015).
[10]
David F Dinges, Robert L Rider, Jillian Dorrian, Eleanor L McGlinchey, Naomi L Rogers, Ziga Cizman, Siome K Goldenstein, Christian Vogler, Sundara Venkataraman, and Dimitris N Metaxas. 2005. Optical computer recognition of facial expressions associated with stress induced by performance demands. Aviation, space, and environmental medicine 76, 6 (2005), B172--B182.
[11]
Paul Ekman. 1993. Facial expression and emotion. American psychologist 48, 4 (1993), 384.
[12]
Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004. Language efficiency and visual technology: Minimizing collaborative effort with visual information. Journal of language and social psychology 23, 4 (2004), 491--517.
[13]
Darren Gergle, Robert E Kraut, and Susan R Fussell. 2013. Using visual information for grounding and awareness in collaborative tasks. Human--Computer Interaction 28, 1 (2013), 1--39.
[14]
Darren Gergle, Carolyn P Rosé, and Robert E Kraut. 2007. Modeling the impact of shared visual information on collaborative reference. In Proceedings of the SIGCHI conference on Human factors in computing systems. 1543--1552.
[15]
Dimitris Giakoumis, Anastasios Drosou, Pietro Cipresso, Dimitrios Tzovaras, George Hassapis, Andrea Gaggioli, and Giuseppe Riva. 2012. Using activity-related behavioural features towards more effective automatic stress detection. (2012).
[16]
C Heeter. 1992. Being There: the subjective experience of presence, presence: teleoperators and virtual environments. (1992).
[17]
Jim Hollan and Scott Stornetta. 1992. Beyond being there. In Proceedings of the SIGCHI conference on Human factors in computing systems. 119--125.
[18]
Patrick Raoul Jermann. 2004. Computer support for interaction regulation in collaborative problem-solving. Ph. D. Dissertation. Verlag nicht ermittelbar.
[19]
Sin-Hwa Kang, James H Watt, and Sasi Kanth Ala. 2008. Communicators' perceptions of social presence as a function of avatar realism in small display mobile communication devices. In Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008). IEEE, 147--147.
[20]
Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203--220.
[21]
Jarek Krajewski, David Sommer, Udo Trutschel, Dave Edwards, and Martin Golz. 2009. Steering wheel behavior based estimation of fatigue. In Driving assessment conference, Vol. 5. University of Iowa.
[22]
Eva Krumhuber, Antony SR Manstead, Darren Cosker, Dave Marshall, Paul L Rosin, and Arvid Kappas. 2007. Facial dynamics as indicators of trustworthiness and cooperative behavior. Emotion 7, 4 (2007), 730.
[23]
Kwan Min Lee. 2004. Presence, explicated. Communication theory 14, 1 (2004), 27--50.
[24]
Guido Makransky, Lau Lilleholt, and Anders Aaby. 2017. Development and validation of the Multimodal Presence Scale for virtual reality environments: A confirmatory factor analysis and item response theory approach. Computers in Human Behavior 72 (2017), 276--285.
[25]
Divine Maloney, Guo Freeman, and Donghee Yvette Wohn. 2020. "Talking without a Voice" Understanding Non-verbal Communication in Social Virtual Reality. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1--25.
[26]
Catherine S Oh, Jeremy N Bailenson, and Gregory F Welch. 2018. A systematic review of social presence: Definition, antecedents, and implications. Frontiers in Robotics and AI (2018), 114.
[27]
Adriana Pena and Angelica De Antonio. 2010. Inferring interaction to support collaborative learning in 3D virtual environments through the user's avatar Non-Verbal Communication. International Journal of Technology Enhanced Learning 2, 1-2 (2010), 75--90.
[28]
Sam Redfern and Niall Galway. 2002. Collaborative virtual environments to support communication and community in internet-based distance education. Journal of Information Technology Education. Research 1 (2002), 201.
[29]
Daniel Roth, Constantin Klelnbeck, Tobias Feigl, Christopher Mutschler, and Marc Erich Latoschik. 2018. Beyond replication: Augmenting social behaviors in multi-user virtual realities. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 215--222.
[30]
Carlo Salvato, Jeffrey J Reuer, and Pierpaolo Battigalli. 2017. Cooperation across disciplines: A multilevel perspective on cooperative behavior in governing inter-firm relations. Academy of Management Annals 11, 2 (2017), 960--1004.
[31]
John Short, Ederyn Williams, and Bruce Christie. 1976. The social psychology of telecommunications. Toronto; London; New York: Wiley.
[32]
Richard T Stone and Chen-Shuang Wei. 2011. Exploring the linkage between facial expression and mental workload for arithmetic tasks. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 55.SAGE Publications Sage CA: Los Angeles, CA, 616--619.
[33]
Christopher D Wickens. 1984. The multiple resources model of human performance: Implications for display design. (1984).
[34]
Yuanjie Wu, Yu Wang, Sungchul Jung, Simon Hoermann, and Robert W Lindeman. 2021. Using a fully expressive avatar to collaborate in virtual reality: Evaluation of task performance, presence, and attraction. Frontiers in Virtual Reality 2 (2021), 641296.

Cited By

View all
  • (2024)Impact of Augmented Engagement Model for Collaborative Avatars on a Collaborative Task in Virtual RealityProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656684(1-9)Online publication date: 3-Jun-2024
  • (2024)Exploring Influence of Social Anxiety on Embodied Face Perception during Affective Social Interactions in VRProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673952(1-5)Online publication date: 16-Sep-2024

Index Terms

  1. Effect of Avatar Facial Expressiveness on Team Collaboration in Virtual Reality

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      IVA '23: Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents
      September 2023
      376 pages
      ISBN:9781450399944
      DOI:10.1145/3570945
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 December 2023

      Check for updates

      Author Tags

      1. Collaboration
      2. Facial expressions
      3. Non verbal communication
      4. Virtual Reality

      Qualifiers

      • Extended-abstract
      • Research
      • Refereed limited

      Funding Sources

      Conference

      IVA '23
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 53 of 196 submissions, 27%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)135
      • Downloads (Last 6 weeks)15
      Reflects downloads up to 17 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Impact of Augmented Engagement Model for Collaborative Avatars on a Collaborative Task in Virtual RealityProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656684(1-9)Online publication date: 3-Jun-2024
      • (2024)Exploring Influence of Social Anxiety on Embodied Face Perception during Affective Social Interactions in VRProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673952(1-5)Online publication date: 16-Sep-2024

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media