skip to main content
10.1145/3441000.3441070acmotherconferencesArticle/Chapter ViewAbstractPublication PagesozchiConference Proceedingsconference-collections
research-article

Designing for Heterogeneous Cross-Device Collaboration and Social Interaction in Virtual Reality

Published: 15 February 2021 Publication History

Abstract

Collaboration in virtual reality (VR) across heterogeneous devices poses challenges for effectively supporting manipulation, navigation, and communication through different interfaces. We explored these in the design and development of a collaborative VR system that allows interaction through a mobile touchscreen device (Handheld User) and a head-mounted display (HMD User). In a qualitative evaluation, we further analyzed how these interfaces affect social roles and user interactions during collaboration. Our observations reveal that Handheld Users achieved presence in the virtual environment, despite the non-immersive interface, and assumed similar spatial positions in a conversational scenario as they would in the real world. In a collaborative building task, we found leadership roles not tied to immersion, but potentially influenced by users’ eye-level. Further, Handheld Users exhibited stronger movement patterns than HMD Users. Based on such behavioral patterns, we contribute a classification framework for Handheld Users that facilitates future analysis of interactions in shared virtual environments through handheld devices. Finally, we offer several design considerations for collaborative cross-device VR, which are based on our observations and exemplified in our presented system.

References

[1]
Steve Benford and Lennart Fahlén. 1993. A spatial model of interaction in large virtual environments. In Proceedings of the Third European Conference on Computer-Supported Cooperative Work 13–17 September 1993, Milan, Italy ECSCW’93. Springer, Springer, Dordrecht, Milan, Italy, 109–124.
[2]
Frank Biocca, Chad Harms, and Judee K. Burgoon. 2003. Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria. Presence: Teleoperators and Virtual Environments 12 (2003), 456–480. https://doi.org/10.1162/105474603322761270
[3]
Chuck Blanchard, Scott Burgess, Young Halvill, Jaron Lanier, Ann Lasko, Mark Oberman, and Michael Teite. 1990. Reality Built For Two: A Virtual Reality Tool. ACM SIGGRAPH Computer Graphics 24 (1990), 35–36.
[4]
John Brooke. 1996. SUS: A “quick and dirty” usability scale., 189 pages.
[5]
Christer Carlsson and Olof Hagsand. 1993. DIVE A multi-user virtual reality system. In Proceedings of IEEE Virtual Reality Annual International Symposium. IEEE, New York, New York, USA, 394–400. https://doi.org/10.1109/VRAIS.1993.380753
[6]
Matthew Ciolek and Adam Kendon. 1980. Environment and the Spatial Arrangement of Conversational Encounters. Sociological Inquiry 50(1980), 237–271. https://doi.org/10.1111/j.1475-682X.1980.tb00022.x
[7]
Christopher Codella, Reza Jalili, Lawrence Koved, J. Bryan Lewis, Daniel T. Ling, James S. Lipscomb, David A. Rabenhorst, Chu P. Wang, Alan Norton, Paula Sweeney, and et al.1992. Interactive Simulation in a Multi-Person Virtual World. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Monterey, California, USA) (CHI ’92). ACM, New York, NY, USA, 329–334. https://doi.org/10.1145/142750.142825
[8]
Carolina Cruz-Neira, Daniel J. Sandin, and Thomas A. DeFanti. 1993. Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (Anaheim, CA) (SIGGRAPH ’93). ACM, New York, NY, USA, 135–142. https://doi.org/10.1145/166117.166134
[9]
Carolina Cruz-Neira, Daniel J. Sandin, Thomas A. DeFanti, Robert V. Kenyon, and John C. Hart. 1992. The CAVE: Audio Visual Experience Automatic Virtual Environment. Commun. ACM 35(1992), 64–72. https://doi.org/10.1145/129888.129892
[10]
Eyal Dim and Tsvi Kuflik. 2013. Social F-formation in blended reality. In Proceedings of the International Conference on Intelligent User Interfaces. ACM Press, Santa Monica, CA, USA., 25–28.
[11]
Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of Human-Computer Studies 131 (2019), 81–98.
[12]
David Geerts. 2006. Comparing voice chat and text chat in a communication tool for interactive television. In Proceedings of the 4th Nordic conference on Human-computer interaction changing roles - NordiCHI ’06. ACM Press, New York, NY, USA, 461–464. https://doi.org/10.1145/1182475.1182537
[13]
Erving Goffman. 1978. The presentation of self in everyday life.
[14]
Chris Greenhalgh and Steven Benford. 1995. MASSIVE: A Co Environment for Iaborative Virtual Teleconferencing. ACM Transactions on Computer-Human Interaction 2 (1995), 239–261.
[15]
Jan Gugenheimer, Evgeny Stemasov, Julian Frommel, and Enrico Rukzio. 2017. ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17. ACM Press, New York, NY, USA, 4021–4033. https://doi.org/10.1145/3025453.3025683
[16]
Edward Twitchell Hall. 1966. The hidden dimension.
[17]
Chad Harms and Frank Biocca. 2004. Internal Consistency and Reliability of the Networked Minds Measure of Social Presence. In Seventh Annual International Workshop: Presence 2004 (Valencia), Mariano Alcaniz and Beatriz Rey (Eds.). Universidad Politecnica de Valencia, Valencia, Spain.
[18]
Kevin Hogan. 2003. Can’t Get Through: Eight Barriers to Communication.
[19]
Wolfgang Hürst and Matthias Helder. 2011. Mobile 3D Graphics and Virtual Reality Interaction. ACM, New York, NY, USA. https://doi.org/10.1145/2071423.2071458
[20]
Hikaru Ibayashi, Yuta Sugiura, Daisuke Sakamoto, Natsuki Miyata, Mitsunori Tada, Takashi Okuma, Takeshi Kurata, Masaaki Mochimaru, and Takeo Igarashi. 2015. Dollhouse VR: A Multi-View, Multi-User Collaborative Design Workspace with VR Technology. In SIGGRAPH Asia 2015 Emerging Technologies (Kobe, Japan) (SA ’15). ACM, New York, NY, USA, Article 8, 2 pages. https://doi.org/10.1145/2818466.2818480
[21]
Jason Jerald. 2015. The VR book: Human-centered design for virtual reality.
[22]
Jean Luc Lugrin, Johanna Latt, and Marc Erich Latoschik. 2015. Avatar anthropomorphism and illusion of body ownership in VR. In 2015 IEEE Virtual Reality Conference, VR 2015 - Proceedings. IEEE, Arles, Camargue, Provence, France, 229–230. https://doi.org/10.1109/VR.2015.7223379
[23]
Andrew Maimone, Xubo Yang, Nate Dierk, Andrei State, Mingsong Dou, and Henry Fuchs. 2013. General-purpose telepresence with head-worn optical see-through displays and projector-based lighting. In 2013 IEEE Virtual Reality (VR). IEEE, Orlando, Florida, USA, 23–26. https://doi.org/10.1109/VR.2013.6549352
[24]
Paul Marshall, Yvonne Rogers, and Nadia Pantidi. 2011. Using F-Formations to Analyse Spatial Patterns of Interaction in Physical Environments. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (Hangzhou, China) (CSCW ’11). ACM, New York, NY, USA, 445–454. https://doi.org/10.1145/1958824.1958893
[25]
Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1995. Augmented reality: a class of displays on the reality-virtuality continuum. In Telemanipulator and Telepresence Technologies, Hari Das (Ed.), Vol. 2351. International Society for Optics and Photonics, SPIE, Boston, MA, United States, 282 – 292. https://doi.org/10.1117/12.197321
[26]
Mahdi Nabiyouni, Ayshwarya Saktheeswaran, Doug A Bowman, and Ambika Karanth. 2015. Comparing the performance of natural, semi-natural, and non-natural locomotion techniques in virtual reality. In 2015 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, IEEE, Arles, France, 3–10.
[27]
Sarah Nichols, Clovissa Haldane, and John R. Wilson. 2000. Measurement of presence and its consequences in virtual environments. International Journal of Human-Computer Studies 52 (2000), 471–491. https://doi.org/10.1006/ijhc.1999.0343
[28]
Mitchell Norman, Gun A Lee, Ross T Smith, and Mark Billingurst. 2019. The impact of remote user’s role in a mixed reality mixed presence system. In The 17th International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, New York, NY, USA, 1–9.
[29]
Sergio Orts-Escolano, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Christoph Rhemann, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, Shahram Izadi, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, and Sameh Khamis. 2016. Holoportation: Virtual 3D Teleportation in Real-time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST ’16. ACM Press, New York, NY, USA, 741–754. https://doi.org/10.1145/2984511.2984517
[30]
Christopher J. Patrick, John J. Curtin, and Auke Tellegen. 2002. Development and validation of a brief form of the Multidimensional Personality Questionnaire.Psychological Assessment 14 (2002), 150–163. https://doi.org/10.1037/1040-3590.14.2.150
[31]
Mark Peter, Robin Horst, and Ralf Dörner. 2018. Vr-guide: A specific user role for asymmetric virtual reality setups in distributed virtual reality applications. In Proceedings of the 10th Workshop Virtual and Augmented Reality of the GI Group VR/AR. Ges. für Informatik, Düsseldorf, Germany.
[32]
Thammathip Piumsomboon, Arindam Day, Barrett Ens, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. Exploring enhancements for remote mixed reality collaboration. In SIGGRAPH Asia 2017 Mobile Graphics & Interactive Applications on - SA ’17. ACM Press, New York, NY, USA, 1–5. https://doi.org/10.1145/3132787.3139200
[33]
Thammathip Piumsomboon, Arindam Dey, Barrett Ens, Gun Lee, and Mark Billinghurst. 2019. The Effects of Sharing Awareness Cues in Collaborative Mixed Reality. Frontiers in Robotics and AI 6 (2019), 1–18. https://doi.org/10.3389/frobt.2019.00005
[34]
Thammathip Piumsomboon, Gun A. Lee, Jonathon D. Hart, Barrett Ens, Robert W. Lindeman, Bruce H. Thomas, and Mark Billinghurst. 2018. Mini-Me: An Adapative Avatar for Mixed Reality Remote Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173620
[35]
Thammathip Piumsomboon, Gun A Lee, Andrew Irlitti, Barrett Ens, Bruce H Thomas, and Mark Billinghurst. 2019. On the shoulder of the giant: A multi-scale mixed reality collaboration with 360 video sharing and tangible interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems. ACM, New York, NY, USA, 1–17.
[36]
Thammathip Piumsomboon, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. CoVAR: a collaborative virtual and augmented reality system for remote collaboration. In SIGGRAPH Asia 2017 Emerging Technologies on - SA ’17, Vol. 106. ACM Press, New York, NY, USA, 1–2. https://doi.org/10.1145/3132818.3132822
[37]
Troels Ammitsbøl Rasmussen and Kaj Gronbak. 2019. Tailorable Remote Assistance with RemoteAssistKit: A Study of and Design Response to Remote Assistance in the Manufacturing Industry. In Collaboration Technologies and Social Computing, Hideyuki Nakanishi, Hironori Egi, Irene-Angelica Chounta, Hideyuki Takada, Satoshi Ichimura, and Ulrich Hoppe (Eds.). Springer International Publishing, Cham, 80–95.
[38]
Thomas W Schubert. 2005. Your highness: vertical positions as perceptual symbols of power.Journal of personality and social psychology 89, 1(2005), 1.
[39]
Martijn J Schuemie, Peter Van Der Straaten, Merel Krijn, and Charles APG Van Der Mast. 2001. Research on presence in virtual reality: A survey. CyberPsychology & Behavior 4 (2001), 183–201.
[40]
Sofia Seinfeld, Tiare Feuchtner, Antonella Maselli, and Jörg Müller. 2020. User Representations in Human-Computer Interaction. Human–Computer Interaction 0, 0 (2020), 1–39.
[41]
Sofia Seinfeld, Tiare Feuchtner, Johannes Pinzek, and Jörg Müller. 2020. Impact of Information Placement and User Representations in VR on Performance and Embodiment. IEEE Transactions on Visualization and Computer Graphics 0, 0(2020), 1–13. https://doi.org/10.1109/TVCG.2020.3021342
[42]
Anton Shufutinsky, James Sibel, Dmitri Shufutinsky, and Ryan Cox. 2018. Holoportation: The Disruptive Technology that Will Change the Practice of Conflict Analysis and Resolution. OD Practitioner 50(2018), 51–61.
[43]
Mel Slater, Amela Sadagic, Martin Usoh, and Ralph Schroeder. 2000. Small-Group Behavior in a Virtual and Real Environment: A Comparative Study. Presence: Teleoperators and Virtual Environments 9 (2000), 37–51. https://doi.org/10.1162/105474600566600
[44]
Mel Slater and Sylvia Wilbur. 1997. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators & Virtual Environments 6, 6(1997), 603–616.
[45]
Anthony Steed, Mel Slater, Amela Sadagic, Adrian Bullock, and Jolanda Tromp. 1999. Leadership and collaboration in shared virtual environments. In Proceedings IEEE Virtual Reality. IEEE, New York, NY, USA, 112–115. https://doi.org/10.1109/VR.1999.756941
[46]
Richard C. Waters, David B. Anderson, John W. Barrus, David C. Brogan, Michael A. Casey, Stephan G. McKeown, Tohei Nitta, Ilene B. Sterns, and William S. Yerazunis. 1997. Diamond Park and Spline:Social Virtual Reality with 3D Animation, Spoken Interaction, and Runtime Extendability. Presence: Teleoperators and Virtual Environments 6 (1997), 461–481. https://doi.org/10.1162/pres.1997.6.4.461
[47]
Richard C Waters and John W Barrus. 1997. The rise of shared virtual environments. IEEE Spectrum 34(1997), 20–25. https://doi.org/10.1109/6.576004
[48]
Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments 7 (1998), 225–240. https://doi.org/10.1162/105474698565686
[49]
Haijun Xia, Sebastian Herscher, Ken Perlin, and Daniel Wigdor. 2018. Spacetime: Enabling Fluid Individual and Collaborative Editing in Virtual Reality. In The 31st Annual ACM Symposium on User Interface Software and Technology - UIST ’18. ACM Press, New York, NY, USA, 853–866. https://doi.org/10.1145/3242587.3242597

Cited By

View all
  • (2025)Technology in Digital Escape RoomsDigital Escape Room Designs in Education10.4018/979-8-3693-4219-0.ch003(107-156)Online publication date: 28-Feb-2025
  • (2024)There Is More to Avatars Than Visuals: Investigating Combinations of Visual and Auditory User Representations for Remote Collaboration in Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/36981488:ISS(540-568)Online publication date: 24-Oct-2024
  • (2024)Evaluating Layout Dimensionalities in PC+VR Asymmetric Collaborative Decision MakingProceedings of the ACM on Human-Computer Interaction10.1145/36981308:ISS(112-132)Online publication date: 24-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
OzCHI '20: Proceedings of the 32nd Australian Conference on Human-Computer Interaction
December 2020
764 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. building task
  2. conversational scenario
  3. f-formation
  4. mobile interaction
  5. multi-user VR
  6. perceived leadership
  7. social VR

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Danish Innovation Fund

Conference

OzCHI '20

Acceptance Rates

Overall Acceptance Rate 362 of 729 submissions, 50%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)158
  • Downloads (Last 6 weeks)5
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Technology in Digital Escape RoomsDigital Escape Room Designs in Education10.4018/979-8-3693-4219-0.ch003(107-156)Online publication date: 28-Feb-2025
  • (2024)There Is More to Avatars Than Visuals: Investigating Combinations of Visual and Auditory User Representations for Remote Collaboration in Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/36981488:ISS(540-568)Online publication date: 24-Oct-2024
  • (2024)Evaluating Layout Dimensionalities in PC+VR Asymmetric Collaborative Decision MakingProceedings of the ACM on Human-Computer Interaction10.1145/36981308:ISS(112-132)Online publication date: 24-Oct-2024
  • (2024)CrossGAIProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435428:1(1-27)Online publication date: 6-Mar-2024
  • (2024)Mixed-reality art as shared experience for cross-device users: Materialize, understand, and exploreInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103291190(103291)Online publication date: Oct-2024
  • (2024)Exploring Virtual Collaboration Platforms for Product Design Courses: A Preliminary Evaluation StudyInnovative Technologies and Learning10.1007/978-3-031-65884-6_13(124-133)Online publication date: 14-Aug-2024
  • (2023)UniteXR: Joint Exploration of a Real-World Museum and its Digital TwinProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615708(1-10)Online publication date: 9-Oct-2023
  • (2023)Being Social in VR Meetings: A Landscape Analysis of Current ToolsProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595959(1789-1809)Online publication date: 10-Jul-2023
  • (2023)Facilitating Asymmetric Interaction between VR Users and External Users via Wearable Gesture-Based Interface2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW58643.2023.00346(1001-1002)Online publication date: Mar-2023
  • (2023)What And How Together: A Taxonomy On 30 Years Of Collaborative Human-Centered XR Tasks2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00047(322-335)Online publication date: 16-Oct-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media