research-article

Public Access

Measuring and Comparing Collaborative Visualization Behaviors in Desktop and Augmented Reality Environments

Authors:

Michael Kintscher,

Anjana Arunkumar,

Chris BryanAuthors Info & Claims

VRST '23: Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology

Article No.: 22, Pages 1 - 11

https://doi.org/10.1145/3611659.3615691

Published: 09 October 2023 Publication History

All formats PDF

Abstract

Augmented reality (AR) provides a significant opportunity to improve collaboration between co-located team members jointly analyzing data visualizations, but existing rigorous studies are lacking. We present a novel method for qualitatively encoding the positions of co-located users collaborating with head-mounted displays (HMDs) to assist in reliably analyzing collaboration styles and behaviors. We then perform a user study on the collaborative behaviors of multiple, co-located synchronously collaborating users in AR to demonstrate this method in practice and contribute to the shortfall of such studies in the existing literature. Pairs of users performed analysis tasks on several data visualizations using both AR and traditional desktop displays. To provide a robust evaluation, we collected several types of data, including software logging of participant positioning, qualitative analysis of video recordings of participant sessions, and pre- and post-study questionnaires including the NASA TLX survey. Our results suggest that the independent viewports of AR headsets reduce the need to verbally communicate about navigating around the visualization and encourage face-to-face and non-verbal communication. Our novel positional encoding method also revealed the overlap of task and communication spaces vary based on the needs of the collaborators.

References

[1]

Cédric Bach and Dominique L Scapin. 2004. Obstacles and perspectives for evaluating mixed reality systems usability. In Acte du Workshop MIXER, IUI-CADUI, Vol. 4. Citeseer.

[2]

Kevin M Baird and Woodrow Barfield. 1999. Evaluating the effectiveness of augmented reality displays for a manual assembly task. Virtual Reality 4, 4 (1999), 250–259.

Digital Library

[3]

Daniel Belcher, Mark Billinghurst, SE Hayes, and Randy Stiles. 2003. Using augmented reality for visualizing complex graphs in three dimensions. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings. IEEE, 84–93.

[4]

Mark Billinghurst and Hirokazu Kato. 1999. Collaborative mixed reality. In Proceedings of the First International Symposium on Mixed Reality. 261–284.

[5]

Mark Billinghurst, Hirokazu Kato, Kiyoshi Kiyokawa, Daniel Belcher, and Ivan Poupyrev. 2002. Experiments with face-to-face collaborative AR interfaces. Virtual Reality 6, 3 (2002), 107–121.

[6]

Matthew Brehmer and Tamara Munzner. 2013. A multi-level typology of abstract visualization tasks. IEEE transactions on visualization and computer graphics 19, 12 (2013), 2376–2385.

Digital Library

[7]

Wolfgang Broll, Irma Lindt, Jan Ohlenburg, Michael Wittkämper, Chunrong Yuan, Thomas Novotny, Chiron Mottram, Andreas Strothmann, 2004. Arthur: A collaborative augmented environment for architectural design and urban planning. JVRB-Journal of Virtual Reality and Broadcasting 1, 1 (2004).

[8]

Wolfgang Büschel, Patrick Reipschläger, Ricardo Langner, and Raimund Dachselt. 2017. Investigating the use of spatial interaction for 3d data visualization on mobile devices. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces. 62–71.

Digital Library

[9]

Jeffrey W Chastine, Kristine Nagel, Ying Zhu, and Luca Yearsovich. 2007. Understanding the design space of referencing in collaborative augmented reality environments. In Proceedings of graphics interface 2007. 207–214.

Digital Library

[10]

Arindam Dey, Mark Billinghurst, Robert W Lindeman, and J Swan. 2018. A systematic review of 10 years of augmented reality usability studies: 2005 to 2014. Frontiers in Robotics and AI 5 (2018), 37.

[11]

Suyang Dong, Amir H Behzadan, Feng Chen, and Vineet R Kamat. 2013. Collaborative visualization of engineering processes using tabletop augmented reality. Advances in Engineering Software 55 (2013), 45–55.

Digital Library

[12]

Andreas Dünser, Raphaël Grasset, and Mark Billinghurst. 2008. A survey of evaluation techniques used in augmented reality studies. Human Interface Technology Laboratory New Zealand.

[13]

Ardita Dylgjeri and Ledia Kazazi. 2013. Deixis in modern linguistics and outside. Academic Journal of Interdisciplinary Studies 2, 4 (2013), 87–87.

[14]

Rob Edsall and Kelli L Larson. 2006. Decision making in a virtual environment: Effectiveness of a semi-immersive “decision theater” in understanding and assessing human-environment interactions. In Proceedings of AutoCarto, Vol. 6. 25–28.

[15]

Clarence A Ellis, Simon J Gibbs, and Gail Rein. 1991. Groupware: some issues and experiences. Commun. ACM 34, 1 (1991), 39–58.

Digital Library

[16]

Niklas Elmqvist, Pierre Dragicevic, and Jean-Daniel Fekete. 2008. Rolling the dice: Multidimensional visual exploration using scatterplot matrix navigation. IEEE transactions on Visualization and Computer Graphics 14, 6 (2008), 1539–1148.

[17]

Charles J Fillmore. 1966. Deictic categories in the semantics of’come’. Foundations of language (1966), 219–227.

[18]

Susan R. Fussell, Leslie D. Setlock, and Elizabeth M. Parker. 2003. Where Do Helpers Look? Gaze Targets during Collaborative Physical Tasks. In CHI ’03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI EA ’03). Association for Computing Machinery, New York, NY, USA, 768–769. https://doi.org/10.1145/765891.765980

Digital Library

[19]

Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004. Action as language in a shared visual space. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. 487–496.

Digital Library

[20]

Edwin L Hutchins, James D Hollan, and Donald A Norman. 1985. Direct manipulation interfaces. Human–computer interaction 1, 4 (1985), 311–338.

Digital Library

[21]

Kori Marie Inkpen. 1997. Adapting the human-computer interface to support collaborative learning environments for children. Ph. D. Dissertation. University of British Columbia.

[22]

Petra Isenberg, Danyel Fisher, Sharoda A Paul, Meredith Ringel Morris, Kori Inkpen, and Mary Czerwinski. 2011. Co-located collaborative visual analytics around a tabletop display. IEEE Transactions on visualization and Computer Graphics 18, 5 (2011), 689–702.

Digital Library

[23]

Hiroshi Ishii, Minoru Kobayashi, and Kazuho Arita. 1994. Iterative design of seamless collaboration media. Commun. ACM 37, 8 (1994), 83–97.

Digital Library

[24]

Robert Johansen, David Sibbet, Suzyan Benson, Alixia Martin, Robert Mittman, and Paul Saffo. 1991. Leading business teams: How teams can use technology and group process tools to enhance performance. Addison-Wesley Longman Publishing Co., Inc.

Digital Library

[25]

Beatrice John, D Lang, Henrik von Wehrden, Ruediger John, and Arnim Wiek. 2018. Advancing decision-visualization environments: Empirically informed design guidelines.

[26]

Kiyoshi Kiyokawa, Mark Billinghurst, Sohan E Hayes, Anoop Gupta, Yuki Sannohe, and Hirokazu Kato. 2002. Communication behaviors of co-located users in collaborative AR interfaces. In Proceedings. International Symposium on Mixed and Augmented Reality. IEEE, 139–148.

[27]

Robert E Kraut, Susan R Fussell, and Jane Siegel. 2003. Visual information as a conversational resource in collaborative physical tasks. Human–computer interaction 18, 1-2 (2003), 13–49.

[28]

Robert E Kraut, Darren Gergle, and Susan R Fussell. 2002. The use of visual information in shared visual spaces: Informing the development of virtual co-presence. In Proceedings of the 2002 ACM conference on Computer supported cooperative work. 31–40.

Digital Library

[29]

Robert E Kraut, Mark D Miller, and Jane Siegel. 1996. Collaboration in performance of physical tasks: Effects on outcomes and communication. In Proceedings of the 1996 ACM conference on Computer supported cooperative work. 57–66.

Digital Library

[30]

Heidi Lam, Enrico Bertini, Petra Isenberg, Catherine Plaisant, and Sheelagh Carpendale. 2011. Empirical studies in information visualization: Seven scenarios. IEEE transactions on visualization and computer graphics 18, 9 (2011), 1520–1536.

[31]

Matthew Lombard and Theresa Ditton. 1997. At the heart of it all: The concept of presence. Journal of computer-mediated communication 3, 2 (1997), JCMC 321.

[32]

Tim Marsh. 2003. Staying there: an activity-based approach to narrative design and evaluation as an antidote to virtual corpsing. In Being there: Concepts, effects and measurement of user presence in synthetic environments, studies on new technologies and practices in communication series. Citeseer.

[33]

Andreas Mathisen, Tom Horak, Clemens Nylandsted Klokmose, Kaj Grønbæk, and Niklas Elmqvist. 2019. InsideInsights: Integrating Data-Driven Reporting in Collaborative Visual Analytics. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 649–661.

[34]

Mark McGill, Daniel Boland, Roderick Murray-Smith, and Stephen Brewster. 2015. A dose of reality: Overcoming usability challenges in vr head-mounted displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2143–2152.

[35]

Microsoft. 2020. HoloLens 2-Overview, Features, and Specs: Microsoft HoloLens. https://www.microsoft.com/en-us/hololens/hardware. https://www.microsoft.com/en-us/hololens/hardware.

[36]

Jens Müller, Johannes Zagermann, Jonathan Wieland, Ulrike Pfeil, and Harald Reiterer. 2019. A Qualitative Comparison Between Augmented and Virtual Reality Collaboration with Handheld Devices. In Proceedings of Mensch und Computer 2019. 399–410.

Digital Library

[37]

Tamara Munzner. 2014. Visualization analysis and design. CRC press.

[38]

Andrew YC Nee, SK Ong, George Chryssolouris, and Dimitris Mourtzis. 2012. Augmented reality applications in design and manufacturing. CIRP annals 61, 2 (2012), 657–679.

[39]

Jiazhi Ou, Lui Min Oh, Jie Yang, and Susan R. Fussell. 2005. Effects of Task Properties, Partner Actions, and Message Content on Eye Gaze Patterns in a Collaborative Task. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA) (CHI ’05). Association for Computing Machinery, New York, NY, USA, 231–240. https://doi.org/10.1145/1054972.1055005

Digital Library

[40]

Kyoung S Park, Abhinav Kapoor, and Jason Leigh. 2000. Lessons learned from employing multiple perspectives in a collaborative virtual environment for visualizing scientific data. In Proceedings of the Third International Conference on Collaborative Virtual Environments. 73–82.

Digital Library

[41]

Dheva Raja, Doug Bowman, John Lucas, and Chris North. 2004. Exploring the benefits of immersion in abstract information visualization. In Proc. Immersive Projection Technology Workshop. 61–69.

[42]

Dieter Schmalstieg, Anton Fuhrmann, Gerd Hesina, Zsolt Szalavári, L Miguel Encarnaçao, Michael Gervautz, and Werner Purgathofer. 2002. The studierstube augmented reality project. Presence: Teleoperators & Virtual Environments 11, 1 (2002), 33–54.

Digital Library

[43]

Mickael Sereno, Xiyao Wang, Lonni Besancon, Michael J Mcguffin, and Tobias Isenberg. 2020. Collaborative Work in Augmented Reality: A Survey. IEEE Transactions on Visualization and Computer Graphics (2020), 1–1. https://doi.org/10.1109/TVCG.2020.3032761

Digital Library

[44]

Ben Shneiderman. 2003. The eyes have it: A task by data type taxonomy for information visualizations. In The craft of information visualization. Elsevier, 364–371.

[45]

Rebecca R Springmeyer, Meera M Blattner, and Nelson L Max. 1992. A characterization of the scientific data analysis process. In Proceedings of the 3rd conference on Visualization’92. IEEE Computer Society Press, 235–242.

[46]

Zsolt Szalavári, Dieter Schmalstieg, Anton Fuhrmann, and Michael Gervautz. 1998. “Studierstube”: An environment for collaboration in augmented reality. Virtual Reality 3, 1 (1998), 37–48.

Digital Library

[47]

Arthur Tang, Frank Biocca, and Lynette Lim. 2004. Comparing differences in presence during social interaction in augmented reality versus virtual reality environments: An exploratory study. Proceedings of PRESENCE (2004), 204–208.

[48]

Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI conference on Human factors in computing systems. 73–80.

Digital Library

[49]

Anthony Tang, Melanie Tory, Barry Po, Petra Neumann, and Sheelagh Carpendale. 2006. Collaborative coupling over tabletop displays. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 1181–1190.

Digital Library

[50]

Melanie Tory, Arthur E Kirkpatrick, M Stella Atkins, and Torsten Moller. 2005. Visualization task performance with 2D, 3D, and combination displays. IEEE transactions on visualization and computer graphics 12, 1 (2005), 2–13.

[51]

April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How data scientists use computational notebooks for real-time collaboration. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–30.

Digital Library

[52]

Xiangyu Wang and Phillip S Dunston. 2011. Comparative effectiveness of mixed reality-based virtual environments in collaborative design. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 41, 3 (2011), 284–296.

Digital Library

[53]

Duncan J Watts and Steven H Strogatz. 1998. Collective dynamics of ‘small-world’ networks. nature 393, 6684 (1998), 440–442.

[54]

Matt Whitlock, Stephen Smart, and Danielle Albers Szafir. 2020. Graphical Perception for Immersive Analytics. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 616–625.

[55]

Jian Zhao, Michael Glueck, Petra Isenberg, Fanny Chevalier, and Azam Khan. 2017. Supporting handoff in asynchronous collaborative sensemaking using knowledge-transfer graphs. IEEE transactions on visualization and computer graphics 24, 1 (2017), 340–350.

Cited By

Rebol MJefferson CBelay SHood CSikka N(2024)Augmented Reality Teaching Methods for Smartphone Coaching2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00115(409-410)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00115

Index Terms

Measuring and Comparing Collaborative Visualization Behaviors in Desktop and Augmented Reality Environments
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction paradigms
      1. Collaborative interaction
      2. Mixed / augmented reality

Recommendations

Collaborative Augmented Reality: Multi-user Interaction in Urban Simulation
IVIC '09: Proceedings of the 1st International Visual Informatics Conference on Visual Informatics: Bridging Research and Practice

Augmented reality (AR) environment allows user or multi-user to interact with 2D and 3D data. AR simply can provide a collaborative interactive AR environment for urban simulation, where users can interact naturally and intuitively. AR collaboration ...
Collaborative Augmented Reality Approach for Multi-user Interaction in Urban Simulation
ICIMT '09: Proceedings of the 2009 International Conference on Information and Multimedia Technology

Augmented reality (AR) environment allows user or multi-user to interact with 2D and 3D data. AR simply can provide a collaborative interactive AR environment for urban simulation, where users can interact naturally and intuitively. AR collaboration ...
A Qualitative Comparison Between Augmented and Virtual Reality Collaboration with Handheld Devices
MuC '19: Proceedings of Mensch und Computer 2019

Handheld Augmented Reality (AR) displays offer a see-through option to create the illusion of virtual objects being integrated into the viewer's physical environment. Some AR display technologies also allow for the deactivation of the see-through option,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '23: Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology

October 2023

542 pages

ISBN:9798400703287

DOI:10.1145/3611659

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

U.S. National Science Foundation

Conference

VRST 2023

Sponsor:

VRST 2023: 29th ACM Symposium on Virtual Reality Software and Technology

October 9 - 11, 2023

Christchurch, New Zealand

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
309
Total Downloads

Downloads (Last 12 months)158
Downloads (Last 6 weeks)33

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rebol MJefferson CBelay SHood CSikka N(2024)Augmented Reality Teaching Methods for Smartphone Coaching2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00115(409-410)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00115

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten