skip to main content
10.1145/3242671.3242675acmconferencesArticle/Chapter ViewAbstractPublication Pageschi-playConference Proceedingsconference-collections
research-article

Up to the Finger Tip: The Effect of Avatars on Mid-Air Pointing Accuracy in Virtual Reality

Published: 23 October 2018 Publication History

Abstract

Avatars in virtual reality (VR) increase the immersion and provide an interface between the user's physical body and the virtual world. Thus, avatars enable referential gestures, which are essential for targeting, selection, locomotion, and collaboration in VR. However, players of immersive games can have another virtual appearance deviating from human-likeness and previous work suggests that avatars can have an effect on the accuracy of referential gestures in VR. One of the most important referential gestures is mid-air pointing. It has been shown that mid-air pointing is affected by systematic errors, which can be compensated using different methods. Thus, it is unknown if the avatar must be considered in corrections of the systematic error. In this paper, we investigate the effect of the avatar on pointing accuracy. We show that the systematic error in pointing is significantly affected by the virtual appearance but does not correlate with the degree to which the appearance deviates from the perceived human-likeness. Moreover, we confirm that people only rely on their fingertip and not on their forearm or index finger orientation. We present compensation models and contribute with design implications to increase the accuracy of pointing in VR.

References

[1]
Deepak Akkil and Poika Isokoski. 2016. Accuracy of Interpreting Pointing Gestures in Egocentric View. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16). ACM, New York, NY, USA, 262--273.
[2]
Ferran Argelaguet, Carlos Andujar, and Ramon Trueba. 2008. Overcoming Eye-hand Visibility Mismatch in 3D Pointing Selection. In Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology (VRST '08). ACM, New York, NY, USA, 43--46.
[3]
Ferran Argelaguet, Ludovic Hoyet, Michaël Trico, and Anatole Lécuyer. 2016. The role of interaction in virtual embodiment: Effects of the virtual hand representation. In IEEE Virtual Reality (VR '16). Greenville, SC, USA, 3--10.
[4]
Tiziana Aureli, Maria Spinelli, Mirco Fasolo, Maria Concetta Garito, Paola Perucchini, and Laura D'Odorico. 2017. The Pointing-Vocal Coupling Progression in the First Half of the Second Year of Life. Infancy (2017), 18.
[5]
Bethesda Game Studios. 2017. Fallout 4 VR. Game {PC}. (December 2017). Bethesda Softworks, Rockville, ML, USA. Last played July 2018.
[6]
Bethesda Game Studios. 2018. Skyrim VR. Game {PC}. (3 April 2018). Bethesda Softworks, Rockville, ML, USA. Last played July 2018.
[7]
Richard A. Bolt. 1980. Put-that-there: Voice and gesture at the graphics interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques (SIGGRAPH '80). ACM, New York, NY, USA, 262--270.
[8]
Sebastian Boring, Marko Jurmu, and Andreas Butz. 2009. Scroll, Tilt or Move It: Using Mobile Phones to Continuously Control Pointers on Large Public Displays. In Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7 (OZCHI '09). ACM, New York, NY, USA, 161--168.
[9]
Luigia Camaioni, Paola Perucchini, Francesca Bellagamba, and Cristina Colonnesi. 2004. The Role of Declarative Pointing in Developing a Theory of Mind. Infancy 5, 3 (2004), 291--308.
[10]
Andrea Corradini and Philip R. Cohen. 2002. Multimodal speech-gesture interface for handfree painting on a virtual paper using partial recurrent neural networks as gesture recognizer. (2002).
[11]
Giambattista Della Porta. 1593. De refractione Optices Parte: Libri Novem... Ex officina Horatii Salviani, apud Jo. Jacobum Carlinum, & Antonium Pacem.
[12]
H. Henrik Ehrsson, Charles Spence, and Richard E. Passingham. 2004. Thattextquoterights My Hand! Activity in Premotor Cortex Reflects Feeling of Ownership of a Limb. Science 305, 5685 (2004), 875--877.
[13]
J. M. Foley and Richard Held. 1972. Visually directed pointing as a function of target distance, direction, and available cues. Perception & Psychophysics 12, 3 (01 May 1972), 263--268.
[14]
Michael Steven Anthony Graziano. 1999. Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. Proceedings of the National Academy of Sciences 96, 18 (1999), 10418--10421.
[15]
Michael S. A. Graziano, Dylan F. Cooke, and Charlotte S. R. Taylor. 2000. Coding the Location of the Arm by Sight. Science 290 (2000), 1782--1786. http://www.jstor.org/stable/3081775
[16]
Sandra G. Hart. 2006. NASA-Task Load Index (NASA-TLX); 20 years later. In Proceedings of the Human Factors and Ergonomic Society annual meeting, Vol. 50. SAGE Publications, SAGE Publications, Los Angeles, CA, USA, 904--908.
[17]
John B. Haviland. 2003. How to point in Zinacantán. Psychology Press, Mahwah, New Jersey, USA, 139--169.
[18]
Chin Chang Ho and Karl F. MacDorman. 2017. Measuring the Uncanny Valley Effect: Refinements to Indices for Perceived Humanness, Attractiveness, and Eeriness. International Journal of Social Robotics 9, 1 (jan 2017), 129--139.
[19]
Hartwig Holzapfel, Kai Nickel, and Rainer Stiefelhagen. 2004. Implementation and Evaluation of a Constraint-based Multimodal Fusion System for Speech and 3D Pointing Gestures. In Proceedings of the 6th International Conference on Multimodal Interfaces (ICMI '04). ACM, New York, NY, USA, 175--182.
[20]
Ricardo Jota, Miguel A. Nacenta, Joaquim A. Jorge, Sheelagh Carpendale, and Saul Greenberg. 2010. A Comparison of Ray Pointing Techniques for Very Large Displays. In Proceedings of Graphics Interface 2010 (GI '10). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 269--276. http://dl.acm.org/citation.cfm?id=1839214.1839261
[21]
Adam Kendon. 2004. Gesture: Visible Action as Utterance. Cambridge University Press, Cambridge. IX, 400 S. pages. https://books.google.de/books?id=WY2bBQAAQBAJ
[22]
Konstantina Kilteni, Ilias Bergstrom, and Mel Slater. 2013. Drumming in Immersive Virtual Reality: The Body Shapes the Way We Play . 19, 4 (2013), 597--605.
[23]
Pascal Knierim, Valentin Schwind, Anna Maria Feit, Florian Nieuwenhuizen, and Niels Henze. 2018. Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands, In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018-04-21). Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018).
[24]
Alfred Kranstedt, Andy Lücking, Thies Pfeiffer, Hannes Rieser, and Marc Staudacher. 2006. Measuring and Reconstructing Pointing in Visual Contexts. In Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (brandial'06). Universitätsverlag Potsdam, Potsdam, 82--89. http://opus.kobv.de/ubp/volltexte/2006/1048/pdf/brandial06_proceedings.pdf
[25]
Lorraine Lin and Sophie Jö rg. 2016. Need a hand? How Appearance Affects the Virtual Hand Illusion. In Proceedings of the ACM Symposium on Applied Perception (SAP '16). ACM Press, New York, New York, USA, 69--76.
[26]
Jean-Luc Lugrin, Johanna Latt, and Marc Erich Latoschik. 2015. Avatar anthropomorphism and illusion of body ownership in VR. In 2015 IEEE Virtual Reality (VR), Masataka Imura, Pablo Figueroa, and Betty Mohler (Eds.). IEEE, 229--230.
[27]
Sven Mayer, Valentin Schwind, Robin Schweigert, and Niels Henze. 2018. The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments, In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018-04-21). Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018).
[28]
Sven Mayer, Katrin Wolf, Stefan Schneegass, and Niels Henze. 2015. Modeling Distant Pointing for Compensating Systematic Displacements. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 4165--4168.
[29]
Walter R. Miles. 1930. Ocular Dominance in Human Adults. The journal of general psychology 3, 3 (1930), 412--430.
[30]
Mark R. Mine. 1995. Virtual Environment Interaction Techniques. Technical Report. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. http://www.cs.unc.edu/techreports/95-018.pdf
[31]
Mark R. Mine, Frederick P. Brooks Jr., and Carlo H. Sequin. 1997. Moving Objects in Space: Exploiting Proprioception in Virtual-environment Interaction. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '97). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 19--26.
[32]
Masahiro Mori, Karl F. MacDorman, and Norri Kageki. 1970/2012. The Uncanny Valley {From the Field}. Robotics & Automation Magazine, IEEE 19, 2 (1970/2012), 98--100.
[33]
Steven Morrison and J. Keogh. 2001. Changes in the dynamics of tremor during goal-directed pointing. Human Movement Science 20, 4 (2001), 675--693.
[34]
Brad A. Myers, Rishi Bhatnagar, Jeffrey Nichols, Choon Hong Peck, Dave Kong, Robert Miller, and A. Chris Long. 2002. Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, New York, NY, USA, 33--40.
[35]
Kai Nickel and Rainer Stiefelhagen. 2003. Pointing Gesture Recognition Based on 3D-tracking of Face, Hands and Head Orientation. In Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI '03). ACM, New York, NY, USA, 140--146.
[36]
Dan R. Olsen, Jr. and Travis Nielsen. 2001. Laser Pointer Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 17--22.
[37]
T. Pfeiffer, M. E. Latoschik, and I. Wachsmuth. 2008. Conversational Pointing Gestures for Virtual Reality Interaction: Implications from an Empirical Study. In Proceedings of the 2008 IEEE Virtual Reality Conference (IEEE/VR'08). 281--282.
[38]
Katrin Plaumann, Matthias Weing, Christian Winkler, Michael Müller, and Enrico Rukzio. 2017. Towards accurate cursorless pointing: the effects of ocular dominance and handedness. Personal and Ubiquitous Computing (07 Dec 2017).
[39]
Valentin Schwind, Pascal Knierim, Lews Chuang, and Niels Henze. 2017a. "Where's Pinky?": The Effects of a Reduced Number of Fingers in Virtual Reality. In Proceedings of the 2017 CHI Conference on Computer-Human Interaction in Play (CHI PLAY'17). ACM, New York, NY, USA, 6.
[40]
Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017b. "These Are Not My Hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 1577--1582.
[41]
Valentin Schwind, Katharina Leicht, Solveigh Jäger, Katrin Wolf, and Niels Henze. 2017c. Is there an Uncanny Valley of Virtual Animals? A Quantitative and Qualitative Investigation. International Journal of Human-Computer Studies 111C (2017), 49--61.
[42]
Valentin Schwind, Lorraine Lin, Massimiliano Di Luca, Sophie Jörg, and James Hillis. 2018. Touch with Foreign Hands: The Effect of Virtual Hand Appearance on Visual-Haptic Integration. In Proceedings of the ACM Symposium on Applied Perception (SAP '18) (2018-08-03). ACM, New York, NY.
[43]
Mel Slater, Daniel Perez-marcos, and H. Henrik Ehrsson. 2009. Inducing illusory ownership of a virtual body. Frontiers in neuroscience 3, 2 (Sep 2009), 214--220.
[44]
Mel Slater, Martin Usoh, and Anthony Steed. 1995. Taking Steps: The Influence of a Walking Technique on Presence in Virtual Reality. ACM Transactions on Computer-Human Interaction 2, 3 (Sept. 1995), 201--219.
[45]
Vertigo Games and Jaywalkers Interactive. 2016. Arizona Sunshine. Game {PC,PS4}. (6 Decembter 2016). Vertigo Games, Odessa, TX, USA. Last played July 2018.
[46]
Vinoba Vinayagamoorthy, Andrea Brogni, Marco Gillies, Mel Slater, and Anthony Steed. 2004. An investigation of presence response across variations in visual realism. In The 7th Annual International Presence Workshop. 148--155.
[47]
Daniel Vogel and Ravin Balakrishnan. 2005. Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST '05). ACM, New York, NY, USA, 33--42.
[48]
Juan Pablo Wachs, Mathias Kölsch, Helman Stern, and Yael Edan. 2011. Vision-based Hand-gesture Applications. Commun. ACM 54, 2 (Feb. 2011), 60--71.

Cited By

View all
  • (2025)Integrating Multimodal Communication and Comprehension Evaluation during Human-Robot Collaboration for Increased Reliability of Foundation Model-based Task Planning Systems2025 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII59315.2025.10871045(1053-1059)Online publication date: 21-Jan-2025
  • (2025)A Literature Review and Taxonomy of In-VR Questionnaire User InterfacesImmersive Learning Research Network10.1007/978-3-031-80475-5_7(95-111)Online publication date: 20-Feb-2025
  • (2024)Object center of mass predicts pointing endpoints in virtual realityFrontiers in Virtual Reality10.3389/frvir.2024.14020845Online publication date: 13-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI PLAY '18: Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play
October 2018
563 pages
ISBN:9781450356244
DOI:10.1145/3242671
  • General Chairs:
  • Florian 'Floyd' Mueller,
  • Daniel Johnson,
  • Ben Schouten,
  • Program Chairs:
  • Phoebe O. Toups Dugas,
  • Peta Wyeth
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. accuracy
  2. avatars
  3. pointing
  4. ray-casting
  5. virtual reality

Qualifiers

  • Research-article

Funding Sources

Conference

CHI PLAY '18
Sponsor:

Acceptance Rates

CHI PLAY '18 Paper Acceptance Rate 43 of 123 submissions, 35%;
Overall Acceptance Rate 421 of 1,386 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)89
  • Downloads (Last 6 weeks)16
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Integrating Multimodal Communication and Comprehension Evaluation during Human-Robot Collaboration for Increased Reliability of Foundation Model-based Task Planning Systems2025 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII59315.2025.10871045(1053-1059)Online publication date: 21-Jan-2025
  • (2025)A Literature Review and Taxonomy of In-VR Questionnaire User InterfacesImmersive Learning Research Network10.1007/978-3-031-80475-5_7(95-111)Online publication date: 20-Feb-2025
  • (2024)Object center of mass predicts pointing endpoints in virtual realityFrontiers in Virtual Reality10.3389/frvir.2024.14020845Online publication date: 13-Aug-2024
  • (2024)Target Selection with Avatars in Mixed Reality2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00408(1086-1087)Online publication date: 16-Mar-2024
  • (2024)How Do Users Regulate Interaction Behaviors While Performing a Drag-and-Drop Task in a Virtual Reality Environment?International Journal of Human–Computer Interaction10.1080/10447318.2024.2342085(1-10)Online publication date: 22-Apr-2024
  • (2023)WorldPoint: Finger Pointing as a Rapid and Natural Trigger for In-the-Wild Mobile InteractionsProceedings of the ACM on Human-Computer Interaction10.1145/36264787:ISS(357-375)Online publication date: 1-Nov-2023
  • (2023)UniteXR: Joint Exploration of a Real-World Museum and its Digital TwinProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615708(1-10)Online publication date: 9-Oct-2023
  • (2023)The Effects of Hand Representation on Experience and Performance for 3D Interactions in Virtual Reality GamesProceedings of the ACM on Human-Computer Interaction10.1145/36110667:CHI PLAY(1206-1233)Online publication date: 4-Oct-2023
  • (2023)Bubbleu: Exploring Augmented Reality Game Design with Uncertain AI-based InteractionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581270(1-18)Online publication date: 19-Apr-2023
  • (2023)When XR and AI Meet - A Scoping Review on Extended Reality and Artificial IntelligenceProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581072(1-45)Online publication date: 19-Apr-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media