research-article

Manipulating Virtual Objects in Augmented Reality Using a New Ball-Shaped Input Device

Authors:

Bastian Schmeier,

Jan Patrick Kopetz,

Nicole JochemsAuthors Info & Claims

AH2021: 12th Augmented Human International Conference

Article No.: 7, Pages 1 - 8

https://doi.org/10.1145/3460881.3460935

Published: 28 May 2021 Publication History

Abstract

Today’s Augmented Reality (AR) technology allows users to explore the real world enriched with digital artifacts, learn from it, or shape it (i.e., creating your own virtual objects). To properly use virtual objects in AR space, users must be able to manipulate them (i.e., rotate or move them). The prerequisite for manipulation is an intuitive interaction technique controlled by an input device. To explore novel AR interaction techniques, a new ball-shaped input device called BIRDY is combined with the HoloLens for the first time.

This paper presents findings regarding this combination of devices. Four new interaction techniques were designed that benefit from BIRDY’s orientation invariance. Aiming to identify promising interaction rules, a prototype was developed to evaluate these interaction techniques. Results indicate that using gravity as a placement tool and separating the degrees of freedom when manipulating virtual objects provides the best experience for users. Findings further confirm the potential of using ball-shaped devices for interaction in AR.

References

[1]

Patrick Baudisch, Mike Sinclair, and Andrew Wilson. 2006. Soap: a pointing device that works in mid-air. ACM Press, 43. https://doi.org/10.1145/1166253.1166261

Digital Library

[2]

Benoît Bossavit, Asier Marzo, Oscar Ardaiz, Luis Diaz De Cerio, and Alfredo Pina. 2014. Design Choices and Their Implications for 3D Mid-Air Manipulation Techniques. Presence: Teleoperators and Virtual Environments 23, 4 (Nov. 2014), 377–392. https://doi.org/10.1162/PRES_a_00207

Digital Library

[3]

John Brooke. 1996. SUS-A Quick and Dirty Usability Scale. In Usability Evaluation in Industry, Patrick W. Jordan (Ed.). Taylor & Francis, London, 4–7.

[4]

Fabio Marco Caputo, Marco Emporio, and Andrea Giachetti. 2017. Single-Handed vs. Two Handed Manipulation in Virtual Reality: A Novel Metaphor and Experimental Comparisons. Smart Tools and Apps for Graphics - Eurographics Italian Chapter Conference (2017), 7 pages. https://doi.org/10.2312/stag.20171225

Digital Library

[5]

Michael Chen, S. Joy Mountford, and Abigail Sellen. 1988. A Study in Interactive 3-D Rotation Using 2-D Control Devices. ACM SIGGRAPH Computer Graphics 22, 4 (Aug. 1988), 121–129. https://doi.org/10.1145/378456.378497

Digital Library

[6]

Zeyuan Chen, Christopher Healey, and Robert St. Amant. 2017. Performance Characteristics of a Camera-Based Tangible Input Device for Manipulation of 3D Information. Proceedings of Graphics Interface 2017 Edmonton (2017), 8 pages, 1.94 MB. https://doi.org/10.20380/gi2017.10

Digital Library

[7]

DIN EN ISO 9241-11. 2011. Ergonomie Der Mensch-System-Interaktion - Teil 11: Gebrauchstauglichkeit: Begriffe Und Konzepte.

[8]

H. Durrant-Whyte and T. Bailey. 2006. Simultaneous Localization and Mapping: Part I. IEEE Robotics & Automation Magazine 13, 2 (June 2006), 99–110. https://doi.org/10.1109/MRA.2006.1638022

[9]

Thomas Franke, Christiane Attig, and Daniel Wessel. 2019. A Personal Resource for Technology Interaction: Development and Validation of the Affinity for Technology Interaction (ATI) Scale. International Journal of Human–Computer Interaction 35, 6 (April 2019), 456–467. https://doi.org/10.1080/10447318.2018.1456150

[10]

S. Frees and G.D. Kessler. 2005. Precise and Rapid Interaction through Scaled Manipulation in Immersive Virtual Environments. In IEEE Virtual Reality Conference 2005 (VR’05). IEEE, Bonn, Germany, 99–106. https://doi.org/10.1109/VR.2005.60

Digital Library

[11]

Ken Hinckley, Randy Pausch, John C. Goble, and Neal F. Kassell. 1994. A Survey of Design Issues in Spatial Input. In Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology - UIST ’94. ACM Press, Marina del Rey, California, United States, 213–222. https://doi.org/10.1145/192426.192501

Digital Library

[12]

Ken Hinckley, Joe Tullio, Randy Pausch, Dennis Proffitt, and Neal Kassell. 1997. Usability Analysis of 3D Rotation Techniques. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology - UIST ’97. ACM Press, Banff, Alberta, Canada, 1–10. https://doi.org/10.1145/263407.263408

Digital Library

[13]

Karl M. Kapp. 2012. The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. Pfeiffer, San Francisco, CA.

Digital Library

[14]

Manon Kok, Jeroen D. Hol, and Thomas B. Schön. 2017. Using Inertial Sensors for Position and Orientation Estimation. Foundations and Trends® in Signal Processing 11, 1-2(2017), 1–153. https://doi.org/10.1561/2000000094

Digital Library

[15]

Jan Patrick Kopetz, Svenja Burgsmüller, Ann-Kathrin Vandereike, Michael Sengpiel, Daniel Wessel, and Nicole Jochems. 2019. Finding User Preferences Designing the Innovative Interaction Device “BIRDY” for Intensive Care Patients. In Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018)(Advances in Intelligent Systems and Computing), Sebastiano Bagnara, Riccardo Tartaglia, Sara Albolino, Thomas Alexander, and Yushi Fujita(Eds.). Springer International Publishing, 698–707.

[16]

Börge Kordts, Jan Patrick Kopetz, Adrienne Henkel, Andreas Schrader, and Nicole Jochems. 2019. Requirements and Interaction Patterns for a Novel Interaction Device for Patients in Intensive Care. i-com 18, 1 (2019), 67–78. https://doi.org/10.1515/icom-2019-0004

[17]

Johnny Leporcq. 2018. Position Estimation Using IMU Without Tracking System. Ph.D. Dissertation. Aalto University, Helsinki.

[18]

S. O. H. Madgwick, A. J. L. Harrison, and R. Vaidyanathan. 2011. Estimation of IMU and MARG Orientation Using a Gradient Descent Algorithm. In 2011 IEEE International Conference on Rehabilitation Robotics. IEEE, Zurich, 1–7. https://doi.org/10.1109/ICORR.2011.5975346

[19]

A. Martinet, G. Casiez, and L. Grisoni. 2012. Integrality and Separability of Multitouch Interaction Techniques in 3D Manipulation Tasks. IEEE Transactions on Visualization and Computer Graphics 18, 3 (March 2012), 369–380. https://doi.org/10.1109/TVCG.2011.129

Digital Library

[20]

Daniel Mendes, Fernando Fonseca, Bruno Araujo, Alfredo Ferreira, and Joaquim Jorge. 2014. Mid-Air Interactions above Stereoscopic Interactive Tables. In 2014 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, MN, USA, 3–10. https://doi.org/10.1109/3DUI.2014.6798833

[21]

Daniel Mendes, Filipe Relvas, Alfredo Ferreira, and Joaquim Jorge. 2016. The Benefits of DOF Separation in Mid-Air 3D Object Manipulation. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology - VRST ’16. ACM Press, Munich, Germany, 261–268. https://doi.org/10.1145/2993369.2993396

Digital Library

[22]

Daniel Mendes, Maurício Sousa, Rodrigo Lorena, Alfredo Ferreira, and Joaquim Jorge. 2017. Using Custom Transformation Axes for Mid-Air Manipulation of 3D Virtual Objects. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology - VRST ’17. ACM Press, Gothenburg, Sweden, 1–8. https://doi.org/10.1145/3139131.3139157

Digital Library

[23]

Shio Miyafuji, Toshiki Sato, Zhengqing Li, and Hideki Koike. 2017. Qoom: An Interactive Omnidirectional Ball Display. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology(UIST ’17). ACM, New York, NY, USA, 599–609. https://doi.org/10.1145/3126594.3126607

Digital Library

[24]

Annette Mossel, Benjamin Venditti, and Hannes Kaufmann. 2013. 3DTouch and HOMER-S: Intuitive Manipulation Techniques for One-Handed Handheld Augmented Reality. In Proceedings of the Virtual Reality International Conference on Laval Virtual - VRIC ’13. ACM Press, Laval, France, 1. https://doi.org/10.1145/2466816.2466829

Digital Library

[25]

Gary Perelman, Marcos Serrano, Mathieu Raynal, Celia Picard, Mustapha Derras, and Emmanuel Dubois. 2015. The Roly-Poly Mouse: Designing a Rolling Input Device Unifying 2D and 3D Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15. ACM Press, Seoul, Republic of Korea, 327–336. https://doi.org/10.1145/2702123.2702244

Digital Library

[26]

Ivan Poupyrev, Suzanne Weghorst, and Sidney Fels. 2000. Non-Isomorphic 3D Rotational Techniques. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’00. ACM Press, The Hague, The Netherlands, 540–547. https://doi.org/10.1145/332040.332497

Digital Library

[27]

M. Rath and D. Rocchesso. 2005. Continuous Sonic Feedback from a Rolling Ball. IEEE Multimedia 12, 2 (April 2005), 60–69. https://doi.org/10.1109/MMUL.2005.24

Digital Library

[28]

Houssem Saidi, Marcos Serrano, Pourang Irani, and Emmanuel Dubois. 2017. TDome: A Touch-Enabled 6DOF Interactive Device for Multi-Display Environments. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). ACM, New York, NY, USA, 5892–5904. https://doi.org/10.1145/3025453.3025661

Digital Library

[29]

Jonmichael Seibert and Daniel M. Shafer. 2018. Control Mapping in Virtual Reality: Effects on Spatial Presence and Controller Naturalness. Virtual Reality 22, 1 (March 2018), 79–88. https://doi.org/10.1007/s10055-017-0316-1

Digital Library

[30]

Ann-Katrin Vandereike, Svenja Burgsmüller, Jan Patrick Kopetz, Michael Sengpiel, and Nicole Jochems. 2018. Interaction Paradigms of a Ball-Shaped Input Device for Intensive Care Patients. In Student Conference Proceedings 2018. Infinite Science Publishing, Lübeck, Germany.

[31]

Fabio Varesano and Fabiana Vernero. 2012. Introducing PALLA, a Novel Input Device for Leisure Activities: A Case Study on a Tangible Video Game for Seniors. In Proceedings of the 4th International Conference on Fun and Games(FnG ’12). ACM, New York, NY, USA, 35–44. https://doi.org/10.1145/2367616.2367621

Digital Library

[32]

Manuel Veit, Antonio Capobianco, and Dominique Bechmann. 2009. Influence of Degrees of Freedom’s Manipulation on Performances during Orientation Tasks in Virtual Reality Environments. In Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology - VRST ’09. ACM Press, Kyoto, Japan, 51. https://doi.org/10.1145/1643928.1643942

Digital Library

[33]

Samuel Wilson, Henry Eberle, Yoshikatsu Hayashi, Sebastian O.H. Madgwick, Alison McGregor, Xingjian Jing, and Ravi Vaidyanathan. 2019. Formulation of a New Gradient Descent MARG Orientation Algorithm: Case Study on Robot Teleoperation. Mechanical Systems and Signal Processing 130 (Sept. 2019), 183–200. https://doi.org/10.1016/j.ymssp.2019.04.064

[34]

x-io Technologies. 2012. Open Source IMU and AHRS Algorithms. https://x-io.co.uk/open-source-imu-and-ahrs-algorithms/.

Cited By

Razevicius GRoudaut AKarnik A(2024)HoberUI: An Exploration of Kinematic Structures as Interactive Input DevicesMultimodal Technologies and Interaction10.3390/mti80200138:2(13)Online publication date: 13-Feb-2024
https://doi.org/10.3390/mti8020013
Kim SLee G(2024)Evaluating an In-Hand Ball-Shaped Controller for Object Manipulation in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00025(10-19)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00025
Lürig CMentler T(2024)Comparison of a Kinematic and a Dynamic Docking Task Interaction Technique in an Interactive Physics AR Simulation2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00170(592-595)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00170
Show More Cited By

Index Terms

Manipulating Virtual Objects in Augmented Reality Using a New Ball-Shaped Input Device
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
    2. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Index terms have been assigned to the content through auto-classification.

Recommendations

Experiencing 3D interactions in virtual reality and augmented reality
EUSAI '04: Proceedings of the 2nd European Union symposium on Ambient intelligence

We demonstrate basic 2D and 3D interactions in both a Virtual Reality (VR) system, called the Personal Space Station, and an Augmented Reality (AR) system, called the Visual Interaction Platform. Since both platforms use identical (optical) tracking ...
Stiffness modulation for Haptic Augmented Reality: Extension to 3D interaction
HAPTIC '10: Proceedings of the 2010 IEEE Haptics Symposium

Haptic Augmented Reality (AR) allows a user to touch a real environment augmented with synthetic haptic stimuli. For example, medical students can palpate a virtual tumor inside a real mannequin using a haptic AR system to practice cancer detection. To ...
Augmented reality technologies, systems and applications

This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AH2021: 12th Augmented Human International Conference

May 2021

73 pages

ISBN:9781450390309

DOI:10.1145/3460881

Editors:
François Boyer
University Hospital Centre Reims, France
,
Jean-Marc Seigneur
University of Geneva, Switzerland
,
Amine Choukou
University of Manitoba, Canada
,
Redha Taiar
University of Reims Champagne-Ardenne, France

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AH2021

AH2021: 12th Augmented Human International Conference

May 27 - 28, 2021

Geneva, Switzerland

Acceptance Rates

Overall Acceptance Rate 121 of 306 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
257
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)4

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Razevicius GRoudaut AKarnik A(2024)HoberUI: An Exploration of Kinematic Structures as Interactive Input DevicesMultimodal Technologies and Interaction10.3390/mti80200138:2(13)Online publication date: 13-Feb-2024
https://doi.org/10.3390/mti8020013
Kim SLee G(2024)Evaluating an In-Hand Ball-Shaped Controller for Object Manipulation in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00025(10-19)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00025
Lürig CMentler T(2024)Comparison of a Kinematic and a Dynamic Docking Task Interaction Technique in an Interactive Physics AR Simulation2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00170(592-595)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00170
Paredes LIpsita AMesa JMartinez Garrido RRamani K(2022)StretchARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503056:3(1-26)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550305
Zhang XHe WBillinghurst MYang LFeng SLiu D(2022)Design and Evaluation of Bare-Hand Interaction for Precise Manipulation of Distant Objects in ARInternational Journal of Human–Computer Interaction10.1080/10447318.2022.215852740:9(2282-2296)Online publication date: 26-Dec-2022
https://doi.org/10.1080/10447318.2022.2158527

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten