research-article

Evaluating User Preferences for Augmented Reality Interactions with the Internet of Things

Authors:

Frank MaurerAuthors Info & Claims

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

Article No.: 20, Pages 1 - 9

https://doi.org/10.1145/3399715.3399716

Published: 02 October 2020 Publication History

Abstract

We investigate user preferences for controlling IoT devices with headset-based Augment Reality (AR), comparing gestural control and voice control. An elicitation study is performed with 16 participants to gather their preferred voice commands and gestures for a set of referents. We analyzed 784 inputs (392 gestures and 392 voice) as well as observations and interviews to develop an empirical basis for design recommendations that form a guideline for future designers and implementors of such voice commands and gestures for interacting with the IoT via headset-based AR.

References

[1]

AR Critic. What are the Magic Leap One Input Methods?. 2018. Retrieved: September 13, 2019 from https://arcritic.com/2293/what-are-the-magic-leap-one-input-methods/.

[2]

Augmented Reality | Definition of Augmented Reality by Lexico. 2019. Retrieved: September 13, 2019 from https://www.lexico.com/en/definition/augmented_reality.

[3]

Ceylan Beşevli, Oğuz Turan Buruk, Merve Erkaya, and Oğuzhan Özcan. 2018. Investigating the Effects of Legacy Bias.In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (DIS '18). ACM Press, New York, NY, 277--281. http://doi.acm.org/10.1145/3197391.3205449.

[4]

Richard A. Bolt. 1980. "Put-that-there: Voice and gesture at the graphics interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques (SIGGRAPH '80). ACM, New York, NY, USA, 262--270. DOI=http://dx.doi.org/10.1145/800250.807503.

Digital Library

[5]

Edwin Chan. 2017. User-defined Single-hand Microgestures. Master's thesis. University of Calgary, Calgary, Canada. Retrieved: March 28, 2020 from: https://prism.ucalgary.ca/bitstream/handle/11023/3827/ucalgary_2017_chan_edwin.pdf?sequence=3

[6]

Edwin Chan, Frank Maurer. 2016. User Elicitation on Single-hand Microgestures. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM Press, New York, NY, 3403--3414. http://doi.acm.org/10.1145/2858036.2858589.

Digital Library

[7]

Qing Chen. 2018.Immersive Analytics Interaction: User Preferences and Agreements by Task Type. Master's thesis. University of Calgary, Calgary, Canada. Retrieved: March 28, 2020 from: https://prism.ucalgary.ca/handle/1880/106633

[8]

Shreya Chopra. 2019. Evaluating User Preferences for Augmented Reality Interactions for the Internet of Things. Master's thesis. University of Calgary, Calgary, Canada. Retrieved: March 28, 2020 from https://prism.ucalgary.ca/handle/1880/111455e

[9]

, Lester F. Ludwig. 1991. Multidimensional audio window management, International Journal of Man-Machine Studies, Volume 34, Issue 3, 1991, Pages 319--336, ISSN 0020-7373, https://doi.org/10.1016/0020-7373(91)90023-Z.

Digital Library

[10]

Janet Groeber. 2015. Mixed Reality. Design: Retail. 27, 4 (Apr/May 2015) 22.

[11]

Alaric Hamacher, Jahanzeb Hafeez, Roland Csizmazia. 2019. Augmented Reality User Interface Evaluation - Performance Measurement of Hololens, Moverio and Mouse Input.International Journal of Interactive Mobile Technologies (iJIM). 13, 03 (2019), 95--107. https://doi.org/10.3991/ijim.v13i03.10226

[12]

Alexander G.Hauptmann, Paul McAvinney. 1993. Gestures with speech for graphic manipulation. International Journal of Man-Machine Studies. 38, 2 (Feb. 1993), 231--249. https://doi.org/10.1006/imms.1993.1011

Digital Library

[13]

Help - Google Glass. 2019. Retrieved: September 13, 2019 from https://www.google.com/glass/help/.

[14]

O. Hilliges D. Kim S. Izadi et al. "HoloDesk: direct 3d interactions with a situated see-through display In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems. pp. 2421--2430 2012.

[15]

Helge Hüttenrauch, Mike Tate, Martin Böcker, Rosemary Orr, and Françoise Petersen. 2008. Standardising SPOKEN commands for mobile devices and services. In Proceedings of the International Conference on Mobile Technology, Applications, and Systems (Mobility '08). https://doi.org/10.1145/1506270.1506345

Digital Library

[16]

IFTTT. 2019. Retrieved: September 13, 2019 from https://play.google.com/store/apps/details?id=com.ifttt.ifttt&hl=en.

[17]

Internet of things | Definition of Internet of things by Lexico. 2019. Retrieved: September 13, 2019 from https://www.lexico.com/en/definition/internet_of_things.

[18]

Stephen Kimani. 2009. WIMP Interfaces. In: Ling Liu, M. Tamer Özsu (Eds.) Encyclopedia of Database Systems. (2009), 3529--3533. https://doi.org/10.1007/978-0-387-39940-9_467

[19]

Sang-Su Lee, Jeonghun Chae, Hyunjeong Kim, Youn-kyung Lim, and Kun-pyo Lee. 2013. Towards more natural digital content manipulation via user freehand gestural interaction in a living room. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing (UbiComp '13), 617--626. https://doi.org/10.1145/2493432.2493480

Digital Library

[20]

Cherlynn Low. Lenovo launches ThinkReality AR and VR headset for enterprises. 2019. Retrieved: September 13, 2019 from https://www.engadget.com/2019/05/13/lenovo-thinkreality-ar-vr-headset-hololens-2/#/.

[21]

Meredith Ringel Morris. 2012. Web on the wall: Insights from a Multimodal Interaction Elicitation Study. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces (ITS '12), 95--104. https://doi.org/10.1145/2396636.2396651

Digital Library

[22]

Meredith Ringel Morris, Andreea Danielescu, Steven Drucker, Danyel Fisher, Bongshin Lee, M.C. Schraefel, and Jacob O. Wobbrock. 2014. Reducing legacy bias in gesture elicitation studies. interactions. 21, 3 (May/June 2014), 40--45. https://doi.org/10.1145/2591689

Digital Library

[23]

Meredith Ringel Morris, Jacob O. Wobbrock, & Andrew D. Wilson. (2010). Understanding Users' Preferences for Surface Gestures. Graphics Interface 2010, 261--268. Canadian Information Processing Society.

[24]

Michael Muller. 2014. Curiosity, Creativity, and Surprise as Analytic Tools: Grounded Theory Method. In: Olson J., Kellogg W. (eds) Ways of Knowing in HCI. (2014), 25--48. https://doi.org/10.1007/978-1-4939-0378-8_2.

[25]

Michael Nebeling, Alexander Huber, David Ott and Moira C. Norrie. 2014. Web on the Wall Reloaded. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (ITS '14), 15--24. https://doi.org/10.1145/2669485.2669497.

Digital Library

[26]

Scott Orgera. 2019. The Top 100 Google Assistant and Google Home Commands. Retrieved: September 13, 2019 from https://www.lifewire.com/top-google-assistant-and-google-home-commands-4158256.

[27]

Ekaterina Peshkova, Martin Hitz, David Ahlström. 2016. Exploring User-Defined Gestures and Voice Commands to Control an Unmanned Aerial Vehicle. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. (2016), 47--62. https://doi.org/10.1007/978-3-319-49616-0_5

[28]

T. Piumsomboon A. Clark M. Billinghurst P. Kotzé G. Marsden G. Lindgaard et al. User-Defined Gestures for Augmented Reality. In Proceedings of the International Conference on Human-Computer Interaction (INTERACT 2013). Heidelberg:Springer pp. 282--299 2013.

[29]

Jaime Ruiz, Daniel Vogel. 2015. Soft-Constraints to Reduce Legacy and Performance Bias to Elicit Whole-body Gestures with Low Arm Fatigue. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 3347--3350. https://doi.org/10.1145/2702123.2702583

Digital Library

[30]

Jim Tanous. 2019. The Lenovo ThinkReality A6 Is a Lightweight AR Headset for Businesses. Retrieved: September 13, 2019 from https://pcper.com/2019/05/lenovo-thinkreality-a6/.

[31]

Radu-Daniel Vatavu, Jacob O. Wobbrock. 2016. Between-Subjects Elicitation Studies: Formalization and Tool Support. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 3390--3402. https://doi.org/10.1145/2858036.2858228.

Digital Library

[32]

Jacob O. Wobbrock, Meredith Ringel Morris, Andrew D. Wilson. 2009. User-Defined Gestures for Surface Computing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '09), 1083--1092. https://doi.org/10.1145/1518701.1518866.

Digital Library

Cited By

Wang NZielasko DMaurer F(2024)User Preferences for Interactive 3D Object Transitions in Cross Reality - An Elicitation StudyProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656698(1-9)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656698
O'Hagan JSaeghe PGugenheimer JMedeiros DMarky KKhamis MMcGill M(2023)Privacy-Enhancing Technology and Everyday Augmented RealityProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35695016:4(1-35)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569501
Pronika Mudgal DYati G(2023)Augmented Reality for Advanced Home Applications2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT)10.1109/ICSSIT55814.2023.10060904(118-123)Online publication date: 23-Jan-2023
https://doi.org/10.1109/ICSSIT55814.2023.10060904
Show More Cited By

Index Terms

Evaluating User Preferences for Augmented Reality Interactions with the Internet of Things
1. Human-centered computing
  1. Human computer interaction (HCI)
  2. Interaction design
    1. Empirical studies in interaction design

Recommendations

Exploring Unimodal Notification Interaction and Display Methods in Augmented Reality
VRST '23: Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology

As we develop computing platforms for augmented reality (AR) head-mounted display (HMDs) technologies for social or workplace environments, understanding how users interact with notifications in immersive environments has become crucial. We researched ...
Collecting and Analyzing the Mid-Air Gestures Data in Augmented Reality and User Preferences in Closed Elicitation Study
Virtual, Augmented and Mixed Reality
Abstract
For users of AR-HMDs (i.e., AR glasses), one option for interaction with the HMD is through midair gestures. This modality allows users to move their hands to grab objects, move objects, select menus, and interact with a multiple set of features ...
"Megereality": Leveraging Physical Affordances for Multi-Device Gestural Interaction in Augmented Reality
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

We present a novel gestural interaction strategy for multi-device interactions in augmented reality (AR), in which we leverage existing physical affordances of everyday products and spaces for intuitive interactions in AR. To explore this concept, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

September 2020

613 pages

ISBN:9781450375351

DOI:10.1145/3399715

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AVI '20

AVI '20: International Conference on Advanced Visual Interfaces

September 28 - October 2, 2020

Salerno, Italy

Acceptance Rates

AVI '20 Paper Acceptance Rate 36 of 123 submissions, 29%;

Overall Acceptance Rate 128 of 490 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
173
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang NZielasko DMaurer F(2024)User Preferences for Interactive 3D Object Transitions in Cross Reality - An Elicitation StudyProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656698(1-9)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656698
O'Hagan JSaeghe PGugenheimer JMedeiros DMarky KKhamis MMcGill M(2023)Privacy-Enhancing Technology and Everyday Augmented RealityProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35695016:4(1-35)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569501
Pronika Mudgal DYati G(2023)Augmented Reality for Advanced Home Applications2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT)10.1109/ICSSIT55814.2023.10060904(118-123)Online publication date: 23-Jan-2023
https://doi.org/10.1109/ICSSIT55814.2023.10060904
Abraham MSaeghe PMcgill MKhamis M(2022)Implications of XR on Privacy, Security and Behaviour: Insights from ExpertsNordic Human-Computer Interaction Conference10.1145/3546155.3546691(1-12)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546691
Wang NMaurer F(2022)User-Centered Prototyping for Single-User Cross-Reality Virtual Object Transitions2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00039(171-174)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00039
Chopra S(2021)Virtual Control Panel API: An Artificial Intelligence Driven Directive to Allow Programmers and Users to Create Customizable, Modular, and Virtual Control Panels and Systems to Control IoT Devices via Augmented RealityHCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence10.1007/978-3-030-90963-5_30(391-400)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1007/978-3-030-90963-5_30
Chopra SAddam O(2021)Towards an Ambient Smart City: Using Augmented Reality to Geospatially Filter the Right Airbnb via Artificial IntelligenceArtificial Intelligence in HCI10.1007/978-3-030-77772-2_31(472-489)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1007/978-3-030-77772-2_31

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten