skip to main content
research-article

Experience from Designing Augmented Reality Browsing Interfaces for Real-world Walking Scenarios

Published: 24 September 2024 Publication History

Abstract

Mobile phones have enabled users to browse information in varying mobility contexts. For high-mobility settings such as walking, however, phones pose several usability challenges, particularly safety and limited screen sizes. While Augmented Reality (AR) has been proposed to address these issues, prior work has yet to investigate AR interface design in real-world walking conditions beyond text readability and notification design. This paper presents the first exploration of AR browsing interface design and extended usage while walking in the wild. We first conducted design sessions with 12 UI designers while walking in varied environments to design the window size, distance, opacity, anchor type, and placement for three categories of apps: text, video, and mixed content. Results show that traffic level significantly affects the designed window size, whereas content type significantly affects window size, distance, opacity, and vertical placement. To gain further insights from real-world usage, we conducted a multi-day observational study with 5 participants and observed that participants on average switched among window layouts every 3.3 minutes, for reasons such as safety and the level of extended visual attention.

References

[1]
Yifei Cheng, Yukang Yan, Xin Yi, Yuanchun Shi, and David Lindlbauer. 2021. SemanticAdapt: Optimization-Based Adaptation of Mixed Reality Layouts Leveraging Virtual-Physical Semantic Connections. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST'21). Association for Computing Machinery, New York, NY, USA, 282--297. https://doi.org/10.1145/3472749.3474750
[2]
Ashley Colley, Juho Rantakari, and Jonna Häkkilä. 2014. Augmenting the Home to Remember: Initial User Perceptions. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (Seattle, Washington) (UbiComp '14 Adjunct). Association for Computing Machinery, New York, NY, USA, 1369--1372. https://doi.org/10.1145/2638728.2641717
[3]
Stephen DiVerdi, Daniel Nurmi, and Tobias Höllerer. 2003. ARWin-A Desktop Augmented Reality Window Manager. In Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '03). IEEE Computer Society, USA, 298.
[4]
Barrett Ens, Juan David Hincapié-Ramos, and Pourang Irani. 2014. Ethereal Planes: A Design Framework for 2D Information Space in 3D Mixed Reality Environments. In Proceedings of the 2nd ACM Symposium on Spatial User Interaction (Honolulu, Hawaii, USA) (SUI '14). Association for Computing Machinery, New York, NY, USA, 2--12. https://doi.org/10.1145/2659766.2659769
[5]
Barrett Ens, Eyal Ofek, Neil Bruce, and Pourang Irani. 2015. Spatial constancy of surface-embedded layouts across multiple environments. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction. ACM, New York, NY, United States, 65--68.
[6]
João Marcelo Evangelista Belo, Anna Maria Feit, Tiare Feuchtner, and Kaj Grønbæk. 2021. Xrgonomics: Facilitating the creation of ergonomic 3d interfaces. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1--11.
[7]
João Marcelo Evangelista Belo, Mathias N Lystbæk, Anna Maria Feit, Ken Pfeuer, Peter Kán, Antti Oulasvirta, and Kaj Grønbæk. 2022. Auit--the adaptive user interfaces toolkit for designing xr applications. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 1--16.
[8]
Steven Feiner, Blair MacIntyre, Marcus Haupt, and Eliot Solomon. 1993. Windows on the world: 2D windows for 3D augmented reality. In Proceedings of the 6th annual ACM symposium on User interface software and technology. ACM, Atlanta, Georgia, United States, 145--155.
[9]
Andreas Fender, Philipp Herholz, Marc Alexa, and Jörg Müller. 2018. Optispace: Automated placement of interactive 3d projection mapping content. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, United States, 1--11.
[10]
Andreas Fender, David Lindlbauer, Philipp Herholz, Marc Alexa, and Jörg Müller. 2017. Heatspace: Automatic placement of displays by empirical analysis of user behavior. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, United States, 611--621.
[11]
Shogo Fukushima, Takeo Hamada, and Ari Hautasaari. 2020. Comparing World and Screen Coordinate Systems in Optical See-Through Head-Mounted Displays for Text Readability while Walking. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, USA, 649--658. https://doi.org/10.1109/ISMAR50242.2020.00093
[12]
Ran Gal, Lior Shapira, Eyal Ofek, and Pushmeet Kohli. 2014. FLARE: Fast layout for augmented reality applications. In 2014 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, IEEE, USA, 207--212.
[13]
Raiful Hasan and Ragib Hasan. 2022. Pedestrian safety using the Internet of Things and sensors: Issues, challenges, and open problems. Future Generation Computer Systems 134 (2022), 187--203. https://doi.org/10.1016/j.future.2022.03.036
[14]
Inc. Hepta Platforms. 2024. Heptabase, A note-taking tool for visual learning. https://heptabase.com/
[15]
Nate Hill. 2023. 3D WebView: the ultimate cross-platform web browser for Unity. https://developer.vuplex.com/ webview/overview
[16]
Samat Imamov, Daniel Monzel, and Wallace S Lages. 2020. Where to display? how interface position aects comfort and task switching time on glanceable interfaces. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, IEEE, USA, 851--858.
[17]
Apple Inc. 2023. Design considerations for vision and motion - WWDC23 - videos. https://developer.apple.com/ videos/play/wwdc2023/10078
[18]
Meta Inc. 2024. Cast to a screen with Meta Quest. https://www.meta.com/en-us/help/quest/articles/in-vr-experiences/ oculus-features/cast-with-quest/
[19]
Meta Inc. 2024. Interaction SDK Overview. https://developer.oculus.com/documentation/unity/unity-isdk-interaction- sdk-overview/
[20]
Shaun K. Kane, Jacob O. Wobbrock, and Ian E. Smith. 2008. Getting o the Treadmill: Evaluating Walking User Interfaces for Mobile Devices in Public Spaces. In Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '08). Association for Computing Machinery, New York, NY, USA, 109--118. https://doi.org/10.1145/1409240.1409253
[21]
Elisa Maria Klose, Nils Adrian Mack, Jens Hegenberg, and Ludger Schmidt. 2019. Text Presentation for Augmented Reality Applications in Dual-Task Situations. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, USA, 636--644. https://doi.org/10.1109/VR.2019.8797992
[22]
Wallace S. Lages and Doug A. Bowman. 2019. Walking with Adaptive Augmented Reality Workspaces: Design and Usage Patterns. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI '19). Association for Computing Machinery, New York, NY, USA, 356--366. https://doi.org/10.1145/3301275.3302278
[23]
Hyunjin Lee, Sunyoung Bang, and Woontack Woo. 2023. Eects of coordinate system and position of AR notication while walking. Virtual Reality 27, 2 (2023), 829--848.
[24]
David Lindlbauer, Anna Maria Feit, and Otmar Hilliges. 2019. Context-aware online adaptation of mixed reality interfaces. In Proceedings of the 32nd annual ACM symposium on user interface software and technology. ACM, New York, NY, United States, 147--160.
[25]
Jiazhou Liu, Arnaud Prouzeau, Barrett Ens, and Tim Dwyer. 2020. Design and Evaluation of Interactive Small Multiples Data Visualisation in Immersive Spaces. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, USA, 588--597. https://doi.org/10.1109/VR46266.2020.00081
[26]
Feiyu Lu and Doug A. Bowman. 2021. Evaluating the Potential of Glanceable AR Interfaces for Authentic Everyday Uses. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, USA, 768--777. https://doi.org/10.1109/VR50410.2021.00104
[27]
Feiyu Lu, Shakiba Davari, Lee Lisle, Yuan Li, and Doug A. Bowman. 2020. Glanceable AR: Evaluating Information Access Methods for Head-Worn Augmented Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, USA, 930--939. https://doi.org/10.1109/VR46266.2020.00113
[28]
Feiyu Lu and Yan Xu. 2022. Exploring Spatial UI Transition Mechanisms with Head-Worn Augmented Reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI'22). Association for Computing Machinery, New York, NY, USA, Article 550, 16 pages. https://doi.org/10.1145/3491102. 3517723
[29]
Allison Martin. 2006. Factors inuencing pedestrian safety: a literature review. TRL Wokingham, Berks, UK. PPR241.
[30]
Florian Müller, Joshua McManus, Sebastian Günther, Martin Schmitz, Max Mühlhäuser, and Markus Funk. 2019. Mind the Tap: Assessing Foot-Taps for Interacting with Head-Mounted Displays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3290605.3300707
[31]
Florian Müller, Martin Schmitz, Daniel Schmitt, Sebastian Günther, Markus Funk, and Max Mühlhäuser. 2020. Walk The Line: Leveraging Lateral Shifts of the Walking Path as an Input Modality for Head-Mounted Displays. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI'20). Association for Computing Machinery, New York, NY, USA, 1--15. https://doi.org/10.1145/3313831.3376852
[32]
Benjamin Nuernberger, Eyal Ofek, Hrvoje Benko, and Andrew D Wilson. 2016. Snaptoreality: Aligning augmented reality to the real world. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, United States, 1233--1244.
[33]
Joris Peereboom, Wilbert Tabone, Dimitra Dodou, and Joost de Winter. 2023. Head-locked, world-locked, or conformal diminished-reality? An examination of dierent AR solutions for pedestrian safety in occluded scenarios. ResearchGate ResearchGate (06 2023).
[34]
Xun Qian, Fengming He, Xiyun Hu, Tianyi Wang, Ananya Ipsita, and Karthik Ramani. 2022. Scalar: Authoring semantically adaptive augmented reality experiences in virtual reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, United States, 1--18.
[35]
Ashwin Ram and Shengdong Zhao. 2021. LSVP: Towards Eective On-the-go Video Learning Using Optical Head- Mounted Displays. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 1, Article 30 (mar 2021), 27 pages. https://doi.org/10.1145/3448118
[36]
Rufat Rzayev, Pawe? W Wo?niak, Tilman Dingler, and Niels Henze. 2018. Reading on smart glasses: The eect of text position, presentation type and walking. In Proceedings of the 2018 CHI conference on human factors in computing systems. ACM, New York, NY, United States, 1--9.
[37]
Marc Satkowski, Rufat Rzayev, Eva Goebel, and Raimund Dachselt. 2022. ABOVE & BELOW: Investigating Ceiling and Floor for Augmented Reality Content Placement. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, USA, 518--527. https://doi.org/10.1109/ISMAR55827.2022.00068
[38]
Philip Stoker, Andrea Garnkel-Castro, Meleckidzedeck Khayesi, Wilson Odero, Martin N Mwangi, Margie Peden, and Reid Ewing. 2015. Pedestrian safety and the built environment: a review of the risk factors. Journal of Planning Literature 30, 4 (2015), 377--392.
[39]
Nicholas DA Thomas, James D Gardiner, Robin H Crompton, and Rebecca Lawson. 2020. Look out: an exploratory study assessing how gaze (eye angle and head angle) and gait speed are inuenced by surface complexity. PeerJ 8 (2020), e8838.
[40]
Maarten Van Someren, Yvonne F Barnard, and J Sandberg. 1994. The think aloud method: a practical approach to modelling cognitive. London: AcademicPress 11 (1994), 29--41.
[41]
Leena Ventä-Olkkonen, Jonna Häkkilä, and Kaisa Väänänen-Vainio-Mattila. 2014. Exploring the Augmented Home Window: User Perceptions of the Concept. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia (Melbourne, Victoria, Australia) (MUM '14). Association for Computing Machinery, New York, NY, USA, 190--198. https://doi.org/10.1145/2677972.2677994
[42]
Jacob O Wobbrock, Leah Findlater, Darren Gergle, and James J Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI conference on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, 143--146.
[43]
Chen Zhou, Katherine Fennedy, Felicia Fang-Yi Tan, Shengdong Zhao, and Yurui Shao. 2023. Not All Spacings are Created Equal: The Eect of Text Spacings in On-the-go Reading Using Optical See-Through Head-Mounted Displays. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI'23). Association for Computing Machinery, New York, NY, USA, Article 720, 19 pages. https://doi.org/10.1145/3544548. 3581430

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 8, Issue MHCI
MHCI
September 2024
1136 pages
EISSN:2573-0142
DOI:10.1145/3697825
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 September 2024
Published in PACMHCI Volume 8, Issue MHCI

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. augmented reality
  2. user interface design
  3. walking user interface

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 264
    Total Downloads
  • Downloads (Last 12 months)264
  • Downloads (Last 6 weeks)58
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media