skip to main content
10.1145/3605655.3605656acmotherconferencesArticle/Chapter ViewAbstractPublication PagesecceConference Proceedingsconference-collections
research-article

Move’n’Hold: Scalable Device-Based Interaction for Mixed Reality Handheld Displays

Published: 21 September 2023 Publication History

Abstract

While handheld displays provide a widely available, low-budget access to Mixed Reality, developing appropriate interaction techniques remains challenging. For example, one-handed touch- and gestures-based methods are prone to fatigue and occlusion issues and two-handed device-based techniques are constrained to small-range manipulations or their pre-defined thresholds limit user control. In this paper, we introduce Move’n’Hold – a universally applicable interaction paradigm for translating and rotating virtual objects solely through a handheld display’s movement and peripheral touch. Move’n’Hold combines direct mapping between device and object manipulations when only left-thumb-touch is applied with automated repetitions of these initial manipulations that are started or stopped when right-thumb-touch is added or released. Hence, our technique allows switching individually between natural manipulation for small, precise movements and continuous manipulation for large, coarse movements. Our evaluation revealed Move’n’Hold as an intuitive and easy-to-learn input technique for multidimensional object translations and rotations. The technique also provides high scalability in terms of the distance, direction, complexity, and speed of manipulation. At the same time, it supports different user preferences and interaction styles. Our results further show that learning translation prior to rotation enhanced the overall user experience.

References

[1]
National Aeronautics and Space Administration. 2022. NASA TLX Paper and Pencil Version Instruction Manual. Retrieved January 11, 2023 from https://humansystems.arc.nasa.gov/groups/tlx/tlxpaperpencil.php
[2]
Jonas Blattgerste, Kristina Luksch, Carmen Lewa, and Thies Pfeiffer. 2021. TrainAR: A Scalable Interaction Concept and Didactic Framework for Procedural Trainings Using Handheld Augmented Reality. Multimodal Technologies and Interaction 5, 7 (2021). https://doi.org/10.3390/mti5070030
[3]
Jean Botev, Joe Mayer, and Steffen Rothkugel. 2019. Immersive Mixed Reality Object Interaction for Collaborative Context-Aware Mobile Training and Exploration. In Proceedings of the 11th ACM Workshop on Immersive Mixed and Virtual Environment Systems (Amherst, Massachusetts) (MMVE ’19). 4–9. https://doi.org/10.1145/3304113.3326117
[4]
Varunyu Fuvattanasilp, Yuichiro Fujimoto, Alexander Plopski, Takafumi Taketomi, Christian Sandor, Masayuki Kanbara, and Hirokazu Kato. 2021. SlidAR+: Gravity-aware 3D object manipulation for handheld augmented reality. Computers & Graphics 95 (2021), 23–35. https://doi.org/10.1016/j.cag.2021.01.005
[5]
Eg Su Goh, Mohd Shahrizal Sunar, and Ajune Wanis Ismail. 2019. 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review. IEEE Access 7 (2019), 40581–40601. https://doi.org/10.1109/ACCESS.2019.2906394
[6]
Jerônimo G Grandi, Henrique G Debarba, Iago Bemdt, Luciana Nedel, and Anderson Maciel. 2018. Design and Assessment of a Collaborative 3D Interaction Technique for Handheld Augmented Reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Tuebingen/Reutlingen, Germany). 49–56. https://doi.org/10.1109/VR.2018.8446295
[7]
Rebecca A. Grier. 2015. How High is High? A Meta-Analysis of NASA-TLX Global Workload Scores. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 59, 1 (2015), 1727–1731. https://doi.org/10.1177/1541931215591373
[8]
Minseok Kim and Jae Yeol Lee. 2016. Touch and hand gesture-based interactions for directly manipulating 3D virtual objects in mobile augmented reality. Multimedia Tools and Applications 75, 23 (2016), 16529–16550. https://doi.org/10.1007/s11042-016-3355-9
[9]
Asier Marzo, Benoît Bossavit, and Martin Hachet. 2014. Combining Multi-Touch Input and Device Movement for 3D Manipulations in Mobile Augmented Reality Environments. In Proceedings of the 2nd ACM Symposium on Spatial User Interaction (Honolulu, Hawaii, USA) (SUI ’14). 13–16. https://doi.org/10.1145/2659766.2659775
[10]
Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1995. Augmented reality: a class of displays on the reality-virtuality continuum. In Telemanipulator and Telepresence Technologies, Hari Das (Ed.). Vol. 2351. SPIE, 282 – 292. https://doi.org/10.1117/12.197321
[11]
Annette Mossel, Benjamin Venditti, and Hannes Kaufmann. 2013. 3DTouch and HOMER-S: Intuitive Manipulation Techniques for One-Handed Handheld Augmented Reality. In Proceedings of the Virtual Reality International Conference: Laval Virtual (Laval, France) (VRIC ’13). Article 12, 10 pages. https://doi.org/10.1145/2466816.2466829
[12]
Anja Naumann and Jörn Hurtienne. 2010. Benchmarks for Intuitive Interaction with Mobile Devices. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (Lisbon, Portugal) (MobileHCI ’10). 401–402. https://doi.org/10.1145/1851600.1851685
[13]
Ali Samini and Karljohan Lundin Palmerius. 2016. A Study on Improving Close and Distant Device Movement Pose Manipulation for Hand-Held Augmented Reality. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology (Munich, Germany) (VRST ’16). 121–128. https://doi.org/10.1145/2993369.2993380
[14]
Goh Eg Su, Mohd Shahrizal Sunar, and Ajune Wanis Ismail. 2020. Device-based manipulation technique with separated control structures for 3D object translation and rotation in handheld mobile AR. International Journal of Human-Computer Studies 141 (2020), 102433. https://doi.org/10.1016/j.ijhcs.2020.102433

Cited By

View all
  • (2024)Move'n'Hold Pro: Consistent Spatial Interaction Techniques for Object Manipulation with Handheld and Head-mounted Displays in Extended RealityProceedings of the European Conference on Cognitive Ergonomics 202410.1145/3673805.3673814(1-8)Online publication date: 8-Oct-2024

Index Terms

  1. Move’n’Hold: Scalable Device-Based Interaction for Mixed Reality Handheld Displays

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ECCE '23: Proceedings of the European Conference on Cognitive Ergonomics 2023
    September 2023
    189 pages
    ISBN:9798400708756
    DOI:10.1145/3605655
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 September 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Augmented Reality
    2. Device-based Interaction
    3. Handheld Display
    4. Mixed Reality

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    Conference

    ECCE 2023
    ECCE 2023: European Conference in Cognitive Ergonomics
    September 19 - 22, 2023
    Swansea, United Kingdom

    Acceptance Rates

    Overall Acceptance Rate 56 of 91 submissions, 62%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)99
    • Downloads (Last 6 weeks)10
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Move'n'Hold Pro: Consistent Spatial Interaction Techniques for Object Manipulation with Handheld and Head-mounted Displays in Extended RealityProceedings of the European Conference on Cognitive Ergonomics 202410.1145/3673805.3673814(1-8)Online publication date: 8-Oct-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media