skip to main content
10.1145/3025453.3025783acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Efficient Typing on a Visually Occluded Physical Keyboard

Published: 02 May 2017 Publication History

Abstract

The rise of affordable head-mounted displays (HMDs) has raised questions about how to best design user interfaces for this technology. This paper focuses on the use of HMDs for home and office applications that require substantial text input. A physical keyboard is a familiar and effective text input device in normal desktop computing. But without additional camera technology, an HMD occludes all visual feedback about a user's hand position over the keyboard. We describe a system that assists HMD users in typing on a physical keyboard. Our system has a virtual keyboard assistant that provides visual feedback inside the HMD about a user's actions on the physical keyboard. It also provides powerful automatic correction of typing errors by extending a state-of-the-art touchscreen decoder. In a study with 24 participants, we found our virtual keyboard assistant enabled users to type more accurately on a visually-occluded keyboard. We found users wearing an HMD could type at over 40 words-per-minute while obtaining an error rate of less than 5%.

References

[1]
Bowman, D. A., Rhoton, C. J., and Pinho, M. S. Text input techniques for immersive virtual environments: An empirical comparison. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, 26 (2002), 2154--2158.
[2]
Budhiraja, P., Sodhi, R., Jones, B., Karsch, K., Bailey, B., and Forsyth, D. Where's my drink? Enabling peripheral real world interactions while using HMDs. arXiv preprint arXiv:1502.04744 (2015).
[3]
Lee, M., and Woo, W. ARKB: 3D vision-based augmented reality keyboard. In Proceeding of the 13th International Conference on Artificial Reality and Telexistence (2003).
[4]
McGill, M., Boland, D., Murray-Smith, R., and Brewster, S. A dose of reality: Overcoming usability challenges in VR head-mounted displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, ACM (New York, NY, USA, 2015), 2143--2152.
[5]
Vertanen, K., and Kristensson, P. O. A versatile dataset for text entry evaluations based on genuine mobile emails. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, MobileHCI '11, ACM (New York, NY, USA, 2011), 295--298.
[6]
Vertanen, K., Memmi, H., Emge, J., Reyal, S., and Kristensson, P. O. VelociTap: Investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, ACM (New York, NY, USA, 2015), 659--668.
[7]
Walker, J., Kuhl, S., and Vertanen, K. Decoder-assisted typing using an HMD and a physical keyboard. In Extended Abstracts of the the ACM Conference on Human Factors in Computing Systems, CHI '16 workshop (New York, NY, USA, 2016).
[8]
Yi, X., Yu, C., Zhang, M., Gao, S., Sun, K., and Shi, Y. ATK: Enabling ten-finger freehand typing in air based on 3d hand tracking data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST '15, ACM (New York, NY, USA, 2015), 539--548.

Cited By

View all
  • (2025)Retransmitting Messages on Social Media in DisastersJournal of Global Information Management10.4018/JGIM.36796733:1(1-26)Online publication date: 24-Jan-2025
  • (2024)GradualReality: Enhancing Physical Object Interaction in Virtual Reality via Interaction State-Aware BlendingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676463(1-14)Online publication date: 13-Oct-2024
  • (2024)TouchEditorProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314547:4(1-29)Online publication date: 12-Jan-2024
  • Show More Cited By

Index Terms

  1. Efficient Typing on a Visually Occluded Physical Keyboard

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
    May 2017
    7138 pages
    ISBN:9781450346559
    DOI:10.1145/3025453
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. decoder
    2. head-mounted display
    3. physical keyboard
    4. text entry

    Qualifiers

    • Research-article

    Conference

    CHI '17
    Sponsor:

    Acceptance Rates

    CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)102
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Retransmitting Messages on Social Media in DisastersJournal of Global Information Management10.4018/JGIM.36796733:1(1-26)Online publication date: 24-Jan-2025
    • (2024)GradualReality: Enhancing Physical Object Interaction in Virtual Reality via Interaction State-Aware BlendingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676463(1-14)Online publication date: 13-Oct-2024
    • (2024)TouchEditorProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314547:4(1-29)Online publication date: 12-Jan-2024
    • (2024)FanPad: A Fan Layout Touchpad Keyboard for Text Entry in VR2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00045(222-232)Online publication date: 16-Mar-2024
    • (2024)Study of User Experience Benefits Amount Tangible User Interface for Spatial Computing2024 International Workshop on Intelligent Systems (IWIS)10.1109/IWIS62722.2024.10706027(1-4)Online publication date: 28-Aug-2024
    • (2023)Quantifying the 'Gorilla Arm' Effect in a Virtual Reality Text Entry Task via Ray-Casting: A Preliminary Single-Subject StudyProceedings of the 25th Symposium on Virtual and Augmented Reality10.1145/3625008.3625046(274-278)Online publication date: 6-Nov-2023
    • (2023)A Scoping Survey on Cross-reality SystemsACM Computing Surveys10.1145/361653656:4(1-38)Online publication date: 21-Oct-2023
    • (2023)Does One Keyboard Fit All? Comparison and Evaluation of Device-Free Augmented Reality Keyboard DesignsProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615692(1-11)Online publication date: 9-Oct-2023
    • (2023)A Dataset of Noisy Typing on QWERTY KeyboardsCompanion Proceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581754.3584174(251-254)Online publication date: 27-Mar-2023
    • (2023)From 2D to 3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808297:1(1-25)Online publication date: 28-Mar-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media