skip to main content
10.1145/3491101.3519900acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
demonstration

Demonstrating TapType for mobile ten-finger text entry anywhere

Published: 28 April 2022 Publication History

Abstract

We demonstrate a mobile text entry system that brings full-size ten-finger typing to everyday surfaces, allowing users to type anywhere. Our wearable wristband TapType integrates accelerometers that sense vibrations arising from finger taps against a passive surface, from which our Bayesian neural network estimates a probability distribution over the fingers of the hand. Given a pre-defined key-finger mapping, our text entry decoder fuses these predictions with the character priors of an n-gram language model to decode the input text entered by the user. TapType combines high portability with sustained rapid bimanual input across the full space, which we demonstrate at the example of supplementing text input on mobile touch devices, in eyes-free scenarios using audio feedback, and in a situated Mixed Reality scenario to enable typing outside visual control with passive haptic feedback.

Supplementary Material

VTT File (3491101.3519900-walkthrough.vtt)
MP4 File (3491101.3519900-walkthrough.mp4)
Video Figure (Demo Walkthrough video)
MP4 File (3491101.3519900-preview.mp4)
Video Preview

References

[1]
Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D Wilson. 2017. Sparse haptic proxy: Touch feedback in virtual environments using a general passive prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 3718–3728.
[2]
Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language modeling for soft keyboards. In Proceedings of the 7th international conference on Intelligent user interfaces. 194–195.
[3]
Manuel Meier, Paul Streli, Andreas Fender, and Christian Holz. 2021. TapID: Rapid Touch Interaction in Virtual Reality using Wearable Sensing. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 519–528.
[4]
D Raj Reddy 1977. Speech understanding systems: A summary of results of the five-year research effort. department of computer science.
[5]
Paul Streli, Jiaxi Jiang, Andreas Fender, Manuel Meier, Hugo Romat, and Christian Holz. 2022. TapType: Ten-finger text entry on everyday surfaces via Bayesian inference. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3491102.3501878
[6]
Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology(Charlotte, NC, USA) (UIST ’15). Association for Computing Machinery, New York, NY, USA, 539–548. https://doi.org/10.1145/2807442.2807504

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems
April 2022
3066 pages
ISBN:9781450391566
DOI:10.1145/3491101
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Check for updates

Author Tags

  1. Bayesian inference
  2. Bayesian neural network
  3. invisible interfaces
  4. mobile text entry
  5. n-gram language model
  6. virtual reality

Qualifiers

  • Demonstration
  • Research
  • Refereed limited

Conference

CHI '22
Sponsor:
CHI '22: CHI Conference on Human Factors in Computing Systems
April 29 - May 5, 2022
LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 198
    Total Downloads
  • Downloads (Last 12 months)28
  • Downloads (Last 6 weeks)3
Reflects downloads up to 22 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media