skip to main content
10.1145/3485279.3485302acmconferencesArticle/Chapter ViewAbstractPublication PagessuiConference Proceedingsconference-collections
abstract

Toward Using Machine Learning-Based Motion Gesture for 3D Text Input

Published: 09 November 2021 Publication History

Abstract

In this paper, we discuss our implementation of a gesture-based 3-dimensional typing system within virtual reality. Rather than the conventional point-and-click keyboard commonly found in immersive technologies, we explore using unique gestures with the controller to enter a specific key. To map these gestures and movements to their respective key, we utilize machine learning techniques to avoid naive hard-coded implementations. The result of the trained model is a text input system that adapts to the user’s gestures, rather than forcing the user to conform to the system’s definition of a gesture. Our goal is to work toward a viable alternative to standard virtual reality keyboards and typing systems.

References

[1]
Jens Grubert, Lukas Witzani, Eyal Ofek, Michel Pahud, Matthias Kranz, and Per Ola Kristensson. 2018. Effects of Hand Representations for Typing in Virtual Reality. IEEE Virtual Reality. https://www.microsoft.com/en-us/research/uploads/prod/2018/02/2018_ieee_vr_hands_representations.pdf
[2]
Janis Gisel Jimenez. 2017. A Prototype for Text Input in Virtual Reality with a Swype-like Process Using a Hand-tracking Device. University of California San Diego. https://escholarship.org/uc/item/43r3t7m8
[3]
Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-Based Text Entry in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174221
[4]
Zhen Yang, Cheng Chen, Yuqing Lin, Duming Wang, Hongting Li, and Weidan Xu. 2019. Effect of spatial enhancement technology on input through the keyboard in virtual reality environment. Applied Ergonomics 78, 164–175. https://doi.org/10.1016/j.apergo.2019.03.006
[5]
Powen Yao, Vangelis Lympouridis, Tian Zhu, Michael Zyda, and Ruoxi Jia. 2020. Punch Typing: Alternative Method for Text Entry in Virtual Reality. In Symposium on Spatial User Interaction. 1–2.

Cited By

View all
  • (2024)Design and evaluation of alphabetic and numeric input methods for virtual realityComputers and Graphics10.1016/j.cag.2024.103955122:COnline publication date: 1-Aug-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SUI '21: Proceedings of the 2021 ACM Symposium on Spatial User Interaction
November 2021
206 pages
ISBN:9781450390910
DOI:10.1145/3485279
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2021

Check for updates

Author Tags

  1. gesture typing
  2. machine learning
  3. text entry
  4. virtual reality

Qualifiers

  • Abstract
  • Research
  • Refereed limited

Conference

SUI '21
SUI '21: Symposium on Spatial User Interaction
November 9 - 10, 2021
Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 86 of 279 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)13
  • Downloads (Last 6 weeks)3
Reflects downloads up to 25 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Design and evaluation of alphabetic and numeric input methods for virtual realityComputers and Graphics10.1016/j.cag.2024.103955122:COnline publication date: 1-Aug-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media