Hao Zhang
ABSTRACT
In Human-Computer Interactions (HCI), to reduce the dependency of bulky devices like physical keyboards and joysticks, many gesture-based HCI schemes are adopted. As a typical HCI technology, text input has aroused much concern and many virtual or wearable keyboards have been proposed. To further remove the keyboard and allow people to type in a device-free way, we propose AirTyping, i.e., a mid-air typing scheme based on Leap Motion. During the typing process, the Leap Motion Controller captures the typing gestures with cameras and provides the coordinates of finger joints. Then, AirTyping detects the possible keystrokes, infers the typed words based on Bayesian method, and outputs the inputted word sequence. The experiment results show that our system can detect the keystrokes and infer the typed text efficiently, i.e., the true positive rate of keystroke detection is 92.2%, while the accuracy that the top-1 inferred word is the typed word achieves 90.2%.
- Microsoft Hololens, https://www.microsoft.com/en-us/hololens.Google Scholar
- L. Xie and C. Wang and A. X. Liu and J. Sun and S. Lu, Multi-Touch in the Air: Concurrent Micromovement Recognition Using RF Signals, in IEEE/ACM Transactions on Networking, 2018.Google Scholar
- Y. Yin and Q. Li and L. Xie and S. Yi and E. Novak and S. Lu, CamK: Camera-Based Keystroke Detection and Localization for Small Mobile Devices, in IEEE Transactions on Mobile Computing, 2018.Google Scholar
- X. Yi and C. Yu and M. Zhang and S. Gao and K. Sun and Y. Shi, ATK: Enabling ten-finger freehand typing in air based on 3d hand tracking data, in ACM Symposium on User Interface Software Technology, 2015.Google ScholarDigital Library
- A. Boudjelthia and S. Nasim and J. Eskola and J. Adeegbe and O. Hourula and S. Klakegg and D. Ferreira, Enabling Mid-air Browser Interaction with Leap Motion, in ACM International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, 2018.Google ScholarDigital Library
- Leap Motion, https://developer.leapmotion.com.Google Scholar
- An average professional typist types usually in speeds of 50 to 80 wpm, https://en.wikipedia.org/wiki/Words-per-minute.Google Scholar
- Word Frequency Data Set, https://www.wordfrequency.info.Google Scholar
Index Terms
- AirTyping: a mid-air typing scheme based on leap motion
Recommendations
Leap motion gesture based interface for learning environment by using leap motion
HCIK '15: Proceedings of HCI KoreaNUI(Natural User Interface) which means a natural user manipulation environment uses body as an input device by using a sensor, without using an input device such as a mouse or a keyboard. For these reasons, it has a feature that can be used easily as ...
A leap for touch: proximity sensitive touch targets in cars
AutomotiveUI '15: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsCombining touch screen technology with mid-air gestures into a unified input modality has potential to improve interaction with touch interfaces in cars. Moreover, target objects on a touch screen can be adapted to the proximity of a users' finger in ...
Physics-based gaming: exploring touch vs. mid-air gesture input
IDC '15: Proceedings of the 14th International Conference on Interaction Design and ChildrenPhysics-based games, like Cut the Rope, have become very popular and are now available on different operating systems with different input modalities. In a user study with 20 children aged 11 to 14 years, we investigated the differences in player ...
Comments