Christoph Amma
ABSTRACT
In this work we present a wearable input device which enables the user to input text into a computer. The text is written into the air via character gestures, like using an imaginary blackboard. To allow hands-free operation, we designed and implemented a data glove, equipped with three gyroscopes and three accelerometers to measure hand motion. Data is sent wirelessly to the computer via Bluetooth. We use HMMs for character recognition and concatenated character models for word recognition. As features we apply normalized raw sensor signals. Experiments on single character and word recognition are performed to evaluate the end-to-end system. On a character database with 10 writers, we achieve an average writer-dependent character recognition rate of 94.8% and a writer-independent character recognition rate of 81.9%. Based on a small vocabulary of 652 words, we achieve a single-writer word recognition rate of 97.5%, a performance we deem is advisable for many applications. The final system is integrated into an online word recognition demonstration system to showcase its applicability.
- O. Amft, R. Amstutz, Smailagic, A., D. Siewiorek, and G. Tröster. Gesture-controlled user input to complete questionnaires on wrist-worn watches. In Human-Computer Interaction. Novel Interaction Methods and Techniques, volume 5611 of Lecture Notes in Computer Science, pages 131--140. Springer Berlin / Heidelberg, 2009. Google Scholar
Digital Library
- W.-C. Bang, W. Chang, K.-H. Kang, E.-S. Choi, A. Potanin, and D.-Y. Kim. Self-contained spatial input device for wearable computers. Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings., pages 26--34, 2003. Google ScholarDigital Library
- H. Brashear, T. Starner, P. Lukowicz, and H. Junker. Using multiple sensors for mobile sign language recognition. Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings., pages 45--52, 2003. Google ScholarDigital Library
- S.-J. Cho and J. Kim. Bayesian network modeling of strokes and their relationships for on-line handwriting recognition. Pattern Recognition, 37(2):253--264, 2004.Google ScholarCross Ref
- A. Hein, A. Hoffmeyer, and T. Kirste. Utilizing an accelerometric bracelet for ubiquitous gesture-based interaction. In Universal Access in Human-Computer Interaction. Intelligent and Ubiquitous Interaction Environments, volume 5615 of Lecture Notes in Computer Science, pages 519--527. Springer Berlin / Heidelberg, 2009. Google ScholarDigital Library
- F. Hofmann, P. Heyer, and G. Hommel. Velocity profile based recognition of dynamic gestures with discrete hidden markov models. In Gesture and Sign Language in Human-Computer Interaction, volume 1371 of Lecture Notes in Computer Science, pages 81--95. Springer Berlin / Heidelberg, 1998. Google ScholarDigital Library
- D. Kim, H. Choi, and J. Kim. 3d space handwriting recognition with ligature model. In Ubiquitous Computing Systems, volume 4239/2006 of Lecture Notes in Computer Science, pages 41--56. Springer Berlin / Heidelberg, 2006. Google ScholarDigital Library
- I.-C. Kim and S.-I. Chien. Analysis of 3d hand trajectory gestures using stroke-based composite hidden markov models. Applied Intelligence, 15(2):131--143, 2001. Google ScholarDigital Library
- R.-H. Liang and M. Ouhyoung. A real-time continuous gesture recognition system for sign language. In Third IEEE International Conference on Automatic Face and Gesture Recognition, 1998. Proceedings., pages 558--567, 1998. Google ScholarDigital Library
- R. McGuire, J. Hernandez-Rebollar, T. Starner, V. Henderson, H. Brashear, and D. Ross. Towards a one-way american sign language translator. In Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings., pages 620--625, 2004. Google ScholarDigital Library
- J. Oh, S.-J. Cho, W.-C. Bang, W. Chang, E. Choi, J. Yang, J. Cho, and D. Kim. Inertial sensor based recognition of 3-d character gestures with an ensemble classifiers. Ninth International Workshop on Frontiers in Handwriting Recognition, 2004. IWFHR-9 2004., pages 112--117, 2004. Google ScholarDigital Library
- R. Plamondon and S. Srihari. Online and off-line handwriting recognition: a comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1):63--84, 2000. Google ScholarDigital Library
- L. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. In Proceedings of the IEEE, pages 257--286, 1989.Google ScholarCross Ref
- M. Rehm, N. Bee, and E. André. Wave like an egyptian: accelerometer based gesture recognition for culture specific interactions. In BCS-HCI '08: Proceedings of the 22nd British HCI Group Annual Conference on HCI 2008, pages 13--22, Swinton, UK, UK, 2008. British Computer Society. Google ScholarDigital Library
- J. Wu, G. Pan, D. Zhang, G. Qi, and S. Li. Gesture recognition with a 3-d accelerometer. In Ubiquitous Intelligence and Computing, volume 5585 of Lecture Notes in Computer Science, pages 25--38. Springer Berlin / Heidelberg, 2009. Google ScholarDigital Library
Index Terms
- Airwriting recognition using wearable motion sensors
Recommendations
Airwriting: demonstrating mobile text input by 3D-space handwriting
IUI '12: Proceedings of the 2012 ACM international conference on Intelligent User InterfacesWe demonstrate our airwriting interface for mobile hands-free text entry. The interface enables a user to input text into a computer by writing in the air like on an imaginary blackboard. Hand motion is measured by an accelerometer and a gyroscope ...
Airwriting: a wearable handwriting recognition system
We present a wearable input system which enables interaction through 3D handwriting recognition. Users can write text in the air as if they were using an imaginary blackboard. The handwriting gestures are captured wirelessly by motion sensors applying ...
Online handwriting recognition using depth sensors
ICDAR '15: Proceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR)In this work, we propose an online handwriting solution, where the data is captured with the help of depth sensors. Users may write in the air and our method recognizes it in real time using the proposed feature representation. Our method uses an ...
Comments