Human-Centered Fingertip Mandarin Input System Using Single Camera

Abstract

Designing a user friendly Chinese input system is a challenging task due to the logographic nature of Chinese characters. Using fingertips and cameras to replace pens and touch panels as input devices could reduce the cost and improve ease-of-use and comfort of the computer-human interface. In this work, Chinese character entry is achieved using Mandarin Phonetic Symbol (MPS) recognition via on-line fingertip tracking. In the proposed system, particle filters are applied for robust fingertip tracking. Afterwards, MPS recognition is performed on the tracked fingertip trajectories using Hidden Markov Models. In the proposed system, the challenges of entering, leaving, and virtual strokes caused by video-based fingertip input can be overcome. We conduct experiments to validate that the MPS symbols written by fingertips are successfully and efficiently recognized using the proposed framework.