HCI Using a Robust Hand Tracking Method

Abstract

The interaction between man and computer has become an important topic as it can generalize the use of computers, robots and other intelligent machines. For active H/R interaction scheme, the computer needs to detect human faces in its vicinity and then interpret canonical gestures. In this paper, we firstly present a method to track the hands robustly in videos recorded by a camera, and then by compute the similarity of the trajectories of the hands’ motion, we can interact with computer by some simple sign languages. An efficient color segmentation on the skin-like colored pixels is used. Then we locate the hands by a fast optic flow method which is computed only by the skin-like image blocks. At the last stage, we get the hand motion trajectory and map it to a string. By String Edit Distance method, we can compute the similarity of the string representing input hand command with some template strings denoting sign language pre-trained and stored in database. We can let the computers or some other machines recognize some simple words like "UP","DOWN","OK", "LEFT","RIGHT" and etc. Results of tracking and recognition are illustrated in the paper and show the process robustness in cluttered environments and in various light conditions. The limits of the method and future works are also discussed.