Progress in Sign and Gesture Recognition

Abstract

Sign Language is a visual language (without a conventional written form) and consists of 3 major components, 1) Fingerspelling – used to spell words on a letter by letter basis, 2) Word level sign vocabulary – used for the majority of communication, 3) Non manual features – Facial expressions, tongue/mouth position and body posture used to modify sign meaning. Todate no system exists which has sufficient reliability and vocabulary to convert sign to speech at the level required for translation. This talk will present our work towards this goal. The talk will cover static pose recognition and tracking of the human hand, followed by the use of boosting for head and hand detection. It will be shown how prior statistical models of body configuration can be used to locate body parts, disambiguate the hands of the subject and predict the likely position of elbows and other body parts. Finally, it will be shown how these components are combined in a novel two-stage classifier architecture that provides a flexible monocular vision system capable of recognising sign lexicons far in excess of previous approaches. The approach boasts classification rates as high as 92 training instance per word, outperforming previous approaches where thousands of training examples are required.