Multi-cue-based face and facial feature detection on video segments

Abstract

An approach is presented to detect faces and facial features on a video segment based on multi-cues, including gray-level distribution, color, motion, templates, algebraic features and so on. Faces are first detected across the frames by using color segmentation, template matching and artificial neural network. A PCA-based (Principal Component Analysis) feature detector for still images is then used to detect facial features on each single frame until the resulting features of three adjacent frames, named as base frames, are consistent with each other. The features of frames neighboring the base frames are first detected by the still-image feature detector, then verified and corrected according to the smoothness constraint and the planar surface motion constraint. Experiments have been performed on video segments captured under different environments, and the presented method is proved to be robust and accurate over variable poses, ages and illumination conditions.