Eye Perturbation Approach for Robust Recognition of Inaccurately Aligned Faces

Abstract

Extraction of normalized face from input images is an essential preprocessing step of many face recognition algorithms. Typical face extraction algorithms make use of the locations of facial features, such as the center of eyes, that are marked either manually or automatically. It is not guaranteed, however, that we always obtain the exact or optimal locations of the eye centers, and using inaccurate landmark locations, and consequently poorly registered faces, is one of the main causes of performance degradation in appearance-based face recognition. Moreover, in some applications, it is hard to verify the correctness of the face extraction for every query image. For improved performance and robustness to the eye location variation, we propose an eye perturbation approach that generates multiple face extractions from a query image by using the perturbed eye locations centered at the initial eye locations. The extracted faces are then matched against the enrolled gallery set to produce individual similarity scores. Final decisions can be made by using various committee methods – nearest neighbor, maximum vote, etc. – of combining the results of individual classifiers. We conclude that the proposed eye perturbation approach with nearest neighbor classification improves recognition performance and makes existing face recognition algorithms robust to eye localization errors.