Illumination compensation for facial feature point localization in a single 2D face image

Abstract

Current researches have demonstrated that illumination variation on face images degrades the accuracy of facial identity and emotion recognition. To decrease the impact of illumination variation, researchers have proposed many creative methods of illumination compensation. However, these methods are limited in compensating for the shadow around the nose. On the basis of our previous researches, we now propose a novel approach which can effectively decrease the impact of illumination variation, especially the shadow around the nose. Firstly, we preprocessed the face image with uneven brightness using technologies of illuminant direction estimation and improved Retinex. Secondly, we turn the original face image into a binary image with only shadow region or non-shadow region using region growing technology. Thirdly, we calculate the difference between the intensity of the original input face image and the average intensity of the face images under the frontal illumination. Fourthly, for the face image preprocessed in the first step, we keep its non-shadow region. For the intensity difference, we extract its shadow region whose intensity is reduced by an adaptive value. Fifthly, we synthesize the non-shadow region and the shadow region in step four. Finally, we apply maximum filter to smooth the boundary between them. The proposed method is simple in computation and does not need any training steps or any knowledge of 3D models. The experimental results using extended Yale face database B show that our method achieves better illumination compensation comparing with the existing techniques, and provide more satisfactory experimental data for facial identity and emotion recognition.

Introduction

Facial identity and emotion recognition have been widely used in our daily life and have extensive application prospects in various industries, such as medical care, health, family service, sports, entessrtainment and education. However, there are still many uncontrollable factors restricting the development of facial identity and emotion recognition. Among them, the most representative factor is uneven illumination. In order to address the problems due to uneven illumination, researchers have done some relevant works [1], [2], [3], [4]. Vu and Caplier [1] proposed a new approach of illumination normalization based on Retina modeling by combining two adaptive nonlinear functions and a difference of Gaussians filter. Zhao and Wang [2] used Discrete Cosine Transform (DCT) on the original face images in logarithm domain, and then utilized local normalization method on the inverse discrete cosine transform images to minimize illumination variations in small areas. In order to achieve the illumination invariant eye detection, Jung et al. [3] employed illumination normalization based on Retinex theory. Liu et al. [4] employed Sobel operator to estimate gradients in face images, and then utilized these gradients as weights in an averaging operation. For the purpose of accelerating the operation and removing certain unwanted features, they calculated integral images from the weighted images. Although all above-mentioned methods can partly eliminate uneven illumination in face image and provide barely satisfactory experimental data for subsequent recognition task, they all have the same deficiency that they do not perform well for the shadow around nose. Fig. 1 shows some representative examples from references [1], [2], [3], [4]. We can see that the shadow around nose, which has negative effect on our further image analysis, is obvious. This is because that the assumption of illumination changing slowly and regularly in human face image does not apply to the sudden change of illumination from non-shadow region to shadow region. How to eliminate the occlusive shadow in face images is still an unsolved problem, especially for a single 2D face image. Thus, on the basis of our previous research [5], we propose a novel method which can effectively handle the problems due to uneven illumination, especially the shadow around nose.

The rest of our paper is organized as follows. In Section 2, the framework and details of our proposed illumination compensation method for a single 2D face image are given. Section 3 consists of experimental results and analysis. Finally, conclusions are offered in Section 4.