Face recognition under arbitrary illumination using illuminated exemplars

Abstract

Recently, the importance of face recognition has been increasingly emphasized since popular CCD cameras are distributed to various applications. However, facial images are dramatically changed by lighting variations, so that facial appearance changes caused serious performance degradation in face recognition. Many researchers have tried to overcome these illumination problems using diverse approaches, which have required a multiple registered images per person or the prior knowledge of lighting conditions. In this paper, we propose a new method for face recognition under arbitrary lighting conditions, given only a single registered image and training data under unknown illuminations. Our proposed method is based on the illuminated exemplars which are synthesized from photometric stereo images of training data. The linear combination of illuminated exemplars can represent the new face and the weighted coefficients of those illuminated exemplars are used as identity signature. We make experiments for verifying our approach and compare it with two traditional approaches. As a result, higher recognition rates are reported in these experiments using the illumination subset of Max-Planck Institute face database and Korean face database.

Introduction

Lighting is the most significant factor affecting the appearance of an object. Changes in a person's appearance induced by illumination are larger than differences in appearance of individuals. Person identification from a facial image across lighting variations is still the most challenging problem in face recognition. Fig. 1 shows the 10 illuminated images of the same individual under varying lighting conditions from Yale database [1]. In the past few years, many methods have been proposed to solve this problem with improvements in recognition. Early works in illumination-invariant face recognition focused on image representations that are mostly insensitive to changes under various lighting [2]. Various image representations are compared by measuring distances on a controlled face database. Edge map, second derivatives and 2D Gabor filters are examples of the image representations used. However, these kinds of approaches have some drawbacks. First, the different representations of image can be only extracted once they overcome some degree of illumination variations. Second, features for the person's identity are weakened whereas the illumination-invariant features are extracted.

A photometric stereo method is an approach based on the low dimensionality of the image space. The images of one object with a Lambertian surface taken from a fixed viewpoint and varying illuminations lie in a linear subspace. We can classify the new probe image by checking if it lies in the linear subspace of the registered gallery images. These gallery images are composed of at least three images of the same person under different illuminations. The main restriction in these approaches is that multiple registered images of the same person are required. Since it recognizes the new image by checking that it is spanned in a linear subspace of the multiple gallery images, it cannot handle the new images of a different person which is not included in the gallery set.

To solve the necessity of multiple gallery images, the bilinear analysis approach is proposed [3]. It applies singular value decomposition (SVD) to a variety of vision problems including identity and lighting on a collection of objects in the same class. For bilinear analysis, photometric stereo images of different people under the same set of illuminations are always required. A method based on quotient image was introduced; this method uses the basic idea of bilinear analysis [4]. Under the assumption of a Lambertian surface without shadow, quotient image is created, deduced from the ratio between a probe image and the linear combination of photometric stereo images. The quotient image should be invariant to varying illumination conditions of the input image. This method can synthesize an image under unknown lighting condition and recognize a new face by computing its quotient image. However, the assumption of a fixed shape is not valid for real faces and the quality of the quotient image is influenced by training data. A poor quality in the quotient image results when the training data is unknown or not controlled. The generalized photometric stereo method in the bilinear analysis approach has been proposed with a rank constraint on the product of albedo and surface normal [5]. The observation matrix can be factorized by a rank constraint. The main limitation of these bilinear analysis methods is that prior knowledge of the images like the lighting direction of training data is required.

Unlike the methods described above, Blanz and Vetter use 3D morphable models of a human head [6]. The 3D model is created using a database collected by Cyberware laser scanner. Both geometry and texture are linearly spanned by the training ensemble. This approach enables them to handle illumination, pose and expression variations. But it requires the external 3D model and high computational cost. For illumination-robust face recognition, we have to solve the following problem: Given a single image of a face under the arbitrary illumination, how can the same faces under different illuminations be recognized?

In this paper, we propose a new approach for solving this problem based on illuminated exemplars. The illuminated exemplars are synthesized from photometric stereo images of each person from training data and the new probe image can be represented by a linear combination of these illuminated exemplars. The weighted coefficients are estimated in this representation and can be used as the illumination-invariant identity signature. For face recognition, our proposed method has several distinct advantages over the previously proposed methods. First, the information regarding the lighting condition of training data is not required. We can synthesize the illuminated exemplars under other illumination which is not included in the training data. Second, we can perform recognition with only one gallery image by using linear analysis of illuminated exemplars in the same class. Third, the coefficients of illuminated exemplars are used as the identity signature for face recognition across variation in lighting, which results in high recognition rates.

This paper is organized as follows. Section 2 provides the results of a literature search for past studies related to this work. In Section 3, we describe how the illuminated exemplars are synthesized, how they are analyzed and what can be used as a signature identity for face recognition, and then we propose the face recognition method using the illuminated exemplars. Section 4 gives our experimental results on Max-Planck Institute face database (MPI DB) and Korean face database (KFDB). Finally, we make a conclusion regarding our approach and suggest some areas of further research in Section 5.