Learning the best subset of local features for face recognition

Abstract

We propose a novel, local feature-based face representation method based on two-stage subset selection where the first stage finds the informative regions and the second stage finds the discriminative features in those locations. The key motivation is to learn the most discriminative regions of a human face and the features in there for person identification, instead of assuming a priori any regions of saliency. We use the subset selection-based formulation and compare three variants of feature selection and genetic algorithms for this purpose. Experiments on frontal face images taken from the FERET dataset confirm the advantage of the proposed approach in terms of high accuracy and significantly reduced dimensionality.

Introduction

Face recognition has proved to be a difficult problem in computer vision. The main reason for this is that intra-personal variations caused by facial expressions, view point changes, and illumination variations are significant when compared to inter-personal variations. Many researchers have therefore focused on face representation techniques that are invariant to some of these variations [1], [2]. These can be grouped into two, as holistic and local feature-based: In the first type, faces are represented as a whole and statistical techniques are used to extract features from faces [3], [4], [5], [6]. The second type depends on the localization of salient facial features such as the eyes and the mouth [7], [8], [9]. There are also hybrid approaches which incorporate complementary knowledge from both [10]. Our work in this paper belongs to the second type, with the distinction that the salient local regions are not predicted but are learned from data.

The main idea in a feature-based face representation scheme is the extraction and analysis of local facial features. Salient facial features are first found and then used to code a face. Coding is generally carried out using geometric relationships between these points and extracting local image descriptions around these points. Among different alternatives, 2D Gabor-like filters are found to be very suitable as local descriptors because of their robustness against translation, rotation, and scaling [7], [11], [12]. 2D Gabor wavelets are selective to different orientations and spatial frequencies. Typically, features extracted by 2D Gabor wavelets have a very large dimensionality. It is therefore essential to analyze the contribution of each feature component to the recognition performance. Important parameters of 2D Gabor wavelets are: (1) spatial location of the kernel in the image; (2) kernel orientation; and (3) spatial kernel frequency.

Several studies have concentrated on examining the importance of the Gabor kernel parameters for face analysis. These include: the weighting of Gabor kernel-based features using the simplex algorithm for face recognition [13], the extraction of facial subgraph for head pose estimation [14], the analysis of Gabor kernels using univariate statistical techniques for discriminative region finding [15], the weighting of elastic graph nodes using quadratic optimization for authentication [8], the use of principal component analysis (PCA) to determine the importance of Gabor features [16], boosting Gabor features [17] and Gabor frequency/orientation selection using genetic algorithms [18].

In almost all previous studies, we see two fundamental assumptions: first, the contribution of each feature dimension is analyzed independently of others (independence assumption); and second, Gabor kernel placement over the face region is strongly affected by prior knowledge (saliency assumption). Placing the kernel at visually salient facial points, e.g., eyes, mouth, etc. is one of the frequently used methods. The first assumption of independence of features is not valid, and one should incorporate more complex methodologies to analyze the relationship between the features. Moreover, the effectiveness of the fiducial points should also be studied systematically, and a better solution would be to learn these locations from given training data for a given task. In our previous work, we have analyzed topographically important facial locations for both pose estimation and identity recognition [19], and used feature selection methods to extract optimal local image descriptor parameters for frontal face recognition [20]. We have also used such features to calculate bottom-up saliency in a selective attention-based face recognizer [21].

In this work, our aim is to relax the independence and saliency assumptions for face recognition by reformulating the optimal Gabor basis extraction problem as a feature subset selection problem. Doing this, we allow our approach to detect more complex relationships and correlations between feature dimensions, thus extracting a near-optimal Gabor basis. For this purpose, we have devised a two-stage subset selection mechanism: In the first stage, a genetic algorithm is used to find the most informative facial locations. In the second stage, a floating search method is used to learn the individual parameters, that is, frequency and orientation, of Gabor wavelet-based local descriptors.

The remainder of this paper is organized as follows: Section 2 describes the proposed approach and experimental results, including a sensitivity analysis, are presented in Section 3. We conclude and discuss future research directions in Section 4.