Local appearance based face recognition method using block based steerable pyramid transform

Abstract

In this paper, an efficient local appearance feature extraction method based on Steerable Pyramid (S-P) wavelet transform is proposed for face recognition. Local information is extracted by computing the statistics of each sub-block obtained by dividing S-P sub-bands. The obtained local features of each sub-band are combined at the feature and decision level to enhance face recognition performance. The purpose of this paper is to explore the usefulness of S-P as feature extraction method for face recognition. The proposed approach is compared with some related feature extraction methods such as principal component analysis (PCA), as well as linear discriminant analysis LDA and boosted LDA. Different multi-resolution transforms, wavelet (DWT), gabor, curvelet and contourlet, are also compared against the block-based S-P method. Experimental results on ORL, Yale, Essex and FERET face databases convince us that the proposed method provides a better representation of the class information, and obtains much higher recognition accuracies in real-world situations including changes in pose, expression and illumination.

Introduction

Among face recognition methods, the most popular are holistic appearance-based approaches such as PCA [13], LDA [46], ICA [4] and review of these methods has been presented in [21]. Subspace analysis based methods has been proposed in [47], [48], [49] in order to give an effective feature extraction in high dimensional space. These methods outperform holistic methods in recognition accuracy. Recently, there are more and more attempts to develop face recognition systems based on local features. The approach of analyzing faces locally is believed to outperform the holistic appearance-based approaches, where a local change affects only the corresponding part of the representation and does not modify the representation vector as a whole [32]. These approaches exhibit good performance and robustness in controlled environments but still do not perform well in many real-world situations, due to variations in pose, lighting and expression. In order to address this issue, many researchers propose to deploy a pre-processing step in order to capture more discriminant features for use in the recognition step, such as local binary patterns (LBP) and local discrete cosine transform [33], [34]. In [41] the authors propose to utilize the total variation (TV) mode to factorize an image in order to overcome the illumination limitation and to preserve the edge information of images. However, the TV model is able to only process images in certain scale. In addition, the TV model is an iterative type approach, thus the computational expense is very high. More recently, interest has grown in using multi-resolution methods where multiple evidences (sub-bands) from the same face are obtained allowing extraction of less sensitive features to intrinsic deformations due to expression or due to extrinsic factors, like illumination. These transforms have more edge-preserving ability than TV model in low frequency illumination fields [40]. Multi-resolution methods have been successfully used in many challenging pattern recognition applications including character recognition [2] and face recognition [1], [3], [26].

Among multi-resolution methods for face recognition, the most popular are discrete wavelet transform (DWT) [22], [23], [24], [25], [50] and Gabor wavelets [12]. These methods have proved to be very successful to capture more discriminant features of face images leading to higher performance and robustness against various challenging conditions. However, wavelet-based features are not suitable for face recognition in uncontrolled environments since images do not always exhibit isotropic scaling (horizontal, vertical and diagonal). In [40], authors treated a particular problem of illumination invariance. Since illumination is represented by a convolution, it can be avoided in log-domain by wavelet denoising technique. Their method claimed to be robust in different illumination conditions leading to the best results in Yale B face database. Others have shown that Gabor filters can attain good results in many face recognition applications. However, the use of Gabor filters dramatically increase the computational cost of the face recognition method, requiring that each kernel is convolved with the input image [12]. Contourlet [9], curvelet transforms [11], and steerable pyramid are another multi-resolution transforms similar to the two-dimensional DWT, but with interesting translation and rotation-invariance properties [5]. The steerable pyramid (S-P) is a linear multi-scale, multi-orientation image decomposition which has been developed to overcome the wavelet limitations. Though steerable pyramids provide more scale and orientation than wavelets. The curvelet transform captures curves instead of points as in S-P transform in the continuous domain. Contourlets are an extension of curvelets, which can be approximated in the discrete domain. Contourlets, however, are defined and derived in the discrete domain from the beginning. They both allow for directionality and anisotropy [42].

Applications of Contourlet to face recognition have been investigated in work presented by Boukabou et al. [3]. Authors propose to employ contourlet with PCA in order to extract discriminant features and to obtain higher recognition rates. They have evaluated the proposed method on two different databases (Yale and FERET Database) and stated that the contourlet transform outperforms the original PCA method. More experiments have to be performed on large database and many comparisons against well established existing techniques must be done to assess this result. Mandal et al. [1] propose curvelet based face recognition system by fusing results from multiple SVM classifiers trained with curvelets coefficients from images having different gray scale resolutions (2, 4 and 8 bits). However, this algorithm is computationally expensive since it requires taking the curvelet transform of the original image and its quantized representations. In [26], [27], curvelet transform is introduced in conjunction with different dimensionality reduction tools. These techniques appear to be robust to the changes in facial expression as they show good results for the Essex and the ORL database, but still do not perform well in YALE database that contains images with great variations in illumination and facial expression.

According to previous review on local based approaches, which have proven to be robust to most face recognition challenges comparing to global based approaches, we have proposed a face recognition method based on local presentation of curvelet transform [28]. Curvelet transform is applied to the face image and each of the resulting sub-bands is partitioned into a set of equally sized blocks in a non-overlapping way. Then the statistical measures (mean, variance and entropy) of the energy distribution of the curvelet coefficients for each block in each sub-band at each decomposition level is used to construct the feature vector. Then, we used curvelet transform as an improvement tool of LDA in [45]. The proposed methods have been evaluated on Yale, ORL and FERET databases and have shown better recognition accuracy in comparison to holistic approaches. However, local-based approaches cannot be applied to this technique because of the small sub-band size.

To solve all these mentioned problems, the steerable pyramid can be employed to produce any number of orientation bands. In addition, it conserves the same image resolution in the first scale level which is more adequate for local appearance based approaches. Several studies have investigated the discriminating power of steerable pyramid-based features in various applications including: image denoising [10], textures classification [12], image processing [30], [5] and face hallucination [29]. In [44], the S-P method has been proposed in conjunction with LBP of each sub-band in order to extract a local information of face images. This work gave promised results to investigate extensively the use of S-P transform both in global and local appearance, and feature/score fusion which is the subject of the present paper.

In this paper, we present a face recognition approach based on steerable pyramid decomposition. Following our previous study [43], the main contribution of this paper is to fully investigate the usefulness of steerable pyramids transform in a face recognition framework. Each face image is described by a subset of band filtered images containing steerable pyramid coefficients which characterize the face textures. We divide the S-P sub-bands into small sub-blocks, from which we extract compact and meaningful feature vectors using mean, variance and entropy. We conceive an experiment framework specifically to investigate the improvement in robustness against illumination and facial expression changes. We discuss the important problem of fusing the local observations that utilizes multiple sub-bands. Then, we investigate fusion schemes both at the feature and decision levels. Finally, we show how an efficient and reliable probabilistic metric can be used in order to classify the face feature vectors into person classes. Experimental results are presented using images from the FERET, ORL, ESSEX, YALE and YALE B databases. The efficiency of our approach is firstly analyzed by comparing the results with those obtained using multi-resolution methods such as wavelet, Gabor, contourlet and curvelet. Secondly it is compared to the best given results in the literature and has shown better recognition performance.

The remainder of the paper is organized as follows. In Section 2, face feature extraction based steerable pyramid transform is introduced. Block based S-P face identification proposed method is given in Section 3. In Section 4, three fusion schemes used in the study are explained, namely, data fusion, feature fusion, and decision fusion. Experimental results are presented and discussed in Section 5. Finally, in Section 6, conclusions and future recommendations are given.