Extended local binary patterns for face recognition

Highlights

• A set of six LBP-like features derived from local intensities and differences.

• A labeled dominant pattern scheme is proposed to learn salient information.

• Utilizing whitened PCA to produce more compact, robust and discriminative features.

• Fused WPCA features improves the accuracy and robustness of face recognition.

• Proposed face recognition system is highly robust to illumination variations.

Abstract

This paper presents a simple and novel, yet highly effective approach for robust face recognition. Using LBP-like descriptors based on local accumulated pixel differences – Angular Differences and Radial Differences, the local differences were decomposed into complementary components of signs and magnitudes. Based on these descriptors we developed labeled dominant patterns where the most frequently occurring patterns and their labels were learned to capture discriminative textural information. Six histogram features were obtained from each given face image by concatenating spatial histograms extracted from non-overlapping subregions. A whitened PCA technique was used for dimensionality reduction to produce more compact, robust and discriminative features, which were then fused using the nearest neighbor classifier, with Euclidean distance as the similarity measure.

We evaluated the effectiveness of the proposed method on the Extended Yale B, the large-scale FERET, and CAS-PEAL-R1 databases, and found that that the proposed method impressively outperforms other well-known systems with a recognition rate of 74.6% on the CAS-PEAL-R1 lighting probe set.

Introduction

Face recognition, as one of the most successful applications of image analysis and understanding, has received considerable attention in the past decades due to its challenging nature and vast range of applications [18], [42]. In recent years this field progressed significantly and a number of face recognition and modeling systems have been developed [18]. Even though current face recognition systems have reached a certain level of maturity, the performance of these systems in uncontrolled environments is still far from the capability of the human vision system [18], [42]. In other words, there are still many challenges associated with accurate and robust face recognition in regards to computer vision and pattern recognition, especially under unconstrained environments. Therefore, this remains a stimulating field for further research.

As a classical pattern recognition problem, face recognition primarily consists of two critical subproblems: feature extraction and classifier designation, both of which have been the subject of significant study. Generally, facial feature description plays a relatively more important role if poor features are used, and even the best classifier will fail to achieve good recognition results. Numerous facial feature sets were presented with excellent surveys in reports by Zhao et al. [42] and Li and Jain [18]. Nevertheless, designing useful face descriptors remains a great challenge when faced with three competing goals: computational efficiency, effective discrimination, and robustness to intra-person variations (including changes in illumination, pose, expression, age, blur, and occlusion).

Most facial extraction approaches can be categorized as utilizing either holistic features or local features. The holistic approach uses the entire face region to construct a subspace representing a face image; Influential examples includes Principle Component Analysis (PCA) [36], Linear Discriminant Analysis (LDA) [3], Locally Linear Embedding (LLE) [33] and Locally Preserving Projections (LPP) [11]. In contrast, the local features approach involves local features first being extracted from a subregion of a face and then classified by combining and comparing with corresponding local statistics. Holistic features are unable to capture local variations in face appearance, and are more sensitive to variations in illumination, expression and occlusions [18]. Local feature based approaches are advantageous in that distributions of face images in local feature space are less affected by changes in facial appearance. As a result, local feature based face recognition approaches have been widely studied in recent years.

Among local feature approaches, local binary patterns (LBP) have emerged as one of the most prominent face analysis methods since the pioneering work by Ahonen et al. [1], and have attracted increasing attention due to their distinguished advantages: (1) ease of implementation; (2) invariance to monotonic illumination changes; and (3) low computational complexity. However, the original LBP method still has multiple limitations: (i) production of long histograms; (ii) capture of only very local texture structure and not long range information; (iii) limited discriminative capability based purely on local binarized differences; and (iv) limited noise robustness. On the basis of these issues, many LBP variants [12] have been proposed to improve face recognition performance.

Our research is motivated by recent work on texture classification in [24], where four LBP-like descriptors – Center Intensity based LBP (CI-LBP), Neighborhood Intensity based LBP (NI-LBP), Radial Difference based LBP (RD-LBP) and Angular Difference based LBP (AD-LBP) – were proposed, along with multiscale joint histogram features, which were found to be highly effective to rotation invariant texture classification. We expanded this research [24] to address the face identification problem by proposing a framework and more generalized formulation of the local intensities and differences features. Specifically, the major contributions of our work are summarized as follows:

• We proposed a new family of LBP-like descriptors based on local accumulated pixel differences: Angular Differences (AD) and Radial Differences (RD). The descriptors presented advantages of efficiency, complementarity to LBP, robustness, and the encoding of both microstructures and macrostructures, resulting in a more complete image representation. The extraction of the proposed descriptors did not require any training, and thus this approach showed better generalizability compared with popular learning methods, whose performance is degraded if the distribution of the testing sample varies significantly from that of the training set.

• Using the proposed descriptors, we found that the properties of the original uniform patterns introduced by Ojala et al. [30] did not hold true, and therefore we suggest the use of full patterns. We also proposed a labeled dominant pattern (LDP) scheme, which learns a set of dominant patterns (the most frequently occurring patterns from a set of training images) to capture discriminative textural information, but with lower dimensionality than the full pattern approach.

• Extensive experiments were conducted on the Extended Yale B, the FERET and the large-scale CAS-PEAL-R1 databases. Our approach proved to be highly robust to illumination changes, as evident by our recognition rate of 74.6% on the CAS-PEAL-R1 lighting probe set. This is, to the best of our knowledge, the highest score yet achieved on this data set.

A preliminary version of this work appeared in [22].