Uniform Local Derivative Patterns and Their Application in Face Recognition

Abstract

In recent years, local feature descriptors have received more and more attention due to their effectiveness in the field of face recognition. Local Derivative Patterns (LDPs) for local feature descriptions attract researchers’ great interest. However, an LDP produces 2^p different patterns for p neighbors through the transition of LDPs for an image, which lead to high dimension features for image analysis. In this paper, LDPs are expanded to Uniform Local Derivative Patterns (ULDPs) that have the same binary encoding way as LDPs but different transition patterns by introducing uniform patterns. A uniform pattern is the one that contains at most two bitwise transitions from 0 to 1 or vice versa when the binary bit is circular. Then, the number of the transition patterns is reduced from 2^p to p(p−1)+3 for p neighbors, e.g., 256 to 59 for p = 8. For face recognition, the histogram features are combined together in four directions, and both non-preprocessed and preprocessed images are used to evaluate the performance of the proposed ULDPs method. Extensive experimental results on three publicly available face databases show that the proposed ULDPs approach has better recognition performance than that obtained by using the LDPs method.

1 Introduction

In the past decades, a great deal of attention from the academic and industrial communities has been focused on the face recognition due to its wide applications to information security, network security, and personal identification. There are many well-known methods for face recognition, including Fisherface [1], Eigenface [2, 3], Linear Discriminant Analysis (LDA) [4], and kernel methods [5], which have been extensively studied and proved successful. However, these methods require a great large number of trained images, and actually the images available for training are few or even just one. In addition, they only extract holistic features of the face images since their local features are neglected, while the local features are essential information for face recognition.

Recently, non-statistical local feature descriptors called Local Binary Patterns (LBPs) have become more and more popular in the field of face recognition. Initially, LBPs are designed by Ojala et al. [6] for texture feature classification due to its invariance with monotonic gray level changes. Ahonen et al. [7, 8] first apply LBPs to the area of face recognition. After that, LBP-based face recognition methods gain much more attention due to their robustness in illumination and expression. Naturally, the methods based on LBPs are developed with several variations for improved performance in face recognition.

Since the threshold of LBPs is defined exactly as the value of the central pixel, LBPs are sensitive to noise in uniform regions. To address this problem, Tan and Triggs [9] extend LBPs to Local Ternary Patterns (LTPs) with a 3-value code, in which the threshold is set to be slightly larger than zero. Ahonen et al. [10] propose Soft Histogram for Local Binary Patterns (SLBPs) that contribute one pixel typically to more than one bin meanwhile the sum of contributions of the pixel to all bins is always 1. An SLBP also suffers from the same problem that it is difficult to set a suitable threshold, as the case in LTPs. Furthermore, Heikkila et al. [11] propose Center-Symmetric Local Binary Patterns (CSLBPs) by comparing center-symmetric pairs of pixels around a central pixel instead of comparing each pixel with the central pixel. Liu et al. [12] present a new face recognition method, three-level face features, by using CSLBPs three times.

More ways to improve LBPs are to combine LBPs with other approaches [13, 14]. For example, the Multi-resolution Histograms of Local Variation Patterns (MHLVPs) proposed by Zhang et al. [15] represent face images as the concatenation of the local spatial histogram of local variation patterns computed from the multi-resolution Gabor features. Zhang et al. [16] also report a face recognition method, based on Local Gabor Binary Pattern Histogram Sequence (LGBPHS), which extracts Gabor and LBPs features of face images. The methods mentioned above suffer from a common problem: they capture non-directional first-order features only.

To find more detailed information, Zhang et al. [17] expand LBPs to Local Derivative Patterns (LDPs) and high-order LDPs that encode directional pattern features at four local derivative variations. Thus, they contain more spatial information than traditional LBPs. Their work also designs the Gabor Local Derivative Patterns (GLDPs) that are proved successful in illumination variation face recognition by combining Gabor Wavelet with LDPs. All the methods mentioned above usually cause a high dimensionality that to a large extent limits their applications.

This paper expands LDPs to Uniform Local Derivative Patterns (ULDPs) for face recognition under different appearance variations. In this framework, transition patterns of LDPs are divided into two categories: uniform and non-uniform ones. Therefore, the proposed method produces p(p−1)+3 patterns for p neighbors only, much less than 2^p patterns in normal LDPs such that the size of the features dimensionality and the computational costs are greatly reduced. Different from the existing statistical methods, a ULDP directly extracts local features from the input images or feature images, as done in an LDP, which is why a ULDP has more robustness to unpredictable distribution images than the statistical methods. Similar to other non-statistical methods, a ULDP requires one image for per person for training only. Moreover, preprocessing methods are applied to combine with ULDPs for face recognition in this paper, which effectively enhances the performance of ULDPs in illumination variations.

The remainder of the paper is organized as follows. Section 2 briefly reviews LDPs method and discusses ULDPs in detail. Section 3 proposes the application of the proposed ULDPs method in face recognition. Furthermore, preprocessing methods with γ correction and the Different of Gaussian (DoG) filter are described in this section. Section 4 reports experimental results to evaluate the performance of the methods on face recognition. Finally, Section 5 concludes this paper.

2 Uniform Local Derivative Patterns

In this section, we first briefly review the basic idea of Local Derivative Patterns, and then discuss in detail the proposed method, i.e., Uniform Local Derivative Patterns, in which the number of different patterns is reduced from 2^p to p(p−1)+3 for p neighbors.

2.1 Local Derivative Patterns

LDPs are proposed for face recognition through a high order local pattern descriptor. It is a general framework to encode directive pattern features from local derivative variation. The most notable characteristic of this method is that an LDP captures local features in four directions: 0°, 45°, 90°, and 135°, and concatenates the transition result as a 32-bit binary string. The first-order derivative for a pixel in 0°, 45°, 90°, or 135° direction, denoted as $I_{\alpha}^{\prime }(Z)$, is calculated in a 3 × 3 area around it, where α = 0°, 45°, 90°, and 135°. Let Z ₀ be the central point of a local region I (Z), and Z _i ’s are the neighbors of Z ₀ , where i = 1, 2, ..., 8, as shown in Fig. 1.

Figure 1

Example of LDPs representation.

Hence the first-order derivatives at Z = Z ₀ are as follows [17]:

$$ I_{{{0^{\circ}}}}^{\prime}\left( {{Z_0}} \right)=I\left( {{Z_0}} \right)-I\left( {{Z_4}} \right) $$

(1)

$$ I_{{{45^{^{\circ}}}}}^{\prime}\left( {{Z_0}} \right)=I\left( {{Z_0}} \right)-I\left( {{Z_3}} \right) $$

(2)

$$ I_{{{90^{^{\circ}}}}}^{\prime}\left( {{Z_0}} \right)=I\left( {{Z_0}} \right)-I\left( {{Z_2}} \right) $$

(3)

$$ I_{{{135^{^{\circ}}}}}^{\prime}\left( {{Z_0}} \right)=I\left( {{Z_0}} \right)-I\left( {{Z_1}} \right) $$

(4)

The second-order directional LDPs derived from the first-order derivative, in α direction at Z = Z ₀ , can be defined as:

$$ LDP_{\alpha}^2\left( {{Z_0}} \right)=\left\{ {f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_1}} \right)} \right),f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_2}} \right)} \right),\cdots, f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_8}} \right)} \right)} \right\} $$

(5)

where f(∙,∙) is a binary coding function, determining the types of local pattern transitions. It encodes the co-occurrence of two derivative directions at different neighboring pixels, as shown in [17]:

$$ \begin{array}{*{20}c} {f\left( {I_{\alpha}^{\prime}\left( {{Z_i}} \right),I_{\alpha}^{\prime}\left( {{Z_j}} \right)} \right)=\left\{ \begin{array}{*{20}c} 0, if\;I_{\alpha}^{\prime}\left( {{Z_i}} \right)\cdot I_{\alpha}^{\prime}\left( {{Z_j}} \right)>0 \hfill \\ 1, if\;I_{\alpha}^{\prime}\left( {{Z_i}} \right)\cdot I_{\alpha}^{\prime}\left( {{Z_j}} \right)\leqslant 0 \hfill \\\end{array} \right.} \hfill \\ {\alpha =0^{\circ},45^{\circ},90^{\circ},\;\mathrm{and}\;135^{\circ},\;i=1,\;2,\cdots, 8.} \hfill \\ \end{array} $$

(6)

The second-order directional LDPs can also be defined as:

$$ LDP_{\alpha}^2\left( {{Z_0}} \right)=\sum\limits_{n=1}^8 {f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_n}} \right)} \right)\cdot {2^{n-1 }}} $$

(7)

Finally the second-order LDPs, denoted by LDP ²(Z), is defined as the concatenation of the four second-order directional LDPs, complied with Eq. (8):

$$ LD{P^2}(Z)=\left\{ {LDP_{\alpha}^{\prime }(Z)\left| {\alpha ={0^{^{\circ}}},{45^{^{\circ}}},{90^{^{\circ}}},and\;{135^{^{\circ}}}} \right.} \right\} $$

(8)

A computational example of LDPs is illustrated in Fig. 2. To compute an LDP in α = 0° direction at the central pixels, we calculate the first-order derivatives by Eq. (1). Then, we obtain the second-order directional LDPs in 0° direction by Eq. (5) or Eq. (7), i.e., $LDP_{{{0^{^{\circ}}}}}^2=11000000$, an 8-bit string, which is denoted by transition pattern 3 (the left-most bit is the low bit). Similarly, the calculated second-order LDPs in 45°, 90°, and 135° are “00001100” (transition pattern 48), “00011110” (transition pattern 120), and “00001100” (transition pattern 48), respectively. Finally, all the second-LDPs binary results are concatenated to gain the LDP²(Z), “110000000000110000011111000001100”, a 32-bit string.

Figure 2

Example to obtain the second-order LDPs in α = 0° direction.

2.2 The Theory of ULDPs

An LDP directly encodes the binary result of the second-order derivatives in four directions. For eight neighbors, 256 (2⁸) different local derivative patterns are produced in each direction.

It is most remarkable that some local derivative patterns contain the vast majority of the image features, sometimes over 90% in LDPs, and it is likely to LBPs [18]. These patterns, called uniform ones, have the same specialty in common that contains little spatial transitions. We design uniform local derivative patterns (ULDPs) by introducing a uniform pattern that reduces the number of the transition patterns of LDPs. ULDPs have the same binary encoding way as LDPs, while the final ULDPs transition patterns have a new encoding way that is different from LDPs.

To formally define ULDPs, the transition patterns of LDPs are first divided into two categories: uniform patterns and non-uniform patterns, according to the number of spatial transitions (bitwise 0/1 change) of their patterns. An LDP pattern is said to be uniform if its binary pattern contains at most two bitwise transitions from 0 to 1 or vice versa when the bit is circular, while all the other binary patterns are attributed to non-uniform patterns. For example, 00000000₂ (0 transitions) and 01100000₂ (2 transitions) are uniform whereas 01010000₂ (4 transitions) and 10000010₂ (4 transitions) are non-uniform. For eight neighbors, there are nine types of patterns which are uniformed as shown in Fig. 3, where patterns 11111111₂ and 00000000₂ have zero time transitions, while the others have exactly 2 times transitions when the bit is circular. Each of the nine types of the patterns except for 11111111₂ and 00000000₂, has eight kinds of rotation forms when the bit is circular. Therefore, there are totally 7 × 8 + 2 = 58 different uniform patterns for eight neighbors. Then ULDPs in α direction at Z = Z ₀ for 8 neighbors can be calculated by the following equations:

$$ ULDP=\left\{ {\begin{array}{*{20}c} {\left[ {\sum\limits_{p=1}^8 {f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_p}} \right)} \right){2^{p-1 }}} } \right]} \hfill \\ {-N\left[ {\sum\limits_{p=1}^8 {f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_p}} \right)} \right){2^{p-1 }}} } \right],\;if\;g\left( {LDP} \right)\leqslant 2} \hfill \\ {\quad \quad \quad 58\quad \quad \quad \quad \quad \quad \quad \quad \quad, otherwise} \hfill \\ \end{array}} \right. $$

(9)

$$ \begin{array}{*{20}c} {g\left( {LDP} \right)=\left| {f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_8}} \right)} \right)-f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_1}} \right)} \right)} \right|} \\ {+\sum\limits_{p=1}^7 {\left| {f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_{p+1 }}} \right)} \right)-f\left( {I_{\alpha}^{\prime}\left( {{Z_0}} \right),I_{\alpha}^{\prime}\left( {{Z_p}} \right)} \right)} \right|} } \\ \end{array} $$

(10)

where N(x) is a function to count the total number of the non-uniform patterns, which is less than x, and g(LDP) is used to calculate the number of spatial transitions in which bitwise 0/1 changes of LDPs patterns occur. Equation (10) implies an LDP pattern (an 8-bit string) first moves one bit, and then subtracts the original binary value of LDPs pattern from the resulting pattern after removal. Finally, the number of spatial transitions is obtained by calculating the sum of absolute values of the difference.

Figure 3

Nine uniform binary patterns that can occur in the circularly symmetric neighbor set, where black and white circles correspond to bit values of 0 and 1 in the 8-bit output of the operator, respectively.

It is show in Eq. (9) that a pattern is uniform only if its number of spatial transitions is not more than 2, denoted by g(LDP) ≤2. By the previous descriptions, it is clear that there are exactly p(p-1)+2 uniform patterns numbered from 0 to p(p-1)+1 for p neighbors (i.e., from 0 to 57 for 8 neighbors) and we define all the non-uniform patterns as pattern p(p-1)+2 that is exactly 58 for p = 8. An example of ULDPs computation for eight neighbors is illustrated in Fig. 2. Noted that the 8-bit binary result in Fig. 2, i.e., “11000000”, contains two times bitwise transitions. Consequently, it is a uniform pattern and its value of ULDPs pattern is 3 by Eq. (9). The three results, “00001100”, “000111110”, and “00001100”, are uniform patterns, corresponding to the values of ULDPs patterns: 17, 25, and 17, respectively, which are different from LDPs patterns calculated previously.

A ULDP produces p(p-1)+3 different patterns that consist of p(p-1)+2 uniform patterns and one non-uniform pattern. Compared with 2^p different patterns produced by LDPs, the quantity is reduced greatly. As well known, the final image features used for image analysis are the histogram features accumulated by feature descriptors of an input image. The reason that why ULDPs histogram features provide better discrimination compared against the LDPs histogram features comes down to the difference in their statistical properties. The relative proportion of the non-uniform patterns of all patterns accumulated into a histogram is so small that their probabilities cannot be estimated reliably. In addition, the uniform patterns reduce the influence of the high frequency noise that can be defined as non-uniform patterns. This means that it can extract representative features of the face images with the feature vectors of the uniform and non-uniform patterns. As shown in Fig. 4, the ULDPs produce a 59-dimensional feature vector less than the 256-dimensional feature vector produced by LDPs. Furthermore, we can also see that there are a large number of zeros between patterns 100 and 125 in LDPs histogram, and no such sparse problem exists in ULDPs histogram.

Figure 4

Differences of the histogram between LDPs (middle) and ULDPs (right).

3 ULDPs for Face Recognition

This section discusses the application of ULDPs to face recognition. The overall framework of the proposed method used for histogram feature extraction is illustrated in Fig. 5. In this method, the ULDPs-based face recognition is modeled by the following procedures: (1) normalizing an input face image according to the location of the eyes and preprocessing it by using γ correction and the Different of Gaussian (DoG) filter; (2) labeling the image by the ULDPs descriptor in four directions; (3) dividing the transition results of each image into several rectangular sub-regions and calculating histogram features in each region; and (4) establishing the face model with the concatenation of all the histograms sequence and recognizing the face image by the histogram intersection and the nearest neighbor classifier. In what follows, we discuss these procedures in detail.

Figure 5

The framework of ULDPs-based histograme feature extraction.

3.1 Image Normalization and Preprocessing

For face recognition, images from a database are in a generally normalized size by cropping images according to the location of eyes to eliminate the impact of the background. Moreover, preprocessing methods are often used to decrease noise factors, such as γ correction and the Different of Gaussian (DoG) filter, which are believed to be the preferable methods for this problem.

γ correction is a nonlinear gray-level transformation that replaces gray-level I with I ^γ (for γ >0) or log(I) (for γ = 0), where γ is a user-defined parameter ranging from 0 to 1, reflecting the performance of nonlinear transformation. γ correction increases the contrast of images by enhancing the local dynamic ranges in the shadowed regions of the image while compressing them in bright regions. It is clear that the only factor in γ correction is the value of γ that is widely used in the range [0, 0.5].

To correct the uneven illumination, an image is first changed into the frequency domain by two-dimensional Fourier transform. Then it is filtered by the Different of Gaussian (DoG) filter to remove redundant information, while preserving the useful low-frequency information. The DoG filter is a common filter in the field of computer vision and image analysis, which consists of two Gaussian functions with different standard deviations [19]:

$$ G(s)={A_1}{e^{{-\frac{{{s^2}}}{{2\sigma_1^2}}}}}-{A_2}{e^{{-\frac{{{s^2}}}{{2\sigma_2^2}}}}}{A_1}\geqslant {A_2} > 0,\;{\sigma_1} > {\sigma_2} $$

(11)

It is obvious that the performance of the DoG filter is principally decided by two parameters: σ ₁ and σ ₂. By a series of experiments, we find that when γ = 0.2, A ₁ = A ₂ = 1, σ ₁ = 2 and σ ₂ = 1, the proposed ULDPs can achieve the best performance. Thus, these values are determined as the default setting.

3.2 Histogram of ULDPs

The purposes to develop the ULDPs-based method in the previous sections are used for face recognition. The images collected under various conditions such as illumination, accessory, and expression are really challenges to face recognition. We model the distribution of ULDPs by spatial histograms [8, 17, 20] because of their robustness to the various conditions. A ULDP captures histogram features in four invariant directions: 0°, 45°, 90°, and 135°. To avoid the loss of too much Micro-local information after ULDPs encoding, an image is divided into several sub-regions. In α direction, ULDPs_α are spatially divided into rectangular sub-regions represented by S ₁ , S ₂ , ..., and S _N , and then the spatial histograms of ULDPs, denoted as HULDP [17], can be extracted as follow:

$$ HULDP\left( {i,\alpha } \right)=\left\{ {{H_{{ULD{P_{\alpha }}}}}\left( {{S_i}} \right)\left| {i=1,\cdots, N;} \right.\;\alpha ={0^{^{\circ}}},{45^{^{\circ}}},{90^{^{\circ}}},and\;{135^{^{\circ}}}} \right\} $$

(12)

where ${H_{{ULD{P_{\alpha }}}}}\left( {{S_i}} \right)$ is the ULDPs histogram features extracted from the local sub-region S _i .

For histogram matching, the histogram intersection is low in computational complexity and simple in operations among existing similarity measurements. Thus, it is used for the matching measurement of the two histograms. Histogram intersection is defined as follows:

$$ {D_{HI }}\left( {H,S} \right)=\frac{{{S_{HI }}}}{{\min \left( {\sum\limits_{i=1}^B {{H_i},\sum\limits_{i=1}^B {{S_i}} } } \right)}} $$

(13)

$$ {S_{HI }}\left( {H,S} \right)=\sum\limits_{i=1}^B {\min \left( {{H_i},{S_i}} \right)} $$

(14)

where ${D_{HI }}\left( {H,S} \right)$ is the histogram intersection statistic result with H = (H ₁ ,H ₂ , ...,H _B )^T and S = (S ₁ ,S ₂ , ...,S _B )^T, and B is the total number of the histogram features. The similarity of the two histograms calculated by Eqs. (13) and (14) is used for the classification with the nearest neighbor classifier.

4 Experimental Results and Analysis

The preceding sections deal with theoretical aspects of the proposed framework. To empirically test the main premises of our work and the effectiveness of the described methodology, we conducted a series of experiments to illustrate the performance of the proposed ULDPs for face recognition that covers various conditions including illumination, expression, pose, and accessory. An extensive set of publicly available face databases, FERET [21], CMU PIE [22], Extended Yale B [23, 24], and CAS-PEAL-R1 [25], were used in the experiments. In what follows, the first subsection conducts experiments on a subset of the widely used the FERET database to test the robustness of ULDPs. Then, we conducted the comparative experiments to test the performance of ULDPs on the CAS-PEAL-R1 database to test ULDPs in expression and accessory. Furthermore, the CMU PIE database with a variety of poses and illuminations was utilized to evaluate the performance of the proposed method. The experiment result on the Extended Yale B database with severe illumination variations was also reported in this work.

Before proceeding to the experiments, we take a quick look at the statistical data of the appearing uniform patterns in ULDPs. In ULDPs, there are 58 uniform patterns among the 256 possible patterns. For the images in the FERET database, the 58 uniform patterns contribute 64.99 %, 73.01 %, 69.72 %, and 73.22 % in the four directions, 0°, 45°, 90°, and 135°, respectively. In the CAS-PEAL-R1 database, they are 81.67 %, 86.53 %, 83.7 %, and 84.65 %, respectively. In the CMU PIE database, they are 73.41 %,79.64 %,69.98 %, and 79.57 %, respectively. In the Extended Yale B database, the proportions of uniform patterns are relatively low: 62.45 %, 68.75 %, 60.87 %, and 68.86 % in the four directions, respectively. It indicates that the uniform patterns occupy the majority of the information in an image, which represents the essential lower frequency information. More to the point, pattern 00000000₂ representing the flat region is the most frequent pattern in the all images.

4.1 Experimental Comparisons on the FERET Database

The comparative experiments between LDPs and ULDPs were first conducted on the FERET face database that is widely used to evaluate face recognition algorithms. The FERET database consists of a total of 14,051 gray-scale images representing 1,199 individuals. The images contain variations in lighting, facial expressions, pose angele, etc. In this work, only frontal faces are considered. These facial images can be divided into five sets as follows:

• fa set, used as a gallery set, contains frontal images of 1,196 people.

• fb set (1,195 images). The subjects were asked for an alternative facial expression than in the fa photograph.

• fc set (194 images). The photos were taken under different lighting conditions.

• dup I set (722 images). The photos were taken later in time between 1 min to 1031 days.

• dup II set (234 images). This is a subset of the dup I set containing those images that were taken at least after 18 months after the corresponding gallery image.

To ensure the reliabilty of the test, the publicly available CSU face identification evaluation system [26] was utilized to normalize the images. All the images were normalized and cropped to 150 × 130 pixels by the coordinates of the eyes. Figure 6 shows an example of a normalized FERET image. In this work, fa containing 1,196 frontal images of 1,196 subjects was used as a gellery set, while fb, fc, dup I, and dup II were used as probe sets.

Figure 6

Example of a normalized FERET image.

The first experiment was designed to explore the influence of the sub-region number on recognition accuracy of the normalized images. In this experiment, images were divided into a variable number of regions, from 4 × 4 to 9 × 9, and the obtained histograms from each region were concatenated to obtain the descriptor. Figure 7 shows that recognition accuracy on all four subsets of the FERET database is improved when the sub-region number increases from 4 × 4 to 8 × 8, while fb and dup II subsets decrease when the number reaches to 9 × 9. Moreover, the average curve of recognition accuracy becomes flatter when the sub-region number increases from 8 × 8 to 9 × 9. Therefore, we chose 8 × 8 as the default size in the experiment on the FERET database.

Figure 7

Comparative face recognition rates of ULDPs on the FERET database with different sub-regions.

To observe the performance of ULDPs under different conditions, the next experiments were conducted on the individual probe sets. Experimental results in Fig. 8(a) demonstrate that ULDPs work better than the other methods: LBPs, ULBPs, and LDPs in average. It is clear that ULDPs achieve much higher recognition accuracy compared against LDPs.

Figure 8

Comparative identification accuracies on the FERET database. a Results on the non-preprocessed images. b Results on the preprocessed images.

The experiments were also designed to evaluate the effectiveness of ULDPs on the preprocessed images with the DoG filter and γ correction. The recognition rates obtained with different descriptors on the preprocessed images are shown in Fig. 8(b). It should be noted that PULDPs (ULDPs with the preprocessing method) perform better than PLBPs (LBPs with the preprocessing method), PULBPs (ULBPs with the preprocessing method) and PLDPs (LDPs with the preprocessing method) in average. Specially, on the fc subset, when gallery and probe images were taken under different illumination, PULDPs give the recognition accuracy as high as 94.85 %. The results on the fc subset show that the preprocessing method is powerful for ULDPs in illumination variation. It is worth mentioning that PULDPs achieve 10.66 % and 13.51 % performance improvements over LBPs in dup I and dup II subsets, in which LBPs give the best result with the original images. It means that PULDPs are more robust to aging than that of ULDPs.

4.2 CAS-PEAL-R1 Database - Expression and Accessory Variant

The CAS-PEAL-R1 face database contains 99,594 images among 1,040 individuals (595 males and 445 females) with different sources of variation, especially Pose, Expression, Accessory, and Lighting (PEAL). We choose the first 300 subjects (one image per person) from the gallery set of CAS-PEAL-R1 as our gallery set. Our probe set contains 1,939 frontal face pictures, including expression (878 pictures) and accessory (1061 pictures), which are chosen from the probe set of CAS-PEAL-R1. Some sample images are shown in Fig. 9. In the following experiments, all the employed images were normalized to 150 × 130 pixels by the CSU face identification evaluation system to eliminate the influence of the background.

Figure 9

Samples of the frontal image of one subject from the CAS-PEAL-R1 database.

To observe the performance of ULDPs under different conditions in expression and accessory, the experiments were conducted on the CAS-PEAL-R1 database that includes variations in facial expressions and accessories. The experimental results as shown in Fig. 10(a) demonstrate that ULPDs obtain better recognition accuracy on expression subset than LDPs but work worse than LDPs on the accessory subset. Since the patterns distribution in histogram introduced by the accessories is different from the face natural variations, ULDPs are of plain robustness to accessories and have slightly better recognition accuracy to expression.

Figure 10

Comparative the face recognition rates of ULDPs, LDPs, ULBPs, and LBPs on the CAS-PEAL-R1 database. a Result on the non-preprocessed images. b Result on the preprocessed images.

We also conducted the experiments to evaluate the effectiveness of the PULDPs on the CAS-PEAL-R1 database. It can be maintained from Fig. 10(b) that PULDPs perform better than PLBPs, PULBPs and PLDPs in both expression and accessory subsets. However, it should be noted from Fig. 10 that PULDPs have some lower accuracy than that of ULDPs. A reasonable explanation is that the preprocessing method smoothes out the minute variations of expression on the expression subset.

4.3 CMU PIE Database - Pose and Illumination Variant

We further explored the impact of the preprocessing on the performance of ULDPs under various poses and illuminations using the CMU PIE database. The database contains totally 41,368 images of 68 individuals, including a variety of poses, illuminations, and expressions. The images with the variations in both pose and illumination are used in the experiment. These images include seven different poses, labeled by Poses 05, 07, 09, 11, 27, 29, and 37, respectively, and each pose has 21 different illumination conditions labeled by Flashes 2 to 22. Here we choose five poses for our experiments, Pose 05, 07, 09, 27, and 29, since the five sets have slight pose variations only. In this experiment, all the images were cropped to 64 × 64 pixels according to the location of eyes, aiming to obtain the whole facial image and eliminate the complex background. Figure 11 shows an example of the normalized images of a person. The frontal images (Pose 27) with the frontal illumination condition (Flash 08) of a person are chosen to build the gallery set. The remaining images are used as the probe set. In this experiment, we use 12 × 12 sized sub-region with 128 histogram bins for both LBPs and LDPs.

Figure 11

Samples of the frontal image of one subject from the CMU PIE database.

Figure 12 depicts the recognition rates of LBPs, ULBPs, LDPs, and ULDPs on both non-preprocessed and preprocessed images by using the probe set. It can be observed from Fig. 12 that the ULDPs have a better recognition rate than the others on both non-preprocessed and preprocessed images.

Figure 12

Comparative face recognition rates of ULDPs, LDPs, ULBPs, and LBPs on the CMU PIE databse.

Compared with LBPs(39.79 %), ULBPs(40.07 %), and LDPs (55.45 %), ULDPs(59.37 %) obtain the highest recognition rate on non-preprocessed images. Moreover, the PULDPs (77.37 %) are also higher than PLDPs (74.1 %) and PLBPs (40.07 %). The results show that ULDPs can obtain more discriminative features than LBPs and LDPs.

We aslo conducted an experiment to explore the robustness of the proposed ULDPs method to poses. Tab. 1 shows the results of this experiment.

The experiment results in Table 1 show that the PULDPs have a better robustness to poses than ULDPs at all the five poses. Compared with other poses, ULDPs(68 %) and PULDPs(88 %) obtain the best recognition accuracy at Pose 27. As stated previously, the images at Pose 27 are the fontal ones. Then, a conclusion can be reached from the results that both ULDPs and PULDPs have poor robutness to pose variations. Although the performance of ULDPs decreases with the pose variations of the images in this database, it still maintains superiority in recognition accuracy over both LBPs and LDPs.

Table 1 Experiment using the CMU PIE database with different poses.

4.4 Extended Yale B Database - Illumination Variant

The Extended Yale B database was used for the comparative studies among ULDPs, LDPs, and LBPs under different illumination variations. The database contains totally 2,432 frontal images of 38 individuals (each subject is imaged under 64 different illumination conditions). All the images have already been manually aligned and cropped to 168 × 192 pixels for public downloading from the database website. In our framework, these images were normalized to 84 × 96. Figure 13 shows the samples of the frontal images. The frontal face images with lighting direction of 0o azimuth (“A + 000”) and 0o elevation (“E + 00”) are used to build the gallery set. All the remaining images belong to the probe set. Here, 12 × 13 pixel windows were selected since they are good trade-off between recognition performance and feature vector length.

Figure 13

Samples of the frontal image of one subject from the Extended Yale B database.

For the Extended Yale B database, the recognition rates on the both non-preprocessed and preprocessed probe sets are illustrated in Fig. 14. It indicates that ULDPs obtain a 3.13 % performance improvement over LDPs. Meanwhile the recognition accuracy of PULDPs reaches its maximum of 99.04 %, and obtains 31.5 % performance improvement than that of ULDPs. It means that PULDPs are more robust against illumination variation. We can also learn from Fig. 14 that both PULPDs and PLDPs get much better recognition performance than LBPs and ULBPs.

Figure 14

Experiment results on the Extended Yale B databse.

In addition, the computational time of the experiments was also considered, as shown in Table 2. Compared with LDPs, the time for ULDPs decreases 70.54 %, 64.89 %, 61.77 % and 62.57 %, on FERET, CAS-PEAL-R1, CMU PIE and Extended Yale B databases, respectively. All the experiments were carried out on a computer running Windows XP with an AMD Athlon II CPU processor (X4 640 @ 3.0 GHz), a 2-GB RAM. For all methods, we used the original MATLAB implementation by using MATLAB 7.10.0 (R2010a).

Table 2 Computational time (CPU seconds) of LDPs and ULDPs on different databases.

5 Conclusions

Local Derivative Patterns (LDPs) are extended to Uniform Local Derivative Patterns (ULDPs) in this paper. Experimental results, on a set of publicly available face databases, demonstrate that the proposed ULDPs have better robustness with the respect to facial expression, illumination and aging than LDPs. Compared with LDPs, the computational time for ULDPs decreases over 60 % in all four databases used in this paper, although ULDPs get worse in some subset with accessories. We could say that the ULDPs method is an excellent choice if the computational time is a major concern with good recognition rates.

In addition, PULDPs demonstrate the validity and efficiency of a combination of ULDPS and the preprocessing method for variable illumination in face recognition. The recognition accuracy of PULDPs reaches its maximum of 99.04 % on the Extended Yale B database.

However, it should be noted that the performance of ULDPs is still relatively low when face images contain various accessory. Future work includes improving their robustness with accessory variations. Another important topic is looking for an effective feature selection method to reduce the length of the feature vector.