Local appearance based face recognition method using block based steerable pyramid transform
Introduction
Among face recognition methods, the most popular are holistic appearance-based approaches such as PCA [13], LDA [46], ICA [4] and review of these methods has been presented in [21]. Subspace analysis based methods has been proposed in [47], [48], [49] in order to give an effective feature extraction in high dimensional space. These methods outperform holistic methods in recognition accuracy. Recently, there are more and more attempts to develop face recognition systems based on local features. The approach of analyzing faces locally is believed to outperform the holistic appearance-based approaches, where a local change affects only the corresponding part of the representation and does not modify the representation vector as a whole [32]. These approaches exhibit good performance and robustness in controlled environments but still do not perform well in many real-world situations, due to variations in pose, lighting and expression. In order to address this issue, many researchers propose to deploy a pre-processing step in order to capture more discriminant features for use in the recognition step, such as local binary patterns (LBP) and local discrete cosine transform [33], [34]. In [41] the authors propose to utilize the total variation (TV) mode to factorize an image in order to overcome the illumination limitation and to preserve the edge information of images. However, the TV model is able to only process images in certain scale. In addition, the TV model is an iterative type approach, thus the computational expense is very high. More recently, interest has grown in using multi-resolution methods where multiple evidences (sub-bands) from the same face are obtained allowing extraction of less sensitive features to intrinsic deformations due to expression or due to extrinsic factors, like illumination. These transforms have more edge-preserving ability than TV model in low frequency illumination fields [40]. Multi-resolution methods have been successfully used in many challenging pattern recognition applications including character recognition [2] and face recognition [1], [3], [26].
Among multi-resolution methods for face recognition, the most popular are discrete wavelet transform (DWT) [22], [23], [24], [25], [50] and Gabor wavelets [12]. These methods have proved to be very successful to capture more discriminant features of face images leading to higher performance and robustness against various challenging conditions. However, wavelet-based features are not suitable for face recognition in uncontrolled environments since images do not always exhibit isotropic scaling (horizontal, vertical and diagonal). In [40], authors treated a particular problem of illumination invariance. Since illumination is represented by a convolution, it can be avoided in log-domain by wavelet denoising technique. Their method claimed to be robust in different illumination conditions leading to the best results in Yale B face database. Others have shown that Gabor filters can attain good results in many face recognition applications. However, the use of Gabor filters dramatically increase the computational cost of the face recognition method, requiring that each kernel is convolved with the input image [12]. Contourlet [9], curvelet transforms [11], and steerable pyramid are another multi-resolution transforms similar to the two-dimensional DWT, but with interesting translation and rotation-invariance properties [5]. The steerable pyramid (S-P) is a linear multi-scale, multi-orientation image decomposition which has been developed to overcome the wavelet limitations. Though steerable pyramids provide more scale and orientation than wavelets. The curvelet transform captures curves instead of points as in S-P transform in the continuous domain. Contourlets are an extension of curvelets, which can be approximated in the discrete domain. Contourlets, however, are defined and derived in the discrete domain from the beginning. They both allow for directionality and anisotropy [42].
Applications of Contourlet to face recognition have been investigated in work presented by Boukabou et al. [3]. Authors propose to employ contourlet with PCA in order to extract discriminant features and to obtain higher recognition rates. They have evaluated the proposed method on two different databases (Yale and FERET Database) and stated that the contourlet transform outperforms the original PCA method. More experiments have to be performed on large database and many comparisons against well established existing techniques must be done to assess this result. Mandal et al. [1] propose curvelet based face recognition system by fusing results from multiple SVM classifiers trained with curvelets coefficients from images having different gray scale resolutions (2, 4 and 8 bits). However, this algorithm is computationally expensive since it requires taking the curvelet transform of the original image and its quantized representations. In [26], [27], curvelet transform is introduced in conjunction with different dimensionality reduction tools. These techniques appear to be robust to the changes in facial expression as they show good results for the Essex and the ORL database, but still do not perform well in YALE database that contains images with great variations in illumination and facial expression.
According to previous review on local based approaches, which have proven to be robust to most face recognition challenges comparing to global based approaches, we have proposed a face recognition method based on local presentation of curvelet transform [28]. Curvelet transform is applied to the face image and each of the resulting sub-bands is partitioned into a set of equally sized blocks in a non-overlapping way. Then the statistical measures (mean, variance and entropy) of the energy distribution of the curvelet coefficients for each block in each sub-band at each decomposition level is used to construct the feature vector. Then, we used curvelet transform as an improvement tool of LDA in [45]. The proposed methods have been evaluated on Yale, ORL and FERET databases and have shown better recognition accuracy in comparison to holistic approaches. However, local-based approaches cannot be applied to this technique because of the small sub-band size.
To solve all these mentioned problems, the steerable pyramid can be employed to produce any number of orientation bands. In addition, it conserves the same image resolution in the first scale level which is more adequate for local appearance based approaches. Several studies have investigated the discriminating power of steerable pyramid-based features in various applications including: image denoising [10], textures classification [12], image processing [30], [5] and face hallucination [29]. In [44], the S-P method has been proposed in conjunction with LBP of each sub-band in order to extract a local information of face images. This work gave promised results to investigate extensively the use of S-P transform both in global and local appearance, and feature/score fusion which is the subject of the present paper.
In this paper, we present a face recognition approach based on steerable pyramid decomposition. Following our previous study [43], the main contribution of this paper is to fully investigate the usefulness of steerable pyramids transform in a face recognition framework. Each face image is described by a subset of band filtered images containing steerable pyramid coefficients which characterize the face textures. We divide the S-P sub-bands into small sub-blocks, from which we extract compact and meaningful feature vectors using mean, variance and entropy. We conceive an experiment framework specifically to investigate the improvement in robustness against illumination and facial expression changes. We discuss the important problem of fusing the local observations that utilizes multiple sub-bands. Then, we investigate fusion schemes both at the feature and decision levels. Finally, we show how an efficient and reliable probabilistic metric can be used in order to classify the face feature vectors into person classes. Experimental results are presented using images from the FERET, ORL, ESSEX, YALE and YALE B databases. The efficiency of our approach is firstly analyzed by comparing the results with those obtained using multi-resolution methods such as wavelet, Gabor, contourlet and curvelet. Secondly it is compared to the best given results in the literature and has shown better recognition performance.
The remainder of the paper is organized as follows. In Section 2, face feature extraction based steerable pyramid transform is introduced. Block based S-P face identification proposed method is given in Section 3. In Section 4, three fusion schemes used in the study are explained, namely, data fusion, feature fusion, and decision fusion. Experimental results are presented and discussed in Section 5. Finally, in Section 6, conclusions and future recommendations are given.
Section snippets
Face feature extraction based steerable pyramid transform
A face image of a person contains similarity (approximation) information of the face as well as discriminatory (detail) information with respect to faces of all other persons. The discriminatory information is due to structural variations of the face which are acquired as intensity variations at different locations of the face. The location and degree of intensity variations in a face for an individual are unique features which discriminate one person from the rest of the population. Steerable
Steerable pyramid feature extraction methods
The S-P transforms can be used for feature extraction in two different ways:
S-P features and subspace analysis
This section presents work that utilizes S-P features and subspace analysis for face identification. Once features are extracted, subspace analysis could be applied for further class separability enhancement and feature dimension reduction.
Fig. 3 shows a flow chart demonstrating the use of S-P features and subspace analysis for face recognition. Initially a set of eight S-P filters are used to extract appropriate features, which are then induced to PCA or LDA. The S-P features extracted from a
Steerable pyramid fusion schemes
The main idea behind using the S-P analysis is firstly, to obtain multiple evidences (sub-band) from the same face, and search for those sub-bands that are less sensitive to intrinsic deformations due to expression or due to extrinsic factors, like illumination. Secondly, fuse the local observations that utilizes multiple sub-bands. Despite the fact that at first sight, these sub-bands can appear somewhat redundant and may contain less information, their prudent combination can prove often to
Experimental results
Four separate experiments are conducted to test the advantage of the S-P face recognition scheme. In the first experiment, we employ traditional transforms (PCA, LDA) to enhance and extract discriminative features in all sub-bands. While in the second experiment the sub-bands that are potentially insensitive to changes in expression and variations in illumination are searched. Whereas in the third experiment, the fusion of the best performing sub-bands is investigated. Finally a comparative
Conclusions and future works
The main contribution of this paper is to investigate a new approach using steerable pyramid coefficients to address the problem of human face recognition from still images. For each face image, S-P is performed to compute different sub-bands from which some statistical measures are extracted using a block-based technique. This is the first time steerable pyramid transform is being explored in face recognition application. In the case of ORL, YALE and ESSEX we have almost obtain the best
References (47)
- et al.
Curvelet based face recognition via dimension reduction
Signal Processing
(2009) - et al.
Multiscale facial structure representation for face recognition under varying illumination
Pattern Recognition
(2009) - T. Mandal, A. Majumdar, Q.M.J. Wu, Face recognition by curvelet based feature extraction, in: ICIAR 2007, Lecture Notes...
Bangla basic character recognition using digital curvelet transform
Journal of Pattern Recognition Research JPRR
(2007)- W.R. Boukabou, A. Bouridane, Contourlet-based feature extraction with PCA for face recognition, in: NASA/ESA Conference...
- et al.
Independent component representations for face recognition
- E.P. Simoncelli, A rotation-invariant pattern signature, in: Third IEEE International Conference on Image Processing,...
- A.N. Belbachir, P.M. Goebel, The contourlet transform for image compression, in: Physics in Signal and Image...
- A. Li, X. Li, S. Wang, H. Li, A multiscale and multidirectional image denoising algorithm based on contourlet...
- E.J. Candes, D.L. Donoho, Curvelets: a surprisingly effective nonadaptive representation for objects with edges,...
Comparison and fusion of multiresolution features for texture classification
Pattern Recognition Letters
Eigenfaces for recognition
Journal of Cognitive Neuroscience
The design and use of steerable filters
IEEE Transactions on Pattern Analysis and Machine Intelligence
The FERET evaluation methodology for face recognition algorithms
IEEE Transactions on Pattern Analysis and Machine Intelligence
Human face recognition using pca on wavelet subband
Journal of Electronic Imaging
Discriminant waveletfaces and nearest feature classifiers for face recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence
Face recognition by applying wavelet subband representation and kernel associative memory
IEEE Transaction on Neural networks
Face recognition: a literature survey
ACM Computing Survey
Wavelets for computer graphics: a primer part 1
IEEE Computer Graphics and Application
Image compression using the Haar wavelet transform
Spelman Science & Mathematics Journal
Image compression through wavelet transform coding
IEEE Transactions on Information Theory
Cited by (47)
Local appearance-based face recognition using adaptive directional wavelet transform
2019, Journal of King Saud University - Computer and Information SciencesCitation Excerpt :Additionally, the LDA not only provides the dimensionality reduction but also provides the discriminant features from LL subband. All these MRA face recognition methods (GFC, curvelet, ridgelet, contourlet, and LGBP) require multiple orientations and scales to describe the face feature vectors effectively (El Aroussi et al., 2011). They have a weakness in terms of high computational costs due to consideration of the large size of feature vectors for achieving a better performance.
Mixed neighborhood topology cross decoded patterns for image-based face recognition
2018, Expert Systems with ApplicationsCitation Excerpt :The holistic face representation approaches extract features from whole face images. A wide variety of holistic methods for face recognition have been proposed in the literature, including approaches based on independent component analysis (ICA) method (Secchi, Vantini, & Zanini, 2016), Zernike moments method (Bereta, Pedrycz, & Reformat, 2013; Singh, Mittal, & Walia, 2011), global Gabor-Zernike feature descriptor (Fathi, Alirezazadeh, & Abdali-Mohammadi, 2016), eigenface method or principal components analysis (PCA) method (Cavalcanti, Ren, & Pereira, 2013; Wen, He, & Shi, 2012); different transforms such as wavelet sub-bands (Huang, Li, Shang, Wang, & Zhang, 2015a; Huang, Li, Wang, & Zhang, 2015b), Gabor filters (Abhishree, Latha, Manikantan, & Ramachandran, 2015; Yu, He, & Cao, 2010), optimal matrix factorization (Guan, Tao, Luo, & Yuan, 2012) and steerable pyramid transform (El Aroussi, El Hassouni, Ghouzali, Rziza, & Aboutajdine, 2011); Fisherface method which uses linear discriminant analysis (LDA) technique (Lu, Jin, & Zou, 2012a; Lu, Zou, & Wang, 2012b; Wen et al., 2012), etc. Despite some merits like low processing time for both feature extraction and computing similarity, holistic features come with the following limitations and disadvantages: i) they usually ignore local details (Yang, Wang, & Zhang, 2016a), ii) they may be oversensitive to pixel’s location and consequently fail to identify important visual characteristics (Datta, Joshi, Li, & Wang, 2008; Halawani, Teynor, Setia, Brunner, & Burkhardt, 2006) and iii) their performance degrades significantly under pose and illumination variations (Patel, Maheshwari, & Raman, 2016; Tolba, El-Baz, & El-Harby, 2006).
A modified technique for face recognition under degraded conditions
2018, Journal of Visual Communication and Image RepresentationSteerable pyramid transform and local binary pattern based robust face recognition for e-health secured login
2016, Computers and Electrical EngineeringCitation Excerpt :SPT was used in several applications of image processing, for example, image denoising [14], forgery detection [15], and texture classification [7]. It has also been investigated in the face recognition system [16]; however, it was not fully explored there. The contributions of this work are (i) the development of an SPT-LBP based face recognition system, (ii) a thorough investigation of different subbands of the SPT towards the recognition of face, and (iii) a selection of subbands that achieve optimum results.
A new Global-Gabor-Zernike feature descriptor and its application to face recognition
2016, Journal of Visual Communication and Image RepresentationCitation Excerpt :In the global methods, features are extracted from all over the facial image and there is less sensitivity to noise [3]. Some of the most important global feature extraction techniques are: the eigenface method or principal components analysis (PCA) method which is applied to the face as a whole [4,5], the Fisherface method which uses linear discriminant analysis (LDA) technique [1,5,6], the methods based on different transforms such as Gabor filters [7,8], wavelet sub-bands [9,10], steerable pyramid transform [11] and optimal matrix factorization [12], the Zernike moments method [3,13], and the independent component analysis (ICA) method [14]. The PCA, LDA, and ICA methods use statistical techniques to obtain the distribution function of data and also to reduce the dimension of feature vector.
Developing a late fusion of multi facial components for facial recognition with a voting method and global weights
2023, International Journal of Computational Vision and Robotics