Multiresolution face recognition

https://doi.org/10.1016/j.imavis.2004.09.002Get rights and content

Abstract

In this paper the contribution of multiresolution analysis to the face recognition performance is examined. We refer to the paradigm that in classification tasks, the use of multiple observations and their judicious fusion at the data, feature or decision level improves the correct decision performance. In our proposed method, prior to the subspace projection operation like principal or independent component analysis, we employ multiresolution analysis to decompose the image into its subbands. Our aim is to search for the subbands that are insensitive to the variations in expression and in illumination. The classification performance is improved by fusing the information coming from the subbands that attain individually high correct recognition rates. The proposed algorithm is tested on face images that differ in expression or illumination separately, obtained from CMU PIE, FERET and Yale databases. Significant performance gains are attained, especially against illumination perturbations.

Introduction

Face recognition problem has become one of the most relevant research areas in pattern recognition. Face recognition debts its popularity to its potential application areas, ranging from human computer interaction to authentication and surveillance.

The holistic or appearance-based approach has been gaining popularity vis-à-vis anthropometrical feature-based approach in face recognition [1]. In the holistic approach, all the pixels in the entire face image are taken as a single signal, and processed to extract the relevant features for classification. Most of the appearance-based face recognition algorithms perform some kind of subspace analysis in the image space to extract the relevant feature vectors. The most widely used subspace analysis tools are the principal component analysis (PCA) [2], linear discriminant analysis (LDA) [3] and a blind source separation technique, called independent component analysis (ICA) [4]. All face recognition algorithms, however, witness a performance drop whenever face appearances are subject to variations by factors such as occlusion, illumination, expression, pose, accessories and aging. In fact, often these factors lead to intra-individual variability of face images, to the extent that they can be larger than the inter-individual variability [5].

In this study, we apply multiresolution techniques in order to mitigate the loss of classification performance due to changes in facial appearance. We design experiments specifically to investigate the gain in robustness against illumination and facial expression changes. The underlying idea in the use of the multiresolution analysis is firstly, to obtain multiple evidences from the same face, and search for those components that are less sensitive to intrinsic deformations due to expression or due to extrinsic factors, like illumination. Secondly, our approach follows the paradigm of fusion that utilizes multiple evidences. Although at first sight, these evidences can appear somewhat redundant and may contain less information, their judicious combination can prove often to be superior for classification.

The most popular multiresolution analysis technique is the wavelet transform. Therefore in this study we use the 2D discrete wavelet transform in order to extract multiple subband face images. These subband images contain coarse approximations of the face as well as horizontal, vertical and diagonal details of faces at various scales. Subsequently, we extract PCA or ICA features from these subbands. We exploit these multiple channels by fusing their information for improved recognition. We have compared three fusion approaches, namely, fusion at the subband data level, fusion at the ICA/PCA feature level, and finally, fusion of the classifier decisions at the subband channel level. The main contribution of the paper is thus to search for most discriminative set of wavelet channels, and to construct face recognition schemes using fusion techniques at different levels of data processing.

Discrete wavelet transform has been used in various studies on face recognition [6], [7], [8], [9], [10]. In [6], three-level wavelet transform is performed to decompose the original image into its subbands, on which the PCA is applied. The experiments on Yale database show that third level diagonal details attain highest correct recognition rate. A wavelet transform-based speaker identification system in a teleconferencing environment is proposed in [7]. In this algorithm a three-level wavelet decomposition is performed. The scaling components at each level as well as the original image are used for classification. The classifier used in this study is a kind of neural network with one-class-in-one-network structure, that is, each subnet is trained separately and there is one subnet per individual. Wavelet packet analysis-based face recognition system is proposed in [8]. The original image is decomposed into its subbands by using two-level wavelet packet decomposition. Out of the 16 subbands, a 21-dimensional feature vector is obtained consisting of variances of 15 detail subbands and three mean values and three variances calculated from different parts of the approximation subband. From this 21 components, only the most meaningful components are selected resulting in a final feature vector size of 11. Bhattacharya distance between these statistical features is used to classify faces. In [9], three-level wavelet decomposition is performed and the resulting approximation subbands at each level are concatenated to produce a new data vector on which PCA is applied. Radial basis functions are used as the classifier of the system. Discriminant waveletfaces approach is proposed in [10]. In this study, third level approximation resulting from three level wavelet decomposition, called the waveletface, is used as the input of the LDA. For classification, presented nearest feature plane (NFP) and nearest feature space (NFS) classifiers are examined. Different from these previous studies, we put into evidence the contribution of wavelet subbands to combat; specifically, illumination and expression factors, and we investigate the interplay of subband information fusion styles, choice of metrics and of features. In other words, the thrust of the paper is to explore how the discriminatory ICA and PCA features can be de-sensitized or rendered more invariant to the effects of illumination and expression via the judicious selection of subbands and via fusion at various levels.

The paper is organized as follows. In Section 2, multiresolution analysis is briefly reviewed. Subspace analysis tools (PCA, ICA) and fusion techniques used in the study are explained in 3 Subspace analysis, 4 Fusion, respectively. In Section 5 experimental results against expression and illumination variations are presented separately. Finally, in Section 6 conclusions are given.

Section snippets

Multiresolution analysis

Multiresolution methods provide powerful signal analysis tools, which are widely used in feature extraction, image compression and denoising applications. Wavelet decomposition is the most widely used multiresolution technique in image processing. Images have typically locally varying statistics that result from different combinations of abrupt features like edges, of textured regions and of relatively low-contrast homogeneous regions. While such variability and spatial nonstationarity defies

Subspace analysis

An m×n resolution face image can be considered as a point in an N=m×n dimensional image space. For example, a 128×128 face image corresponds to a point in 16,384-dimensional huge feature space. On the other hand, face images are very similar, and therefore highly correlated. It follows than that they can be represented in a much lower dimensional feature subspace. PCA and ICA are the two popular methods to descend to such face subspaces.

Fusion

The outcomes from the various wavelet channels are fused to achieve possibly higher correct recognition rates. We investigated three schemes, namely, fusing raw pixel values of the subbands, fusing ICA/PCA feature vectors extracted from the subbands, and fusing the classification decisions of the subbands.

Experiments

Two separate experiments are conducted to test the advantage of the wavelet-based face recognition scheme. In the first experiment, the subbands that are potentially insensitive to changes in expression are searched, whereas in the second experiment the subbands that are insensitive to variations in illumination are searched. In both the experiments, feature vectors are extracted from the subband images via PCA, ICA1 and ICA2. The FastICA algorithm [12] is used to perform ICA. Daubechies 4

Conclusions

In this study, we searched for the frequency subbands that qualify as being insensitive to expression differences and illumination variations on faces. Briefly, it was observed that the frequency subbands containing coarse approximation of the images are successful against expression differences, whereas the subbands containing horizontal details are successful against illumination variations.

Since the recognition performance is not in the first place very adversely affected by changes in

References (17)

  • C. Garcia et al.

    Wavelet packet analysis for face recognition

    Image and Vision Computing

    (2000)
  • B. Li et al.

    When eigenfaces are combined with wavelets

    International Journal of Knowledge-Based Systems

    (2002)
  • A. Hyvärinen et al.

    Independent component analysis: algorithms and applications

    Neural Networks

    (2000)
  • R. Brunelli et al.

    Face recognition: features versus templates

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1993)
  • M. Turk et al.

    Eigenfaces for recognition

    Journal of Cognitive Science

    (1991)
  • P.N. Belhumeur et al.

    Eigenfaces vs. fisherfaces: recognition using class specific linear projection

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1997)
  • M.S. Bartlett et al.

    Independent component representations for face recognition

    (1998)
  • Y. Moses et al.

    Face recognition: the problem of compensating for changes in illumination direction

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1997)
There are more references available in the full text version of this article.

Cited by (145)

  • Cross-resolution face recognition adversarial attacks

    2020, Pattern Recognition Letters
    Citation Excerpt :

    Although FR systems obtain very high performance when trained with datasets comprising images acquired under controlled conditions, e.g., high-resolution, they suffer a drastic drop in reliability when tested against cross-resolution (CR) scenarios [7] that naturally arise, for example, in surveillance applications [8–10]. To counteract such a weakness, Ekenel and Sankur [11] and Luo et al. [12] proposed approaches that were not based on NNs. Instead, only recently such a problem has been tackled in the DL field [13,14].

  • Cross-resolution learning for Face Recognition

    2020, Image and Vision Computing
    Citation Excerpt :

    Generally, the two techniques conceived to deal with LR query images are the Super Resolution [37,39] (SR), and the projection of the LR and the HR images into a common space [15]. Other than DL methods, researchers exploited different approaches to tackle the problem of cross-resolution FR, such as methods based on the discrete wavelet transform [7] and on the dictionary learning [22]. In this work, we considered a different approach.

  • Steerable pyramid transform and local binary pattern based robust face recognition for e-health secured login

    2016, Computers and Electrical Engineering
    Citation Excerpt :

    Nevertheless, another method utilizes multi-resolution transform techniques, such as wavelets, which extract features efficiently by analyzing images into distinct scales of resolution, which gives different sub bands from the same face. After decomposition, the components, which are less sensitive to distortion due to illumination and expressions, are taken [2]. There are two types of face features: holistic and local.

View all citing articles on Scopus
View full text