Dynamic training using multistage clustering for face recognition

Abstract

A novel face recognition algorithm that uses dynamic training in a multistage clustering scheme is presented and evaluated. This algorithm uses discriminant analysis to project the face classes and a clustering algorithm to partition the projected face data, thus forming a set of discriminant clusters. Then, an iterative process creates subsets, whose cardinality is defined by an entropy-based measure, that contain the most useful clusters. The best match to the test face is found when only a single face class is retained. This method was tested on the ORL, XM2VTS and FERET face databases, whereas the UMIST database was used in order to train the proposed algorithm. Experimental results indicate that the proposed framework provides a promising solution to the face recognition problem.

Introduction

Face recognition (FR) is an active research field that has received great attention in the past several years. A FR system usually attempts to determine the identity of the test face by computing and ranking all similarity scores between the test face and all human faces stored in the system database that constitute the training set. However, the performance of many state-of-the-art FR methods deteriorates rapidly when large, in terms of the number of faces, databases are considered [1], [2]. Specifically, the facial feature representation obtained by methods that use linear criteria, which normally require images to follow a convex distribution, is not capable of generalizing all the introduced variations due e.g. to large differences in viewpoint, illumination and facial expression, when large data sets are used. When nonlinear face representation methods are employed, problems such as over-fitting, computational complexity and difficulties in optimizing the involved parameters often appear [1]. Moreover, the performance of FR methods deteriorates when there is lack of a sufficiently large number of training samples for each face in the database as, in this case, the intra-person variations cannot be modelled properly. More specifically, linear methods, such as linear discriminant analysis (LDA), often suffer from the small sample size (SSS) problem, where the dimensionality of the samples is larger than the number of available training samples [3].

Recently, various methods have been proposed in order to restrict the maladies that are imposed by the two aforementioned types of problems on the recognition performance. The ‘divide and conquer’ principle, by which a database is decomposed into smaller sets in order to piecewise learn the complex distribution by a mixture of local linear models, has been widely used. In Ref. [1], a separability criterion is employed to partition a training set from a large database into a set of smaller maximal separability clusters (MSCs) by utilizing an LDA-like technique. Based on these MSCs, a hierarchical classification framework that consists of two levels of nearest neighbour (NN) classifiers is employed and the match is found. The work in Ref. [4] concentrates on the hierarchical partitioning of the feature spaces using hierarchical discriminant analysis (HDA). A space tessellation tree is generated using the most expressive features (MEF), by employing principal component analysis (PCA), and the most discriminating features (MDF), by employing LDA, at each tree level. This is done to avoid the limitations linked to global features, by deriving a recursively better-fitted set of features for each of the recursively subdivided sets of training samples. In general, hierarchical trees have been extensively used for pattern recognition purposes.

LDA is an important statistical tool that has been shown to be effective in FR or verification problems [5], [6]. Traditionally, in order to improve LDA-based methods and provide solutions for the SSS problem, LDA is applied in a lower-dimensional PCA subspace, so as to discard the null space (i.e., the subspace defined by the eigenvectors that correspond to zero eigenvalues) of the within-class scatter matrix of the training data set [5]. However, it has been shown [7] that significant discriminant information is contained in the discarded space and alternative solutions have been sought. Specifically, in Ref. [8] a direct-LDA (DLDA) algorithm is presented that discards the null space of the between-class scatter matrix, which is claimed to contain no useful information, rather than discard the null space of the within-class scatter matrix. More recently, in an attempt to address the SSS problem, the regularized LDA method (RLDA) was presented in Ref. [9], which employs a regularized Fisher's separability criterion. The purpose of regularization is to reduce the high variance related to the eigenvalue estimates of the within-class scatter matrix, at the expense of potentially increased classification bias.

The use of static training structures, where the input data is not involved in determining the system parameters, has been abundant when designing pattern classification systems. However, it has been demonstrated that the classification performance can be improved by employing dynamic training structures. In this spirit, the Dynamic face recognition Committee Machine (DCM) was presented in Ref. [10], consisting of five state-of-the-art pattern classification algorithms. The proposed dynamic structure requires for the input to be directly involved in the combining mechanism that employs an integrating unit to adjust the weight of each expert according to the input. A gating network is used to identify the situation that the input image is taken and assign particular weights to each expert. Experimental results indicate that using this dynamic structure gives higher recognition rates rather than using a static one where the weights for each expert are fixed. In Ref. [11], the authors derive an owner-specific LDA-subspace in order to create a personalized face verification (2-class classification) system, where the owner identity is the true identity. The training set is partitioned into a number of clusters and the cluster that contains face data that are most similar to the owner face is identified. The system assigns the owner training images to this particular cluster and this new data set is used to determine an LDA-subspace that is used to compute the verification thresholds and matching score, when a test face claims the identify of the owner. The authors show that verification performance is enhanced when owner-specific LDA-subspaces are utilized, rather than using the LDA space created by processing the entire training set.

This paper presents a novel framework that uses dynamic training in a multistage clustering process that employs discriminant analysis. For notation compactness, this algorithm shall be referred to as DTMC throughout the rest of this paper. This methodology is not restricted to FR, but is able to deal with any problem that fits into the same formalism. At this point, it is imperative that two terms that are frequently used in this paper are defined: ‘class’ refers to a set of face images from the same person, whereas ‘cluster’ refers to a set of classes.

Initially, facial feature extraction is carried out by making use of the multilevel 2-D wavelet decomposition (MWD2) algorithm [12], [13], which provides dimensionality reduction and its use has been shown to be appropriate for classification purposes [6], [14], [15]. Then, the training and test face feature vectors are projected onto a MDF-space that is created by employing the RLDA method of Ref. [9]. Subsequently, the k-means algorithm is used to partition the training data into a set of discriminant clusters. The distance of the test face from the cluster centroids is used to collect a subset of clusters that are closest to the test face. The cardinality of this subset is set through an entropy-based measure that is calculated by making use of the discrete probability histogram. Then, a new MDF-space is created from this cluster subset with its dimensions set so as to reduce classification problems that stem from possible large variations in the set of images of each face class. The training data projected to this new space are again clustered and a new subset that is closer to the test face is selected. This process is repeated in as many iterations as necessary, until a single cluster is selected that contains just one face class. The identity of this face class is set as the best match to the identity of the test face.

The proposed method is computationally efficient, compared to ‘divide and conquer’ techniques such as the one in Ref. [1] where multiple classification results are produced by applying an individual discriminant analysis process and a NN classifier to each cluster. Our method uses a single discriminant analysis operation at each clustering level, with the number of clustering levels being generally much smaller than the number of clusters since only a small subset of the training data is retained at each level. A heavy computational cost also accompanies algorithms that construct hierarchical trees or space tessellation, as is the case with using the HDA algorithm in Ref. [4]. The purpose of this type of algorithms is to provide a manageable discriminant solution for each and every face class by recursively subdividing the complete set of training samples into smaller classification problems. On the other hand, at each clustering step our algorithm only has to provide a discriminant solution for the face classes that are closer to the test face; the training data that correspond to the remaining face classes are discarded.

The structure of the DTMC algorithm is flexible to the adding of new training faces. Specifically, when a new training face is added to the database the only change needed in the DTMC process is to increase the dimension of the first MDF-space by one. The characteristics of the test face will determine which set of clusters, which may or may not contain the new face class, will be retained for the clustering level that follows. On the contrary, the hierarchical tree structure requires a complete re-learning of the full training space since the new MDF-space at the first tree level may lead to an entirely different decomposition result.

The MDF-spaces that the hierarchical tree or space tessellation structures utilize are generated in the learning phase and are not biased by the characteristics of the test face. On the contrary, the MDF-spaces created at each clustering level of the DTMC algorithm are indeed biased with respect to the characteristics of the test face. Based on the conclusions of Refs. [10], [11] that have been summarized above, more accurate classification results are to be expected by DTMC since it employs a dynamic classification structure that utilizes a series of test-face-specific subspaces.

The outline of this paper is as follows: Section 2 describes the feature extraction method that utilizes the MWD2 algorithm, reviews the RLDA method that is used to extract the MDF-spaces before each clustering process and presents the k-means algorithm that is used to partition the training data as well as the entropy-based measure that is used to define the number of clusters that are retained. Section 3 describes the complete DTMC FR methodology that is proposed in this paper. Experimental results are reported in Section 4, where the DTMC methodology is tested using the well-established UMIST [16], ORL [17], and XM2VTS [18] databases in order to assess its recognition capabilities on standard data sets. Moreover, the performance of DTMC is compared to a number of FR algorithms that have been recently proposed by the research community.