Collaborative probabilistic labels for face recognition from single sample per person

Highlights

• Constructed probabilistic graph propagates discrimination from generic to gallery.

• The adaptive variation type for a given sample can be automatically estimated.

• CPL incorporates a novel probabilistic label reconstruction based.

Abstract

Single sample per person (SSPP) recognition is one of the most challenging problems in face recognition (FR) due to the lack of information to predict the variations in the query sample. To address this problem, we propose in this paper a novel face recognition algorithm based on a robust collaborative representation (CR) and probabilistic graph model, which is called Collaborative Probabilistic Labels (CPL). First, by utilizing label propagation, we construct probabilistic labels for the samples in the generic training set corresponding to those in the gallery set, thus the discriminative information of the unlabeled data can be effectively explored in our method. Then, the adaptive variation type for a given test sample is automatically estimated. Finally, we propose a novel reconstruction-based classifier for the test sample with its corresponding adaptive dictionary and probabilistic labels. The proposed probabilistic graph based model is adaptively robust to various variations in face images, including illumination, expression, occlusion, pose, etc., and is able to reduce required training images to one sample per class. Experimental results on five widely used face databases are presented to demonstrate the efficacy of the proposed approach.

Introduction

Over the past three decades, as one of the most visible applications in computer vision, face recognition (FR) has been receiving significant attention [1] and a large number of algorithms have been proposed in recent years [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25]. Representative and popular algorithms include principal component analysis (PCA) [13], linear discriminant analysis (LDA) [2], locality preserving projections (LPP) [8], sparse representation based classification (SRC) [19] and their weighted, kernelized, and two-dimensional variants [10], [11], [15], [17], [25]. Recently, it has been proved that it is the collaborative representation (CR) [23], [24] but not the l₁-norm sparsity that makes SRC powerful for face classification. What's more, CR has significantly less complexity. However, because it is usually difficult, expensive and time-consuming to collect sufficient labeled samples, many traditional methods, including collaborative representation based classification (CRC), usually suffer from the scenario that only few labeled samples per person are available. To solve this problem, some semi-supervised learning (SSL) algorithms [43], [44], [45], [46], [47], which utilize a large number of unlabeled data to help build a better classifier from the labeled data, have been proposed in recent years.

The performance of above methods in face recognition, however, is heavily influenced by the number of training samples per person [26], [27]. Especially in many practical applications of FR (e.g., law enforcement, e-passport, surveillance, ID card identification, etc.), we can only have a single sample per person. This makes the problem of FR particularly hard since the information used for prediction is very limited while the variations in the query sample are abundant, including background illumination, pose, and facial corruption/disguise such as makeup, beard, and occlusions (glasses and scarves). Practically, this problem is called single sample per person (SSPP) classification. To address the problem of SSPP, many specially designed FR methods have been developed, which can be generally classified into three categories [26]: image partitioning, virtual sample generation and generic learning.

For the first category, the image partitioning based methods [29], [30], [31], [32], [33] always firstly partition each face image into several local patches and then apply the discriminant learning techniques for feature extraction. For example, Lu et al. proposed the Discriminative Multi-Manifold Analysis (DMMA) method [29], [30] by partitioning the image into several local patches as the feature for training, and converted the face recognition to the manifold-manifold matching problem. Although these methods have led to improved FR results, local feature extraction and discriminative learning from local patches are sensitive to image variations (e.g., extreme illumination and occlusion).

For the second category [34], [35], [36], [37], some additional training samples for each person are virtually generated such that discriminative subspace learning can be used for feature extraction. An early such attempt [34] shows that images with the same expression are located on a common subspace, which can be called the expression subspace. By projecting an image with an arbitrary expression into the expression subspaces, we can synthesize new expression images. By means of the synthesized images for individuals with one image sample, we can obtain more accurate estimation of the within-individual variability which contributes to significant improvement in recognition. But one common shortcoming among these methods is that the high-correlated virtually generalized samples may contain much redundancy and some linear transformation (e.g. expression subspaces) may not be suitable for the all face variations, namely pose and uncontrollable noise.

For the third category [38], [39], [40], [41], [42], an additional generic training set with multiple samples per person (MSPP) is applied to extract discriminative features which are subsequently used to identify the people each enrolled with a single sample. Based on the consideration that face variations for different subjects share much similarity, Yang et al. [38] learned a sparse variation dictionary by taking the relationship between the gallery set and an additional generic training set into account. The generic training set with multiple samples per person can bring new and useful information to the SSPP gallery set. And the sparse variation dictionary learning (SVDL) scheme shows state-of-the-art performance in FR with SSPP.

Although many improvements have been reported, SVDL and other generic training based methods just focus on the variations but ignore the discriminative information of the generic training set and the essential collaborative representative relationship between the gallery set and generic training set, which will be detailed described in Section 3.1. Motivated by this consideration and recent advances in CR [23], [24] that samples from other categories can also have contributions to reconstruct the query samples and in SSL [43], [44], [45], [46], [47] to construct a faithful graph utilizing both labeled and unlabeled samples, we propose in this paper a novel Collaborative Probabilistic Labels (CPL) for FR with SSPP, whose framework is illustrated in Fig. 1. The training samples in the gallery set are used to build a gallery dictionary. And the samples in the generic training set, which contain plenty of variations, are used as adaptive probabilistic label dictionaries. Utilizing group sparse coding [48], we can first seek a specific adaptive variation subset from the generic training set for each sample in the gallery set. By extracting collaborative reconstructive relationship between each sample in the gallery set and its corresponding adaptive variation subset and utilizing label propagation techniques, we can then acquire the probabilistic label matrix. At last given a test sample, we construct a novel reconstruction-based classifier with the probabilistic label matrix, the gallery set and an adaptive variation dictionary corresponding to the test sample.