Sparse tensor embedding based multispectral face recognition

Abstract

Face recognition using different imaging modalities has become an area of growing interest. A large number of multispectral face recognition algorithms/systems have been proposed in last decade. How to fuse features of different spectrum has still been a crucial problem for face recognition. To address this problem, we propose a sparse tensor embedding (STE) algorithm which represents a multispectral image as a third-order tensor. STE constructs sparse neighborhoods and the corresponding weights of the tensor. One advantage of the proposed technique is that the difficulty in selecting the size of the local neighborhood can be avoided in the manifold learning based tensor feature extraction algorithms. STE iteratively obtains one spectral space transformation matrix through preserving the sparse neighborhoods. Due to sparse representation, STE can not only keep the underlying spatial structure of multispectral images but also enhance robustness. The experiments on multispectral face databases, Equinox and PolyU-HSFD face databases, show that the performance of the proposed method outperform that of the state-of-the-art algorithms.

Introduction

Multibiometrics refers to the use of a combination of two or more sensor modalities in a single identification system. The reason for combining different sensor modalities is to improve the recognition accuracy. Face recognition using different imaging modalities has become an area of growing interest [1], [2], [3], [4], [5], [6], [7]. Recent studies have found that multispectral face recognition offers a few distinct advantages, including invariance to illumination changes [8], [9]. Multispectral image also reveals anatomical information of a subject that is useful in detecting disguised faces [1]. In [3], [10] several different face recognition algorithms were tested and good performances were achieved. The effect of lighting, facial expression, and passage of time between the gallery and probe images was examined by Chen et al. [11]. Wang et al. [12] demonstrated that color space learning represents a viable approach for enhancing face recognition performance. The image-based fusion performed in the wavelet domain and feature-based fusion performed in the eigenspace domain were present in [13]. Heo et al. [14] proposed to fuse visual and thermal signatures with eyeglass removal for robust face recognition. Multi-sensory face biometric fusion methods were investigated for personal identification [15]. Pan et al. [4], [5] analyzed the facial tissue spectral measurements for multispectral face recognition in the near-infrared spectral range (0.7–1.0 μm). Denes et al. [6] adopted three single visible bands (0.6 μm, 0.7 μm, and 0.8 μm) to test the spectral asymmetry. Chang et al. [16] fused the multispectral images in the visible spectrum (0.4–0.72 μm) into a single image and compared the result with the visible image to validate the improvement of face recognition due to the image fusion. Chou and Bajcsy [7] pre-processed the multispectral images (visible: 0.4–0.72 μm, near-infrared: 0.65–1.1 μm) by principal component analysis (PCA) to extract the first principle component information for face detection. Wong and Zhao [17] adopted kernel PCA for visual information aided thermal image reconstruction.

The above methods are mainly designed to preserve the global structure information of the data. Unsupervised algorithms [7], [9], [17] and supervised algorithms [33] have been widely used in multispectral face recognition. However, research results from manifold learning methods developed in the past decade show that the local geometric structure is more important than the global structure since the high-dimensional data lies on the low-dimensional manifold. Due to the low-dimensional manifold distributions of the face images, the manifold-learning-based linear feature extraction methods such as Laplacianfaces [18] become popular. Recent research shows that the high-order tensor based feature extraction methods such as tensor locality preserving projection (tensor LPP) [19], [20] can obtain better performance than the classical feature extraction method. Unfortunately, the tensor data contain large quantities of information redundancy and thus not all the features/variables are important to feature extraction and classification [21], [22], [23]. It was shown that integrating sparse representation, manifold learning for feature extraction may obtain better performance [24]. Many papers show that the sparse representation methods can obtain better performance than their corresponding non-sparse methods in the real data. And these sparse methods can give an intuitionistic or semantic interpretation for the transformed features [25].

Till now, the field in high order tensor data embedding with sparse manner has not been widely investigated and how to extend the manifold learning algorithms integrating sparseness and manifold structure for multispectral face recognition is unsolved. In this paper, motivated by tensor data embedding and sparse representation, we propose a novel method called sparse tensor embedding (STE) for multispectral image feature extraction. The multispectral image is considered as a third-order tensor. The aim of STE is to obtain transformation matrices through preserving the sparse information of the third-order tensors.

The rest of the paper is organized as follows. In Section 2, we give the related definitions to tensor. In Section 3, the introduction of tensor locality preserving projection is provided. In Section 4, a novel sparse tensor embedding method is presented. Experiments are carried out to evaluate the proposed tensor learning method in Section 5, and conclusions are given in Section 6.