Geometric Preserving Local Fisher Discriminant Analysis for person re-identification

Highlights

• A novel metric learning method is proposed for person re-identification.

• A novel assumption that the re-id data lies on a nonlinear manifold is made.

• Geometric structure is incorporated with nearest neighbor graph.

• The problem is solved effectively without complex iteration.

• Kernel extension of the method is proposed.

Abstract

Recently, Local Fisher Discriminant Analysis (LFDA) has achieved impressive performance in person re-identification. However, the classic LFDA method pays little attention to the intrinsic geometrical structure of the complex person re-identification data. Due to large appearance variance, two images of the same person may be far away from each other in feature space while images of different people may be quite close to each other. The linear topology exploited in LFDA is not sufficient to describe this nonlinear data structure. In this paper, we assume that the data reside on a manifold and propose an effective method termed Geometric Preserving Local Fisher Discriminant Analysis (GeoPLFDA). The method integrates discriminative framework of LFDA with geometric preserving method which approximates local manifold utilizing a nearest neighbor graph. LFDA provides discriminative information by separating different labeled samples and pulling the same labeled samples together. The geometric preserving projection provides local manifold structure of the nonlinear data induced by graph topology. Taking advantage of the complementary between them, the proposed method achieves significant improvement over state-of-the-art approaches. Furthermore, a kernel extension of the GeoPLFDA method is proposed to handle the complex nonlinearity more effectively and to further improve re-identification accuracy. Experiments on the challenging iLIDS, VIPeR, CAVIAR and 3DPeS datasets demonstrate the effectiveness of the proposed method.

Introduction

Person re-identification, which aims at matching people across multiple non-overlapping camera networks, has attracted huge interest over the recent decades [1]. It manages to achieve that when a target disappears from one camera, he/she can be re-identified in another camera deployed far away. It can save a lot of human efforts on exhaustively searching for a target from large amounts of video sequences [2].

In the literature, the methods of re-identification can be divided into two categories: feature extraction and metric learning. Feature based approaches [3], [4] focus on extracting distinctive visual features to represent the human appearance. Metric learning [5], [6], [7] approaches aim at finding an optimal metric that can maximize the distance of samples from different class whilst minimize the distance of samples from the same class. Our approach belongs to the latter category.

Typical metric learning approach such as Large Margin Nearest Neighbor (LMNN) [5], [6] tries to learn a metric that minimizes the distance between each training point and its k nearest similarly labeled neighbors, while maximizing the distance between all differently labeled points. Inspired by LMNN, a bunch of metric learning methods for person re-identification have been proposed, such as ITML [8], RDC [9], PCCA [10], KISSME [11], LFDA [7], [12], [13].¹ While these methods could achieve encouraging re-identification performance, they are limited by linearity and prone to overfitting especially in large scale and high dimensional learning scenarios.

Traditional metric learning approaches [5], [6], [7], [8] often assume that data is linearly distributed, which does not hold true in re-identification. Samples in the same class may undergo dramatic appearance variations due to changes in view angle, illumination, background clutter and occlusion [14] (see Fig. 1). Meanwhile, samples of different people may share similar appearance, e.g., people wearing clothes with similar color or similar pattern. Therefore, traditional linear topology is not sufficient to model the re-identification data.

Furthermore, the metric learning methods are prone to overfitting because of the small sample size (SSS) problem in person re-identification, i.e. the number of samples per subject is far less than the dimension of the feature. For instance, the VIPeR dataset [15] only contains two images of each subject, while the dimension of features is usually thousands or higher. In this case, metric learning methods tend to overfit because pair or triplet-based constraints become much easier to satisfy in a high-dimensional space and thereby lead to poor generalization performance. The absence of regularization further deteriorates recognition performance [16]. As for LFDA, the within-class scatter matrix ${\mathit{S}}^{\mathit{W}}$ cannot be accurately estimated because the number of within-class samples is very limited thus ${\mathit{S}}^{\mathit{W}}$ often becomes singular. The singularity can easily lead to overfitting.

Motivated by these problems, in this paper, we propose a novel algorithm termed Geometric Preserving Local Fisher Discriminate Analysis (GeoPLFDA) which makes a reasonable assumption that the re-identification data reside on a manifold and each sample corresponds to a point on the manifold. The method exploits local manifold approximation derived by nearest neighbor graph [17]. This graph topology provides better approximation to the real world data structure than linear assumptions. To accommodate with LFDA, the data is then projected into a low dimensional linear subspace following the criterion that geometric information should be well preserved. In other words, nearby points on the manifold are mapped to nearby points in the subspace, and faraway points to faraway points. LFDA is performed to improve the intra-class compactness and inter-class separation. Through a linear weighted technique, the geometric preserving techniques are effectively incorporated into the LFDA scheme. In this way, not only the discriminant information is exploited, the geometrical structure is also effectively preserved. The geometric preserving term can serve as a regularization term thus overfitting is alleviated. What׳s more, the proposed GeoPLFDA incorporates global information from the whole feature data which makes up for the fact that the discriminative margin in LFDA is determined by limited nearby data pairs [18]. In addition, we propose the kernel extension of GeoPLFDA which handles the complicated nonlinear high dimensional data structure more effectively. Experimental results demonstrate the effectiveness of the proposed method.

The main contribution of the proposed method is three-folds:

• A more faithful representation of the data structure is proposed, which assumes that the data lies on a nonlinear manifold.

• The proposed method not only exploits discriminant structure utilizing techniques from LFDA, but also effectively incorporates local structure information by constructing the nearest neighbor graph.

• A closed form solution is achieved through generalized eigenvalue decomposition. Hence, complex iterative optimization schemes are not required.

The rest of the paper is organized as follows: a brief view of related works is presented in Section 2. Section 3 introduces the proposed GeoPLFDA algorithm and its kernel extension. Experimental results on iLIDS, VIPeR, CAVIAR and 3DPeS datasets are presented in Section 4. Finally, the concluding remarks and suggestions for future work are discussed in Section 5.