Neighbor similarity and soft-label adaptation for unsupervised cross-dataset person re-identification

Abstract

Most of the existing person re-identification algorithms rely on supervised model learning from a large number of labeled training data per-camera-pair. However, the manual annotations often require expensive human labor, which limits the application of supervised methods for large-scale real-world deployments. To address this problem, we formulate a Neighbor Similarity and Soft-label Adaptation (NSSA) algorithm to transfer the supervised information from source domain to a new unlabeled target dataset. Specifically, we introduce a distance metric on the target domain, which incorporates inner-domain neighbor similarity and inter-domain soft-label adapted from source domain. The unlabeled samples which are close in this metric are considered to share the same pseudo-id and further selected to fine-tune the model. The training is performed iteratively to incorporate more credible sample pairs and incrementally improve the model. Extensive experimental results validate the superiority of our proposed NESSA algorithm, which significantly outperforms the state-of-the-art unsupervised and domain adaptation re-identification methods.

Introduction

Person re-identification (Re-ID) plays an important role in surveillance video analysis because of a wide range of real-world applications, such as searching the target person, analyzing the trace of crowd flow etc.. Given a query pedestrian image, the task aims at matching the same pedestrian from multiple non-overlapping cameras. In spite of various forms of different re-id methods, it shares a common goal of learning an optimal visual representation from image space to feature space, which pulls images of the same identity close to each other while pushing those of different identities apart in the learned feature space.

Deep neural networks (DNNs) have shown prominent advantages in representation learning and have been proven highly effective in supervised person re-identification [1], [4], [7], [21], [32], [33], [41], [46], [47]. With the manually labeled identity for each image, the objective function based on similarity (e.g. pairwise [33], triplet [25] or quadruplet [3] loss) and classification (e.g. Softmax [39], [49] or OIM [40] loss) is applied to train a DNN model, which learns an optimal feature representation of person images. However, the manual annotations require expensive human labor, especially in long-period multi-camera scenarios. For each person appearing in one camera, it needs to traverse all other cameras to find out if the person appears again. The annotation cost limits the application and expansion of supervised re-id methods in the large-scale real-world scenarios.

Hence, unsupervised cross-dataset re-id algorithms have been raised in recent years. Given an annotated source dataset, this task aims at learning the discriminative feature representation on the target dataset without any label. This is a challenging problem due to the objective gap between source and target domains, including the view of cameras, quality of images, change of dressing style due to the different regions and seasons, etc.. A common practice to cross-datasets problem is unsupervised domain adaptation. but this method assumes that the source and target domains share the same set of classes. However, this assumption does not hold for person re-id because the source and target datasets usually contain entirely different identities. A few methods [23], [34] assume that the datasets share the same semantic attributes that can be learned from the source domain and transferred to the target domain. Another kind of methods [6], [36], [52] train an image-to-image translation model and generate images with identities on the target domain.

In this work, we follow the self-supervised methods [9], [20] and propose Neighbor Similarity and Soft-label Adaptation (NSSA), a simple yet effective algorithm for unsupervised cross-dataset person re-id problem. The illustration of this method is shown in Fig. 1. Firstly, a feature representation model $\mathcal{F}$ is trained on the supervised source dataset, then applied on the target domain to extract the feature points on the dataset. Note that there is not any supervised label on the target dataset, the self-supervised mechanism relies on the initial feature distance to obtain similar pairs. Besides the commonly used Euclidean distance, we introduce fused distance metrics, including inner-domain neighbor similarity and inter-domain soft-label, to achieve a better similarity metric. Then we select the samples following the assumption that the feature pair with smaller distance ought to have higher probability of sharing the same identity label. The most credible samples, which are associated by the pseudo-id, are selected to fine-tune the model $\mathcal{F}$. These steps are performed iteratively to incrementally improve the discriminability of model $\mathcal{F}$.

It is worth noting that the fused distance metric in our method consists of three aspects: (1) The vanilla Euclidean distance of features, which is solely applied in previous self- supervised cross-dataset re-id methods [9], [20]. (2) The inner-domain neighbor similarity, which explores the topological relationship between the feature points and utilizes the priority that the group of features from the same identity should be close to each other and have similar neighbors. (3) The inter-domain soft-label, which spreads the identity label from source domain to target domain according to the cross-domain point-wise distance. The soft-label provides effective supervised information on the unlabeled target domain.

The unlabeled data samples are progressively taken into the training schedule. The initial model $\mathcal{F}$ is suboptimal on target dataset at the beginning iteration, hence not all the feature points can be assigned with an accurate pseudo-id at the beginning. Therefore, the samples with higher confidence are sampled to fine-tune the model $\mathcal{F}$. With the improvement of the model at the training stage, more credible samples are selected into the training set.

The remainder of the paper is organized as follows: Related works are reviewed in Section 2. In Section 3, we discuss the detailed implementation of our proposed model. The experimental analysis and comparison with the state-of-the-art methods are presented in Section 4.