Unsupervised person re-identification by Intra–Inter Camera Affinity Domain Adaptation

Highlights

• We develop an Intra–Inter Camera Affinity Domain Adaptation method for person re-id.

• Generative adversarial learning relieves the distribution gap between two domains.

• Affinity transfer learning reduces the distribution inconsistency among camera views.

Abstract

Person re-identification (re-ID) based on unsupervised domain adaptation intends to distill knowledge from annotated source dataset to identify target persons in another dataset. Although the advanced UDA re-ID models are dominated by pseudo-label methods, they almost transform images from various camera views into the same feature space, without considering the camera distribution gaps, which may lead to generate considerably noisy pseudo-labels. In this study, we develop an Intra–Inter Camera Affinity Domain Adaptation (I${}^2$CADA) to tackle these problems for UDA person re-ID. Precisely, I${}^2$CADA framework is composed of two modules. The first one is generative adversarial learning module, aiming to train a feature extractor that can map target data to source feature space by supervised learning and adversarial learning, which can relieve the distribution gap between different datasets (domains). The second one is affinity transfer learning module, which simultaneously considers intra-camera clustering and inter-camera separation among persons with similar appearances in the target domain, thus mitigating the distribution inconsistency among person images collected from multiple target camera views. Besides, comprehensive experiments exhibit that I${}^2$CADA outperforms the existing UDA person re-identification approaches.

Introduction

Person re-identification (re-ID) aims at retrieving specified person images across multiple cameras located in different directions. Re-ID is widely used in intelligent video surveillance criminal, city management, and criminal tracing, etc. It is a challenging task in computer vision since significant variations in body pose, view angle, and illumination across different views can lead to the appearance of the same person in two images look highly different and vice versa. Thanks to the successful application of deep learning technology, impressive improvements have been accomplished in person re-identification under the supervised framework [1], [2], [3], [4], [5], [6]. Nevertheless, when testing on unseen datasets, they usually suffer from dramatic performance degradation because of domain discrepancy caused by the variations of background, illumination, camera disposition, etc. Gathering and labeling sufficient training person images for a specific scene is required, whereas laborious and costly.

To address this problem, increasing efforts are being made on unsupervised domain adaptation (UDA) based person re-ID, which targets to adapt the unlabeled target dataset by exploiting information from a labeled source dataset. These UDA person re-ID methods focus on two aspects: (a) applying the domain adaptation technique to reduce the source and target distribution inconsistency [7], [8], [9], [10], [11], (b) assigning the same pseudo-labels on similar target instances for training via clustering, k-NN search, etc. [12], [13], [14]. The former aims to alleviate domain inconsistency between source and target, and the latter tries to make use of the potential relations among target instances and retrain the model with assigned pseudo-labels. Specifically, the latter, i.e., the pseudo-label-based branch, dominates the state-of-the-art methods. They usually consist of two steps: (1) obtaining a primary discriminative model by supervised training with annotated source data; (2) training on the target domain by pseudo-label prediction and fine-tune iteratively.

Despite their promising performance for person re-ID, the clustering-based branch is mainly limited by the clustering accuracy, which cannot always be guaranteed. Therefore, the assigned pseudo-labels for target samples can be noisy, and there is no doubt that training the model with them will result in performance degradation. The main reason is that each identity can be recorded by multi-cameras with varied parameters and environments, which may significantly change the appearance of the identity. In other words, the distribution gaps among cameras make it difficult to identify samples of the same identity, as well as to optimize intra-class feature similarity. Thus, not merely the domain gap between source and target domains but also the distribution gaps between target cameras need to be alleviated for the UDA person re-ID task. However, existing UDA person re-ID works cannot solve these two problems simultaneously, thus their performance is limited in cross-domain re-ID.

In this study, we try to resolve these problems by developing a novel unsupervised domain adaptation model named Intra–Inter Camera Affinity Domain Adaptation (I${}^2$CADA). To better remove noisy pseudo-labels and bridge domain gaps, we decompose I${}^2$CADA into two modules: generative adversarial learning (GAL) module and affinity transfer learning (ATL) module.

Specifically, the GAL module firstly trains a feature extractor with source labeled data, which contains rich discriminative information constrained by cross-entropy loss and hard-batch triplet loss. In this case, the source knowledge is fully exploited, and a basic discriminative model is trained. In order to map the target data to the same space as the source domain and alleviate the global domain inconsistency, we train the feature extractor by adversarial learning [15]. Unlike [9], [16], [17], they utilize a generative adversarial network (GAN) to train one or multiple independent discriminator-generator pairs to generate new images, discard the discriminator-generator pairs, and train the feature extractor with the generated images. Instead, we share the weights of feature extractor between source and target images via generative adversarial learning at the feature level. Compared with the GAN-based method at the image level, applying adversarial learning at the feature level saves computing resources on the one hand and avoids the severe noise produced by the image-level adversarial learning on the other hand. In the ATL module, we utilize the trained feature extractor to calculate the target pseudo-labels, which can be inferred to the confidence score according to the similarity metrics. Moreover, we design intra-camera and inter-camera affinity matrices to remove noisy pseudo-labels based on the ranked confidence scores. We optimize the network by the constructed intra-camera and inter-camera losses using the corresponding matrices. Benefit from our proposed GAL and ATL modules, I${}^2$CADA can mitigate the domain discrepancy between source and target, and remove the noisy pseudo-labels by mitigating the distribution gaps among target cameras.

We test the proposed model on commonly used Market1501 [18] and DukeMTMC-ReID [19] datasets. Extensive experiments show that each component in our approach is valid in boosting the re-ID performance. The entire model consisting of the three modules exhibits the best performance.

Shortly, we elaborate the novelties of our I${}^2$CADA as below:

• We propose a novel UDA model named I${}^2$CADA for person re-ID. It simultaneously considers both the domain discrepancy between source and target domains and distribution gaps among target cameras by the generative adversarial learning (GAL) and the affinity transfer learning (ATL) modules.

• We design a generative adversarial learning (GAL) module to bridge the domain gap between source and target datasets at the feature level, which is computational resource-saving and significant for noise reduction compared to the GAN-based methods. We utilize the generator to produce domain invariant feature representations for source and target person images and involve a discriminator to strengthen the feature learning capability of the generator. Besides, GAL employs cross-entropy loss and triplet loss to make full use of labeled source images.

• We develop an affinity transfer learning (ATL) module to exploit inherent similarities among cross-camera person images in the target domain, which constructs the intra-camera and inter-camera affinity matrices to filter out the noisy pseudo-labels and develop the corresponding losses.

• We implement ablation studies and prove the effectiveness of I${}^2$CADA quantitatively on three commonly used datasets, and the results reveal the excellent performance on UDA-based person re-ID, surpassing recent advanced models.