Person re-identification with part prediction alignment

Highlights

• We propose a part-based person feature extraction network with Part Prediction Alignment(PPA), the network does not need the external datasets and pose estimator, and this will reduce the complexity of the training process.

• We adopt the teacher–student network for global–local feature extraction. In this way, the extracted features will be more discriminative and achieve a higher score in the testing phase.

• Experiments on three datasets demonstrate that the proposed network in this paper effectively improves the re-id accuracy.

Abstract

The key to success of person re-identification(re-id) is extracting the discriminative person features. Various part-level feature extraction methods are proposed to capture local person features for re-id. A prerequisite of part feature extraction is that each part should be well located. We believe that ID predictions in different parts of the same image should be consistent. Instead of using the external dataset and pose estimator for guiding, we propose a re-id model with Part Prediction Alignment (PPA), which aims at aligning the predicted distributions between each part. Due to the global feature and local feature contains different spacial information, we consider that the combination of two sides will further improve the detection effect. Therefore, in this paper we adopt the teacher–student training strategy based on PPA for global–local feature extraction, and the global feature extraction branch as a teacher to guide the training of local feature branch. Experimental results on Market-1501, DukeMTMC-reID and CUHK03 (including CUHK03_Detected and CUHK03_Labeled) datasets confirm the effectiveness of our proposal, we achieve Rank1 with 92.4%, 85.1%, 65.5%, 69.2% on Market-1501, DukeMTMC-reID, CUHK03_Detected and CUHK03_Labeled, respectively.

Introduction

Person re-identification (re-id) aims at identifying an interest person at other places, it is a challenging task in computer vision (Sun et al., 2018, Binghui et al., 2019, Ruibing et al., 2019, Tianlong et al., 2019). Recently, deep learning becomes the most popular method in computer vision community due to its high discriminative ability, such as object detection, object tracking and person re-id (Junwei et al., 2018, Yi et al., 2019, Zhang et al., 2020, Wang et al., 2018a). Many state of the art re-id models achieve higher accuracy based on deep-learned features (Wei et al., 2017, He et al., 2018, Qian et al., 2018, Cheng et al., 2020, Jieru et al., 2019, Wang et al., 2019). Person feature extraction can be roughly divided into two aspects: global based feature extraction and local based feature extraction. Global features focus on overall information but ignore the spatial structure of a person, so in recent years, many re-id methods mainly extracting local features for re-identification.

The key to extract discriminative local features is that every part should be located accurately (Wei et al., 2017). This relies on pedestrian external information, pose estimator and human pose estimate datasets are needed for information extraction. Pose estimator and external datasets will undoubtedly increase the training complexity, so presently, some re-id models try to abandon the pose cues and achieve competitive accuracy (Sun et al., 2018, Zhang and Huang, 2018, Kumar et al., 2017, Zheng et al., 2017). There are some partition strategies in Fig. 1. The purpose of accurately locate is to enhance the internal consistency of parts, and then the ID of each part will be predicted precisely. Since the ultimate goal of pose estimate is to improve the prediction accuracy of each local area, we reconsider this problem from the perspective of each part ID prediction, aim at aligning the prediction results of each part and then enhance the prediction consistency of parts.

In the re-id community, person images are usually divided into six parts (head, upper body, lower body, upper legs, lower legs and feet). We believe that for the same person images, no matter how many parts are divided, the ID predictions of each part should be the same. However, during the training process, the ID predictions of parts will be different due to the misalignment of person images. As we can see in the second sub-picture of Fig. 1(d), part 1 only contains background, therefore, in ID prediction step the deep learning network will be difficult to judge the ID of part 1. Based on the above considerations, this paper proposes a network with Part Prediction Alignment (PPA) to extract the part-level features for re-id. The architecture of the network is concise, with slight modifications on the ResNet-50 network.

Global features contain the global space information of an image, but lack person’s spatial structure cues, while local features are opposite to the global features. We consider the motivation that if we can combine the advantages of two sides, the extracted features will be more discriminative and achieve higher accuracy. Therefore, in this paper we propose to use teacher–student network to extract the global–local person features for re-id. The structure of teacher–student network in this paper is shown in Fig. 3, the global feature network as a teacher and guide the learning of local feature network. In this paper, we take a whole image as the input and output the person’s feature. Feature extracted in this way will include two sides of information, which will benefit the testing phase.

Our contributions can be summarized as follows:

(1) We propose a part-based person feature extract network with Part Prediction Alignment (PPA), the network does not need the external datasets and pose estimator for guiding, and this will reduce the complexity of the training process.

(2) We adopt the teacher–student network for global–local feature extraction, the extracted features not only contain the global space information but also the spatial structure cues of person. In this way, the extracted features will be more discriminative and achieve a higher score during testing.

(3) Experiments on three datasets including Market-1501, DukeMTMC-ReID and CUHK03 demonstrate that the proposed network in this paper effectively improves the re-id accuracy.