Pose-guided part matching network via shrinking and reweighting for occluded person re-identification

Abstract

Occluded person re-identification (ReID) is a challenging task, which aims at retrieving an occluded person across multiple non-overlapping cameras. To address this issue, we propose a novel framework named Shrinking and Reweighting Network (SRNet) that jointly learns global features by shrinking and reweights part features for matching in an end-to-end framework. Specifically, we use a strong backbone that combines some effective designs and training tricks to learn the robust and discriminative global features. Even so, there exist noise-related features due to the occlusion, so we utilize the Deep Residual Shrinkage Module (DRS Module) to eliminate unimportant features by automatically determining the soft thresholds. When aligning two groups of part features from two images, we view it as a graph matching problem and design an effectively Reweight Module for Part Matching (RMPM) to learn self-adaptive weights for part features before the part matching stage, the proposed RMPM can alleviate the influence of meaningless part features in the part matching stage. Eventually, extensive experimental results on occluded, partial, and holistic re-id datasets clearly demonstrate that the proposed method achieves competitive performance to the state-of-the-art methods. Specifically, our framework remarkably outperforms state-of-the-art by 8.9% mAP scores on Occluded-Duke dataset. Code is available at https://github.com/chenxiangzZ/SRNet.

Introduction

Person re-identification (ReID) [1], [2], [3] targets on retrieving a pedestrian of interest from non-overlapping camera views. Combining with pedestrian detection and tracking, ReID is widely used in video surveillance, security and smart city.

With the explosive development of deep learning, the performance of ReID has been significantly improved in recent years, however, the previous works [4], [5], [6], [7], [8] did not work well when dealing with occluded datasets, due to the assumption that the entire body of the pedestrian is available. In real-world scenarios, the degree of occlusion is more serious, because pedestrians are more likely to be occluded by static obstacles like trees, cars, and roadblocks, etc. Even in a crowded crowd, most of the bodies of the target pedestrian will be occluded by other pedestrians. Thus, it is essential to seek an effective method to solve the matching problem of person images with occlusion, which is known as the occluded person Re-ID problem [9], [10], [11], [12]. Occluded ReID is more challenging due to the following reasons: ① Part-based features have been proved to be efficient [4], but there exists the problem of misalignment in part-to-part matching, which will be more serious in seriously occluded situations. ② Occlusion will not only reduce the discriminative information of the image but also introduce extra noise. ③ If the part area suffers from occlusion or outliers, etc., hard one-to-one part matching is meaningless and not very robust.

In recent years, some works [11], [12], [13] combined with pose estimation can overcome the ①, which can divide person images to avoid the misalignment by locating the semantic parts of the human body. Specifically, we propose a novel framework named SRNet to solve the ② and ③. In Fig. 1(a), We can see that there are severely occluded scenes in both two datasets, hence the network that extracts global features should focus on the more accurate non-occluded pedestrian areas, because paying attention to the occluded areas will introduce additional noise, which is ②. So we first adopt a strong backbone that introduces multiple tricks to improve the ability to extract global features for occluded Re-ID, as shown in Fig. 1(b, c), the effect of the strong backbone is better than ResNet50 [14]. However, the network will inevitably pay attention to areas besides pedestrian, to alleviate this problem, we further improve the performance by adding the DRS Module [15] to the strong backbone. So as to eliminate the unimportant feature, we set the unimportant features below the soft threshold to 0 by learning the soft thresholds of different channels. The result of the strong backbone with the DRS Module is illustrated in Fig. 1(d), and our improved network focuses on non-occluded pedestrian areas more accurately and concentratedly, and can learn more robust and discriminative features. As for ③, it can be seen from Fig. 1(a) that most of the key-points of pedestrians are occluded. In order to weaken the influence of occluded part features in the part matching stage, we designed a Reweight Module for Part Matching (RMPM) to learn self-adaptive weights for part features, the part features that the network focuses on can get larger weights. We do not use the key-points confidence to determine whether the part is occluded or not, because the confidence is completely dependent on the result of the pose estimation model, and it only represents the probability of the key-point, and cannot be equal to the degree that the network should focus on. RMPM make the part matching stage more robust and reduce the influence of the meaningless part features in the point-to-point corresponding graph matching.

In conclusion, the main contributions of this paper are summarized as follows:

• We propose a novel network SRNet that can learn more robust and highly discriminative global features, and obtain soft thresholds through DRS Module to eliminate the noise-related features.

• We design an effective Reweight Module for Part Matching (RMPM). By assigning adaptive weights to each part feature, meaningless part features (occluded, outlier) are weakened, which is beneficial to part matching and learn robust alignment.

• Extensive experimental results on occluded and partial re-id datasets clearly demonstrate that the proposed method achieves remarkable performance to the state-of-the-art methods. In addition, our method has also achieved competitive performance on holistic datasets.