Masking for better discovery: Weakly supervised complementary body regions mining for person re-identification

Highlights

• Weakly supervised data augmentation based network for fine-grained person re-identification.

• Modeling the spatial relationship of a fine-grained region for person re-identification.

• Discovering more complementary feature representation in a weakly supervised fashion.

Abstract

Person re-identification still facing several challenges related to many factors such as complex poses, occlusion, misalignment and bad detection. Recent works in the literature focus on extracting local information from the human body but most of them rely on full supervision during training.

We propose in this paper a new end-to-end trainable neural network, named Attention Dropping Network (ADN), for diverse rich visual cues discovery without extra human semantic parsing. ADN aims to find fine-grained local information to address the shared person re-identification challenges. Concretely, our network consists of two branches. The Attention Global Branch learns pixel-level local regions based on a fine-grained attention mechanism while the Feature Dropping Branch learns additional missed features in a weakly supervised manner. The fine-grained attention mechanism allows our model to be robust to complex pose variations and to avoid the redundant backgrounds. Extensive experiments over three benchmark datasets (Market-1501, DukeMTMC-reID, and CUHK03) demonstrate the effectiveness and robustness of the proposed network in handling the problems of complex poses, misalignment and occlusions.

Introduction

Person re-identification (Re-ID) aims to find specific pedestrian through non-overlapping surveillance cameras deployed at different locations (Gong et al., 2014). It has a very important application in the field of video surveillance (Bialkowski et al., 2012). Nonetheless, person Re-ID still facing many challenges to accurately differentiate specific targets from different surveillance scenarios. The pedestrian images often have background clutter problems (Ghorbel et al., 2019, Tian et al., 2018), illumination variation (Huang et al., 2019), strong occlusion (Huang et al., 2018) and unconstrained pose (Cho & Yoon, 2016). All these problems can affect the performance of person Re-ID. In fact, images that belong to the same person usually come under significantly different viewpoints and poses which make the intra-class variances high (Wang et al., 2015). In addition, images that belong to different classes can be very similar apart from some minor differences making the inter-class variances very low. The main challenge of each person Re-ID system is then to build a rich and suitable feature representation to be robust to all problems cited above and to overcome the inter-class confusion and the intra-class variation. Apart from the difficulties of extracting features, the small sample size in some person Re-ID databases can be an obstacle to learn a good model of each identity’s intra-class variability (Gong et al., 2014).

In the literature, several works have been devoted to deal with these specific challenges. These last years, the convolutional neural networks (CNNs) are widely used to extract many level features, which contributes to having a more relevant representation for each image. Wang et al. (2018) showed that Deep Learning (CNN-based) methods achieve higher recognition rates than the traditional methods. Some existing methods of person Re-ID learn the discriminative features and similarity measurement (Ahmed et al., 2015, Hermans et al., 2017, Zheng et al., 2017). Several recent works have shown that it is crucially important to embed local information of the person’s images to deal with large variances in appearance (Huang et al., 2017, Li et al., 2017, Yao et al., 2019). Similarly, others depend on additional spatial information to prevent background clutter and arbitrary misalignment (Wei et al., 2018, Zheng et al., 2019). The semantic segmentation is also used to guide a model to extract features from a specific region of the image (Gao et al., 2019, Ghorbel et al., 2019, Ghorbel et al., 2020, Song et al., 2018), but such methods require almost large labeled datasets and considerable additional computational costs. Recently, several works tend to use the attention mechanism which provides an effective method to enhance Re-ID networks’ performances (Jiang et al., 2020, Li et al., 2019, Li, Zhu et al., 2018, Si et al., 2018, Xu et al., 2018). However, most of these works are overlooking the fine-grained details that may contain important distinguishing features. In the same direction, weakly supervised learning (Cao et al., 2015, Zhang et al., 2018), which allows to find the most discriminative region of an image using only image level labels, has gained much popularity as a solution to address the problem of labeled data scarcity in computer vision. Such based methods have proven their effectiveness in object localization (Gao et al., 2018, Zhang, Yang et al., 2018), but they only cover the most discriminative part of an object and not the entire object region. Other works have been devoted to design re-ranking methods (Mansouri et al., 2021, Zhong et al., 2017) in order to enhance the person Re-ID performances by resorting the ranking list of any baseline method.

In this work, we propose the Attention Dropping Network (ADN), a weakly supervised method for diverse rich visual cues discovery without extra human semantic parsing. ADN extracts complementary visual features which can enhance traditional global representation for person re-identification. Fig. 1 shows two images of the same person with different pose and different background. We show in this figure how ADN can overcome the specific person Re-id challenges by focusing on the most discriminating region of the image. In addition, our proposed network allows to focus on finer details of the person image and then extract more discriminating features. Our method learns pixel-level local regions based on a fine-grained attention mechanism, that aims to extract more useful fine-grained foreground features and reduce the impact of redundant backgrounds to help to distinguish persons who are dressed in very similar clothes. The Attention Dropping Network is a two-branch network consisting of an Attention Global Branch (AGB) and a Feature Dropping Branch (FDB) where the attention-based dropping data augmentation is applied. The AGB aims to extract fine-grained features based on $M$ local attention maps. In this branch, a bilinear pooling module is used to model the local pairwise feature interactions, based on the local attention maps, across every channel, and keep saving the spatial relationship. Since that attention mechanism focuses on the most relevant features, the AGB branch can skip some less important features that can contribute to the final prediction. In order to remedy this problem, we considered a second branch, named FDB. In Additional local detailed features are learned through the FDB in a weakly supervised manner. Explicitly, through the FDB, we randomly select one attention map and erase out of the image the most discriminative region according to this attention map. This operation is made during the training phase in order to encourage the network to learn from the extent of the image that can contain other useful features that were ignored by the first branch.

The main contributions of this paper are as follows:

(1) Modeling the spatial relationship of a fine-grained region by the use of bilinear pooling operation between the most discriminative region’s attention map and the feature maps extracted from the whole image.

(2) Discovering a more complementary feature representation by the use of the Feature Dropping Branch which is an attentive feature learning module operating in a weakly supervised fashion.

(3) Our proposed network is trained in an end-to-end fashion and achieves state-of-the-art performances over three public person re-identification datasets.