Improving person re-identification by multi-task learning

Abstract

For person re-identification, the core task is to find effective representations of a person image. As Multi-Task Learning can achieve great performance in seeking robust features, we propose a novel Multi-Task Learning Network (MTNet) with four different losses for person re-identification (re-ID). Our MTNet is an end-to-end deep learning framework, which all the parameters and losses can be jointly optimized. In our method we combine two tasks closely corresponding to person re-identification, pedestrian identity task and pedestrian attribute task, who provide complementary information from different perspective by integrating multi-context informations. Attribute focuses on some special aspects of a person, while identity pays more attention to overall contour and appearance. Meanwhile, both classification and verification losses are employed to optimize the distance of samples. Identification losses are used to construct a large class space, while verification losses are applied optimize the space by minimizing the distance between similar images and maximizing the distance between dissimilar images. In the experiments, our MTNet achieves the state-of-the-art results on two typical datasets Market1501 [1] and DukeMTMC-reID [2].

Introduction

Person re-identification has a potential significance in security application. It is usually considered as an image retrieval issue, which matches person from different cameras and ranks the gallery images according to the similarities. Existing methods mainly focus on extracting robust representations [3], [4], [5], [6], [7], [8], [9], [10], [11] and learning matching functions or metrics [5], [12], [13], [14], [15], [16], [17]. With the excellent performance in computer vision tasks, deep learning has also been adopted to Re-ID community [6], [18], [19], [20], [21] and has gained promising results.

This paper aims at learning to extract robust representations and improving the performance of person re-ID in large-scale datasets. Both person identity and attributes are essential information in surveillance videos. In general, the identity feature can be considered as overall feature, which depends on the overall contour and appearance. The attributes can be considered as features that are only related to some aspects of a pedestrian. Identity feature contains overall information, while attribute feature is more focused on details. So identity feature is more effective for re-id, and attribute feature is more effective for attribute recognition. As mentioned in [22], multi-task learning can obtain more robust feature. These two tasks are also complement with each other. As shown in Fig. 1, the identity task and the attribute task could help each other to achieve better results. For example, the classification system fails in the first query example, due to the similar appearance, such as yellow upper clothing and black lower short pant. Nevertheless, with the gender attribute, the boy in yellow clothing is excluded.

To our knowledge, there are three kinds of methods using deep learning for person re-identification. The classification model [1], [6], [19] devotes to distinguishing the different samples. In these methods, images from different categories in the feature space may be very close to each other, which makes it very difficult and challenging to correctly identify the new samples or new identities. So, the verification model [23], [24] is proposed to minimize the distance between similar images, and maximize the distance between dissimilar ones. However, the verification models have weak abilities in expanding feature space. Besides, a number of previous works [18], [25] treat person re-identification as a binary-category classification task or a similarity regression task. In these works, given an input pair of images, the network determine whether the two images represent the same person.

In our paper, MTNet takes advantage of the two tasks of identity and attribute by combing the two methods of classification and verification. In our framework, the verification technology is embedded in the attributes, which makes up the defect of simple attribute identification model in a certain extent. As shown in the Fig. 1, we define a identity label and a set of attribute labels for every pedestrian (the second boxes). Based on these labels, we use four independent branches to train a robust multi-task network (the third boxes), which further achieves the person re-identification task and person attribute recognition task (the forth boxes).

Our main contributions are summarized as follows:

• We proposed a multi-task learning network (MTNet). It learns an end-to-end CNN embedding for person re-ID and an attribute prediction model simultaneously. As shown in Fig. 2, four different tasks are integrated to MTNet including identity classification, identity verification, attribute classification and attribute verification.

• We employ the verification loss for attribute task, which is the first time in the field of person re-ID task.The attribute verification task not only assists attribute classificatioin task but also promotes the identity verification task.

• We achieve the state-of-the-art results on two large-scale person re-ID datasets Market1501 [1] and DukeMTMC-reID [2].