Cross-level reinforced attention network for person re-identification

Highlights

• The fusion of different level features can lay the foundation for generating a more discriminative attention map.

• The combination of hard attention and soft attention can reduce the influence of interference information on the model.

• The improved attention module works on both space and channel to achieve more comprehensive weight adjustments.

• The lightweight person re-identification network achieves outstanding performance.

Abstract

Attention mechanism is a simple and effective method to enhance discriminative performance of person re-identification (Re-ID). Most of previous attention-based works have difficulty in eliminating the negative effects of meaningless information. In this paper, a universal module, named Cross-level Reinforced Attention (CLRA), is proposed to alleviate this issue. Firstly, we fuse features of different semantic levels using adaptive weights. The fused features, containing richer spatial and semantic information, can better guide the generation of subsequent attention module. Then, we combine hard and soft attention to improve the ability to extract important information in spatial and channel domains. Through the CLRA, the network can aggregate and propagate more discriminative semantic information. Finally, we integrate the CLRA with Harmonious Attention CNN (HA-CNN) and form a novel Cross-level Reinforced Attention CNN (CLRA-CNN) to optimize person Re-ID. Experiment results on several public benchmarks show that the proposed method achieves state-of-the-art performance.

Introduction

Person re-identification (Re-ID) targets to find specific pedestrian through non-crossing surveillance cameras deployed in different locations [1], [2], [3], [4]. It has very important application in the field of video surveillance. However, there are still many challenges to accurately distinguish specific targets from different surveillance scenarios. The pedestrian images often have the problems of background clutter, changeable lighting conditions, serious occlusion and large pose difference [5], [6], [7], [8], all of which have a large impact on pedestrian matching. Numerous works have been proposed to relieve this dilemma. Among them, the attention-based methods demonstrate convincing performance.

Attention mechanism provides an effective method to enhance Re-ID networks. According to the difference in calculation method of attention mechanism, it is mainly divided into two types: soft attention and hard attention. The existing methods are usually based on the soft attention [1], [9], [10], [11], [12], which aims to find out the probability distribution relations among a series of elements and obtain an attention map for input features. Each position of the input features is assigned a probability, and the result of the probability distribution reflects the importance of different elements. All the information is reweighted adaptively before being aggregated. In this way, we can extract discriminative information and avoid interference with unimportant information, thereby improving the model. Unlike the general research on soft attention, only a few works are developed on the basis of hard attention. Hard attention only processes the information that is considered most relevant and completely ignores other parts. Using hard attention, we can not only eliminate the interference of irrelevant information, but also obtain higher computational efficiency [9], [13].

According to the difference of attention domain, attention mechanism also can be divided into spatial attention [14] and channel attention [10], [1]. As the name implies, spatial attention can model the interdependence of different spatial positions. By introducing the spatial attention, the convolution layer can sense the whole spatial distribution relationship. Similar to spatial attention, channel attention models the relationship between different channels [10]. Each channel map of high-level features can be viewed as a response to specific class, and different semantic responses are correlated. The interdependence between channel maps can be used to improve the representation of specific semantic features.

Although a lot of attention-based works have made great progress, there are still some defects. First, most of the works [3], [10], [11], [12], [14], [15], [16] tend to involve a single type of attention mechanism. Due to these limitations, they often have difficulty in obtaining the optimal result of weight distribution. For soft attention, it is impossible to completely remove the interference information. Confidence assignment is still a challenge. Meanwhile, hard attention chooses features in a discrete way. This leads to the fact that we cannot choose features by means of some gradient-based learning methods. On the other hand, some works only consider spatial attention or channel attention and cannot make full use of the information.

To solve these issues, we design a Cross-level Reinforced Attention (CLRA). It mainly consists of the Cross-level Feature Fusion (CLFF) module and the Reinforced Attention (RA) module. Low-level features contain more abundant spatial information, while the semantic information of high-level features is more representative [17]. The CLFF fuses features of different levels, enabling subsequent attention module to grasp the dependencies between features more accurately. The RA can fuse soft attention with hard attention to make full use of their respective strengths, which achieves the attention reinforcement of both in the aspect of space and channel. In this work, hard attention helps soft attention to select features, enhancing the weight distribution effect of soft attention. Meanwhile, soft attention provides a stable environment and motivation information for training hard attention. This method can improve the prediction quality of soft attention mechanism and the trainability of hard attention mechanism. Using CLRA, we can enable the network to focus on more discriminating pedestrian features and alleviate the interference of irrelevant factors. The contributions of this work are:

• We design a Cross-level Feature Fusion (CLFF) module, which aims to generate more discriminative features by combining the features of different levels. This module lays the foundation for generating a more discriminative attention map.

• We design a Reinforced Attention (RA) module, which contains two parts: Reinforced Spatial Attention (RSA) and Reinforced Channel Attention (RCA). It combines the characteristics of soft attention and hard attention, and makes use of hard attention to assist in strengthening soft attention.

• We combine the CLFF and RA to form a Cross-level Reinforced Attention (CLRA) module and integrate it into HA-CNN [1] to optimize person Re-ID. We validate the proposed method on several common datasets, Market-1501 [18], DukeMTMC-ReID [19], CUHK03 [20] and MSMT17 [21]. The results verify that our method achieves excellent performance.