Joint usage of global and local attentions in hourglass network for human pose estimation

Abstract

Human pose estimation is a challenging research task in the field of computer vision. The current mainstream work has made great progress in pose estimation, but these works still do not pay enough attention to the negative impact of background on human pose estimation. In this work, we propose a human pose estimation framework characterized by the joint usage of both global and local attention module in an hourglass backbone network. The global attention module aims to reduces the negative impact of background. The local attention module is designed to help refine each joint. We tested our method on two benchmark datasets for human pose estimation, and the experimental results show that the proposed model is superior to current mainstream algorithms.

Introduction

The purpose of human pose estimation is to locate human body joints, e.g. the head, hips, knees, and ankles, from images. Human pose estimation plays an important role in analyzing human behavior based on images or videos. Accurate and efficient human pose estimation may facilitate various applications such as human action recognition [5], [6], [40], person ReID [3], human–computer interaction [24], [35] and video object tracking [32]. However, due to the volatile camera view angle and complex human posture, human pose estimation remains a challenging task after decades of study.

One overlooked problem by the current methods are the ignorance of the negative impact of background. The problem usually occurs in the case of complex background. In addition, the accurate of pose estimation still needs to be improved. To tackle the above two problems, a model could consider representing features from multiple scales. The higher-level features are computed with a large receptive field and thus can capture more global information about the image content, while the lower-level features are obtained with a small receptive field and contain fine-grained information about local regions. The Stacked Hourglass Network (SHN) [1] is a typically used multi-scale feature extraction model in pose estimation, in which the higher-level layer focuses on learning the overall human poses and the lower-level layer concentrates on fine-grained detection of local joints. Therefore, we believe the ‘SHN’ is beneficial for solving the above two problems.

In this paper, we propose a novel framework named as Global and Local Attention Strengthened Stacked Hourglass Network (GLASS-HN) for human pose estimation. This model is characterized by incorporating a Global Attention Module (GAM) and a Local Attention Module (LAM) into the backbone stacked network. The ‘GAM’ is realized by a non-local attention map inserted before each hourglass block. The ‘LAM’ is a self-attention scheme applied to each channel of feature map derived from convolving the output of hourglass block with a distinct kernel. Since the non-local attention mechanism can capture long-range dependencies directly by computing interactions between any two positions, it is beneficial to distinguish a person from his background, thereby reduce negative impact of background. We convolve the output of hourglass block with different kernels such that each channel can focus on a specific body joint with properly labeled training samples, therefore the LAM can help to refine each joint of person and experiments have proved that it has achieve better results.

The contributions of our work are fourfold:

• We propose a new architecture for pose estimation, called Global and Local Attention Strengthened Stacked Hourglass Network (GLASS-HN) in which jointly use the global and local attention mechanisms in Hourglass Network.

• We propose the Global Attention Mechanism(GAM). global attention block is realized by utilize a non-local attention map, which can reduces the negative impact of background.

• We propose the Local Attention Mechanism(LAM). local attention block is realized by channel-wise self-attention model, which has help to refine each joint.

• The experimental results show that the GLASS-HN is superior to current mainstream algorithms on MPII Human Pose dataset and Leeds Sports Poses (LSP) datasets.

The rest part of this paper is organized as follows: Section 2 introduces the related works, Section 3 present the proposed method in detail, Section 4 demonstrates and analyzes the experimental results and Section 5 concludes this paper.