Multiscale Spatial and Temporal Learning for Human Motion Prediction

Abstract

Human motion prediction is a classic computer vision task that has numerous applications in the field of autonomous driving, human-computer interaction and motion synthesis. Recently, a large number of motion prediction methods employ graph convolutional networks to encode the spatial relationships among joints within a pose, which ignore the impact of adjacent poses in the encoding process, resulting in unsatisfactory extraction of spatial information. Moreover, the recurrent neural network is difficult to capture long-term temporal dependencies. To solve these challenges, we design a prediction model that consists of two crucial components. Firstly, A novel spatio-temporal graph convolution module utilizes multi-graph convolution layers to capture the spatial correlation information of the current frame and adjacent frames simultaneously. Secondly, a multiscale temporal encoding module consists of a series of temporal convolutions with different dilation rates to establish various temporal relationships between poses at both short-term and long-term temporal distance. We evaluate the proposed method on the largest 3D human motion capture dataset (i.e., Human 3.6 Million). Extensive experimental results show that our model achieves state-of-the-art performance in both short and long-term predictions compared with existing methods.