Gaussian process motion graph models for smooth transitions among multiple actions☆
Highlights
► We propose a unified model for human motion prior with multiple actions. ► Sample sequences are interpolated for synthesizing possible action transitions. ► Kinematically-realistic poses are synthesized with statistical models of samples. ► Short path search via the synthesized poses provides the smooth action transitions.
Introduction
Motion prior is widely used in human pose tracking, simulation, and synthesis for accuracy and robustness. Motion prior is obtained from samples of real human motion. Motion datasets including the sample sequences have been distributed for motion modeling and evaluation in Computer Vision [1] and Graphics [2], [3] communities. Different kinds of actions (e.g. walking, jogging, dance) were recorded independently in these datasets. The motion model of each action can be leveraged for analyzing that action.
For efficiently and adaptively using motion prior of multiple actions, a unified motion model of these actions is useful. Different actions are smoothly transited from one to another (e.g. from walking to jogging) in a natural scenario, while they are recorded independently in motion datasets. Since it is not practical to record a huge variety of possible transitions among all of the different actions, modeling the smooth transition is important. Smooth transition paths between the different actions enable successful pose modeling over the different actions.
This paper proposes a human motion model for smooth transitions among elemental actions in dataset and its application to pose tracking. After introducing related work (Section 2) and existing models for motion modeling and interpolation (Sections 3 Gaussian process dynamical models, 4 Motion graphs), Section 5 reveals the problems of naive integration of the existing models. Section 6 describes the proposed model. Experimental results of pose tracking with the proposed model are presented in Section 7, and we conclude the paper in Section 8.
Section snippets
Related work
The pose of a human body is modeled by a set of joint positions/angles, which can be measured by a motion capture system. The motion has been modeled by various ways: interpolation [6], Gaussian mixture models [7], HMM [8], Variable Length Markov Model [9], exemplar (retrieval) model [4], autoregressive model [11], the mixtures of autoregressive models [5], and manifold [10]. The motion models are useful for solving the short-lasting ambiguities between a body shape and its pose due to
Overview
Gaussian Process Dynamical Models (GPDM) [16] (Fig. 1) provide us dimensionality reduction and temporally smooth transition in the low-dimensional latent space. Inherence of the GP allows us to optimize the latent space increasing its generalization and conformity with human body structure and kinematics. GPDM with a D-dimensional observation space Y (i.e. Pose observation space in Fig. 1), which is inherently nonlinear, and its d-dimensional latent space X (i.e. Pose latent space in Fig. 1) is
Overview
Motion graphs [24] provide new transition paths between sample sequences with good connectivity and motion quality, which conflict with each other; good connectivity means that transitions are synthesized as many as possible, while good motion quality is achieved by transitions only between similar poses that can make smooth paths.
Motion graphs consist of pose data (i.e. nodes) and possible transitions between them (i.e. directed edges). The state-of-the-art motion graphs proposed in [26] makes
Strengths and limitations of interpolation for new transitions in the latent space
Sections 3 Gaussian process dynamical models, 4 Motion graphs show “action transition with no transition samples in the latent space” and “pose interpolation among different actions in the observation space”, respectively. In this section, interpolation in the latent space by motion graphs is verified. In constructing motion graphs, (1) similarity between two latent variables is measured by the Euclidean distance and (2) interpolation between sample poses is computed linearly, in the latent
New transitions in the GP latent models
The goal of this work is to integrate the advantages of GPDM and motion graphs while avoiding unreasonable poses. We call the resulting latent variable models Gaussian Process Motion Graph Models, GPMGM. GPMGM evaluates pose similarity, smoothness, and distribution both in the observation and latent spaces in order to avoid unreasonable poses. The evaluation is integrated into the shortest path search algorithm for establishing new transition paths.
The steps of GPMGM optimization are described
Algorithm
GPMGM was applied to motion prior in human pose tracking. Pose tracking was achieved by image-to-pose regression with particle filtering, whose overview is shown in Fig. 12.
In the learning process, pose data (i.e. joint angles) at each frame is captured with its respective image features (3D volume features [30] and 2D shape contexts [31] in our experiments). for learning an image-to-pose regression function. Motion prior is obtained from the temporal pose data.
In the tracking process, the
Concluding remarks
We proposed the motion models of multiple actions, GPMGM. GPMGM is learned from independently captured action sequences so that potential transition paths between them are synthesized. Since the transition paths are synthesized in the motion-specific latent space, they reflect the human-body kinematics of the target actions. GPMGM is applicable to any motions because transition paths can be established among any motion trajectories.
In this work, GPMGM is constructed by relying on reasonable
References (34)
- et al.
Temporal motion models for monocular and multiview 3D human body tracking
CVIU
(2006) - et al.
Achieving good connectivity in motion graphs
Graphical Models Journal
(2009) - L. Sigal, M.J. Black, HumanEva: synchronized video and motion capture dataset for evaluation of articulated human...
- CMU Graphics Lab Motion Capture Database:...
- et al.
Articulated mesh animation from multi-view silhouettes
ACM Transactions on Graphics
(2008) - H. Sidenbladh, M.J. Black, L. Sigal, Implicit probabilistic models of human motion for synthesis and tracking, in:...
- A. Agarwal, B. Triggs, Tracking articulated motion using a mixture of autoregressive models, in: ECCV,...
- et al.
People tracking based on motion model and motion constraints with automatic initialization
Pattern Recognition
(2004) - M. Brand, Shadow puppetry, in: ICCV,...
- S. Hou, A. Galata, F. Caillette, N. Thacker, P. Bromiley, Real-time body tracking using a Gaussian process latent...
Nonlinear dimensionality reduction by locally linear embedding
Science
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Probabilistic non-linear principal component analysis with Gaussian process latent variable models
Journal of Machine Learning Research
Gaussian process dynamical models for human motion
PAMI
Cited by (0)
- ☆
This paper has been recommended for acceptance by J.K. Aggarwal.