Articulated motion reconstruction from feature points
Introduction
Visual interpretation of non-rigid articulated motion has lately seen somewhat of a renaissance in computer vision and pattern recognition. The motivation for directing existing motion analysis of rigid objects towards non-rigid articulated objects [1], [2], especially human motion [3], [4], [5], is driven by potential applications such as human–computer interaction, surveillance systems, entertainment and medical studies. A large body of research, dedicated to the task of structure and motion analysis, utilises feature-based methods regardless of parametrisation by points, lines, curves or surfaces. Among these, concise feature-point representation, advantageously abstracting the underlying movement, is usually used as an essential or intermediate correspondence towards the end-product of motion and structure recovery [6], [7], [8].
In the context of vision cues via feature-point representation, the spatio–temporal information is notably reduced to a sequence of unidentified points moving over time. To determine the subject's structure and therefore its underlying skeletal-style movements for the purpose of high-level recognition, two fundamental problems of feature-point tracking and identification need to be solved. Tracking feature points in successive frames has been investigated extensively in the literatures [9], [10], [11], [12]. However, the identities of the subject feature points are not obtainable from inter-frame tracking alone.
Feature-point identification requires the determination of which point in an observed data frame corresponds to which point in its model, thus allowing recovery of structure. The task addresses the difficult problem of automatic model matching and identification, crucial at the start of tracking or on resumption from tracking loss. Currently, most tracking approaches simplify the problem to incremental pose estimation, relying on manual model fitting at the start of tracking, or on an assumption of initial pose similarity and alignment to the model, or on pre-knowledge of a specific motion from which to infer an initial pose [5], [13]. In this sense, the general recovery of non-rigid articulated motion solely from feature points still remains an open problem. There is a relative dearth of algorithmic self-initialisation for articulated motion reconstruction from only a collection of sparse feature points.
Motivated by these observations, we present a dynamic segment-based hierarchical point matching (DSHPM) algorithm to address self-initialising articulated motion reconstruction from sparse feature points. The articulated motion we are considering describes general segmental jointed freeform movement. The motion of each segment can be considered as rigid or nearly rigid, but the motion of the object as a whole is high-dimensionally non-rigid. In our work, the articulated model of an observed subject is a priori known, suggesting a model-based approach. As a general solution to the problem, the algorithm only assumes availability of feature-point motion data, such as obtained in our experiments via a marker-based motion capture system. We do not make the usual simplifying assumptions of model-pose similarity or restricted motion class for tracking initialisation, nor do we require absence of data noise. The algorithm aims to establish one-to-one matches between the model point-set and its freeform motion data to reconstruct the underlying articulated movements in buffered real-time.
Section snippets
Related work
The problem of automatically identifying feature points to retrieve underlying articulated movement can be inherently difficult for a number of reasons: (1) the possibility of globally high dimensionality to depict the articulated structure; (2) relaxation of segment rigidity to allow for limited distortion; (3) data corruption due to missing (occluded) and extra (via the process of feature extraction) data; (4) unrestricted and arbitrary poses in freeform movements; (5) requirements of
Framework of the model-based DSHPM algorithm
The generic task under consideration arose from the need to identify feature-point data to reconstruct underlying skeletal structure of freeform articulated motion. We assume the data capture rate is sufficiently high, as demanded in most real-world applications. This allows the obtaining of feature-point trajectories in successive frames. However, the identities of feature points (or trajectories) are not known.
The DSHPM algorithm
Identification is carried out hierarchically segment by segment in a chosen key-frame containing e.g. over 90% of the model points (Section 4.2), or in a key-frame range when necessary, taking advantage of a dynamic scheme (Section 4.3). In order to make temporal coherence of motion cues exploitable and reduce the search space, feature-point pre-tracking and pre-segmentation are carried out prior to segmental identification (Section 4.1).
Experiments
The algorithm has been implemented in Matlab. We tested it on articulated models, such as human and robot manipulators in various low densities and distributions of feature points. Human motion representing a typical articulated motion with segments of only near-rigidity makes the identification task more difficult than in the case of robot manipulator motion with rigid segments. To reflect this challenge, we report in this section experimental results on real-world human motion capture and its
Conclusion
The proposed dynamic segment-based hierarchical point matching (DSHPM) algorithm addresses a general and currently open problem in pattern recognition: non-rigid articulated motion reconstruction from low-density feature points. The algorithm has a crucial self-initialisation phase of pose estimation, benefiting from our previous work [35], [37]. In the context of a dynamic sequence, the DSHPM algorithm integrates a key-frame-based dynamic hierarchial matching with inter-frame tracking to
Acknowledgements
All model and motion data used in our experiments were obtained by a marker-based optical motion capture system—Vicon-512, installed at the Department of Computer Science, UWA. Some motion trials analysed in this paper were captured for the game project “Dance: UK” in collaboration with Broadsword Interactive Ltd. [39].
About the Author—BAIHUA LI received the B.S. and M.S. degrees in electronic engineering from Tianjin University, China and the Ph.D. degree in computer science from the University of Wales, Aberystwyth in 2003. She is a Lecturer in the Department of Computing and Mathematics, Manchester Metropolitan University, UK. Her current research interests include computer vision, pattern recognition, human motion tracking and recognition, 3D modelling and animation.
References (39)
- et al.
A survey of medical image registration
IEEE Eng. Med. Biol. Mag.
(1998) - et al.
A survey of computer vision-based human motion capture
Comput. Vision Image Understanding
(2001) - et al.
Recent developments in human motion analysis
Pattern Recognition
(2003) Reconstruction of articulated objects from point correspondences in a single uncalibrated image
Comput. Vision Image Understanding
(2000)- et al.
A survey of free-form object representation and recognition techniques
Comput. Vision Image Understanding
(2001) - et al.
Efficient algorithms for robust feature matching
Pattern Recognition
(1999) - et al.
A parametric deformable model to fit unstructured 3D data
Comput. Vision Image Understanding
(1998) - et al.
A new point matching algorithm for non-rigid registration
Comput. Vision Image Understanding
(2003) The measurement of human motion: a comparison of commercially available systems
Human Movement Sci.
(1999)- et al.
Reconstruction of segmentally articulated structure in freeform movement with low density feature points
Image and Vision Comput.
(2004)
An efficient implementation of Reid's multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking
IEEE Trans. Pattern Anal. Mach. Intell.
A generalized S-D assignment algorithm for multisensor–multitarget state estimation
IEEE Trans. Aerosp. Electron. Syst.
Resolving motion correspondence for densely moving points
IEEE Trans. Pattern Anal. Mach. Intell.
Feature point correspondence between consecutive frames based on genetic algorithm
Int. J. Robot. Autom.
Cited by (21)
Efficient point pattern matching algorithm for planar point sets under transform of translation, rotation and scale
2014, Applied Mathematics and ComputationCitation Excerpt :In another work, McAuley et al. [13] presented a novel graph with better performance than that of [12], and exploited it to find matched point sets. In [14], Li et al. proposed a dynamic segment-based hierarchical point matching algorithm for self-initialising articulated motion reconstruction from sparse feature points. Recently, Bhowmick et al. [15] designed a novel data structure called “angular tree” for point pattern matching.
Real time pose recognition of covered human for diagnosis of sleep apnoea
2010, Computerized Medical Imaging and GraphicsCitation Excerpt :As spatial shape features generated from obscured appearance cues may include false positives and limited, fragmented and noisy dynamic motion cues cannot be relied on to define action performed by the subject, the idea of the proposed method is to combine both static and dynamic features to suppress false detection and improve pose estimation on the covered human. Spatio-temporal approaches have been shown to be useful in overcoming self-occlusion and image noise in recent research [19,21,31]. These methods exploit temporal coherency of feature points.
Motion trajectory reproduction from generalized signature description
2010, Pattern RecognitionFast detection of marker pixels in video-based motion capture systems
2009, Pattern Recognition LettersPassive Binocular Optical Motion Capture Technology Under Complex Illumination
2023, Journal of Shanghai Jiaotong University (Science)Toward accurate real-time marker labeling for live optical motion capture
2017, Visual Computer
About the Author—BAIHUA LI received the B.S. and M.S. degrees in electronic engineering from Tianjin University, China and the Ph.D. degree in computer science from the University of Wales, Aberystwyth in 2003. She is a Lecturer in the Department of Computing and Mathematics, Manchester Metropolitan University, UK. Her current research interests include computer vision, pattern recognition, human motion tracking and recognition, 3D modelling and animation.
About the Author—QINGGANG MENG received the B.S. and M.S. degrees in electronic engineering from Tianjin University, China and the Ph.D. degree in computer science from the University of Wales, Aberystwyth in 2003. He is a Lecturer in the Department of Computer Science, Loughborough University, UK. His research interests include biologically/psychologically inspired robot learning and control, machine vision and service robotics.
About the Author—HORST HOLSTEIN received the degree of B.S. in Mathematics from the University of Southampton, UK, in 1963, and obtained a Ph.D. in the field of rheology from University of Wales, Aberystwyth, UK, in 1981. He is a Lecturer in the Department of Computer Science, University of Wales, Aberystwyth, UK. His research interests include motion tracking, computational bioengineering and geophysical gravi-magnetic modelling.