Human 3D motion tracking from video is an emerging research field with many applications demanding highly detailed results. This chapter surveys a high quality generative method, which employs the person’s silhouette extracted from one or multiple camera views for fitting an a priori given 3D body surface model. A coupling between pose estimation and contour extraction allows for reliable tracking in cluttered scenes without the need of a static background. The optic flow computed between two successive frames is used for pose prediction. It improves the quality of tracking in case of fast motion and/or low frame rates. In order to cope with unreliable or insufficient data, the framework is further extended by the use of prior knowledge on static joint angle configurations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agarwal A. and Triggs B. Recovering 3D human pose from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1):44-58, Jan. 2006.
Alvarez L., Weickert J., and Sánchez J. Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision, 39 (1):41-56, Aug. 2000.
Anandan P. A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision, 2:283-310, 1989.
Besl P. and McKay N. A method for registration of 3D shapes. IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 12:239-256, 1992.
Black M.J. and Anandan P. The robust estimation of multiple motions: para-metric and piecewise smooth flow fields. Computer Vision and Image Under-standing, 63(1):75-104, Jan. 1996.
Blake A. and Zisserman A. Visual Reconstruction. MIT Press, Cambridge, MA, 1987.
Bregler C. and Malik J. Tracking people with twists and exponential maps. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 8-15, Santa Barbara, California, 1998.
Bregler C., Malik J. and Pullen K. Twist based acquisition and tracking of animal and human kinematics. International Journal of Computer Vision, 56(3):179-194, 2004.
Brox T., Bruhn A., Papenberg N., and Weickert J. High accuracy optical flow estimation based on a theory for warping. In T. Pajdla and J. Matas, editors, Proc.8th European Conference on Computer Vision, volume 3024 of LNCS, pp. 25-36. Springer, May 2004.
Brox T., Rosenhahn B., Cremers D., and Seidel H.-P. High accuracy optical flow serves 3-D pose tracking: exploiting contour and flow based constraints. In A. Leonardis, H. Bischofand, A. Prinz, editors, Proc.European Conference on Computer Vision, volume 3952 of LNCS, pp. 98-111, Graz, Austria, Springer, May 2006.
Brox T., Rosenhahn B., Kersting U., and Cremers D. Nonparametric density estimation for human pose tracking. In K.F. et al., editor, Pattern Recognition, volume 4174 of LNCS, pp. 546-555, Berlin, Germany, Sept. 2006. Springer.
Brox T. and Weickert J. A TV flow based local scale estimate and its appli-cation to texture discrimination. Journal of Visual Communication and Image Representation, 17(5):1053-1073, Oct. 2006.
Brox T. and Cremers D. On the statistical interpretation of the piecewise smooth Mumford-Shah functional. In Scale Space and Variational Methods in Computer Vision, volume 4485 of LNCS, pp. 203-213 Springer, 2007.
Bruhn A. and Weickert J. Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In Proc.10th International Confer-ence on Computer Vision, pp. 749-755. IEEE Computer Society Press, Beijing, China, Oct. 2005.
Chan T. and Vese L. Active contours without edges. IEEE Transactions on Image Processing, 10(2):266-277, Feb. 2001.
Chetverikov D. A simple and efficient algorithm for detection of high curvature points. In N. Petkov and M. Westenberg, editors, Computer Analysis of Images and Patterns, volume 2756 of LNCS, pp. 746-753, Groningen, Springer, 2003.
Cremers D. Dynamical statistical shape priors for level set based tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1262-1273, Aug. 2006.
Cremers D., Kohlberger T. and Schnörr C. Shape statistics in kernel space for variational image segmentation. Pattern Recognition, 36(9):1929-1943, Sept. 2003.
Cremers D., Osher S., and Soatto S. Kernel density estimation and intrinsic alignment for shape priors in level set segmentation. International Journal of Computer Vision, 69(3):335-351, 2006.
Cremers D., Rousson M., and Deriche R. A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Interna- tional Journal of Computer Vision, 72(2):195-215, 2007.
DeCarlo D. and Metaxas D. Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision, 38(2):99-127, July 2000.
Dempster A., Laird N., and Rubin D. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society series B, 39:1-38, 1977.
Dervieux A. and Thomasset F. A finite element method for the simulation of Rayleigh-Taylor instability. In R. Rautman, editor, Approximation Methods for Navier-Stokes Problems, volume 771 of Lecture Notes in Mathematics, pp. 145-158. Berlin, Springer, 1979.
Dunn D., Higgins W.E. and Wakeley J. Texture segmentation using 2-D Gabor elementary functions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2):130-149, Feb. 1994.
Elgammal A. and Lee C. Inferring 3D body pose from silhouettes using activity manifold learning. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 681-688, Washington DC, 2004.
Gavrila D. and Davis L.3D model based tracking of humans in action: a multiview approach. In ARPA Image Understanding Workshop, pp. 73-80, Palm Springs, 1996.
Grochow K., Martin S.L., Hertzmann A., and Popović Z. Style-based inverse kinematics. In ACM Transactions on Graphics (Proc.SIGGRAPH), volume 23, pp. 522-531, 2004.
Heiler M. and Schnörr C. Natural image statistics for natural image segmenta- tion. International Journal of Computer Vision, 63(1):5-19, 2005.
Horn B. and Schunck B. Determining optical flow. Artificial Intelligence, 17:185-203,1981.
Horprasert T., Harwood D., and Davis L. A statistical approach for real-time robust background subtraction and shadow detection. In International Confer-ence on Computer Vision, FRAME-RATE Workshop, Kerkyra, Greece, 1999. Available at www.vast.uccs.edu/∼tboult/FRAME.
Kadir T. and Brady M. Unsupervised non-parametric region segmentation using level sets. In Proc.Ninth IEEE International Conference on Computer Vision, volume 2, pp. 1267-1274, 2003.
Kim J., Fisher J., Yezzi A., Cetin M., and Willsky A. A nonparametric statistical method for image segmentation using information theory and curve evolution. IEEE Transactions on Image Processing, 14(10):1486-1502, 2005.
Klette R. and Rosenfeld A. Digital Geometry-Geometric Methods for Digital Picture Analysis. Morgan Kaufmann, San Francisco, 2004.
Klette R., Schlüns K., and Koschan A. Computer Vision. Three-Dimensional Data from Images. Singapore, Springer, 1998.
Lawrence N.D. Gaussian process latent variable models for visualisation of high dimensional data. In Neural Information Processing Systems 16.
Leventon M.E., Grimson W.E.L., and Faugeras O. Statistical shape influence in geodesic active contours. In Proc.2000 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, pp. 316-323, Hilton Head, SC, June 2000.
Lowe D. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110, 2004.
Marchand E., Bouthemy P., and Chaumette F. A 2D-3D model-based approach to real-time visual tracking. Image and Vision Computing, 19(13):941-955, Nov. 2001.
McLachlan G. and Krishnan T. The EM Algorithm and Extensions. Wiley series in probability and statistics. Wiley, 1997.
Mémin E. and Pérez P. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Transactions on Image Processing, 7(5):703-719, May 1998.
Mumford D. and Shah J. Optimal approximations by piecewise smooth func- tions and associated variational problems. Communications on Pure and Applied Mathematics, 42:577-685, 1989.
Murray R., Li Z., and Sastry S. Mathematical Introduction to Robotic Manipu- lation. CRC Press, Baton Rouge, 1994.
Nagel H.-H. and Enkelmann W. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8:565-593, 1986.
Osher S. and Sethian J.A. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79:12-49, 1988.
Ö zuysal M., Lepetit V., Fleuret F., and Fua P. Feature harvesting for tracking-by-detection. In Proc.European Conference on Computer Vision, volume 3953 of LNCS, pp. 592-605. Graz, Austria, Springer, 2006.
Paragios N. and Deriche R. Geodesic active regions: A new paradigm to deal with frame partition problems in computer vision. Journal of Visual Communication and Image Representation, 13(1/2):249-268, 2002.
Parzen E. On the estimation of a probability density function and the mode. Annals of Mathematical Statistics, 33:1065-1076, 1962.
Rasmussen C.E. and Williams C.K.I. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA, 2006.
Rosales R. and Sclaroff S. Learning body pose via specialized maps. In Proc. Neural Information Processing Systems, Dec. 2001.
Rosenblatt F. Remarks on some nonparametric estimates of a density function. Annals of Mathematical Statistics, 27:832-837, 1956.
Rosenhahn B., Brox T., Cremers D., and Seidel H.-P. A comparison of shape matching methods for contour based pose estimation. In R. Reulke, U. Eckhardt, B. Flach, U. Knauer and K. Polthier, editors, Proc.International Workshop on Combinatorial Image Analysis, volume 4040 of LNCS, pp. 263-276, Berlin, Germany, Springer, June 2006.
Rosenhahn B., Brox T., Kersting U., Smith A., Gurney J., and Klette R. A system for marker-less motion capture. Künstliche Intelligenz, (1):45-51, 2006.
Rosenhahn B., Brox T., and Weickert J.. Three-dimensional shape knowledge for joint image segmentation and pose tracking. International Journal of Computer Vision, 73(3):243-262, July 2007.
Rousson M., Brox T., and Deriche R. Active unsupervised texture segmentation on a diffusion based feature space. In Proc.International Conference on Com-puter Vision and Pattern Recognition, pp. 699-704, Madison, WI, June 2003.
Shevlin F. Analysis of orientation problems using Plücker lines. In International Conference on Pattern Recognition (ICPR), volume 1, pp. 685-689, Brisbane, 1998.
Shi J. and Tomasi C. Good features to track. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 593-600, 2004.
Sidenbladh H., Black M., and Sigal L. Implicit probabilistic models of human motion for synthesis and tracking. In A. Heyden, G. Sparr, M. Nielsen and P. Johansen, editors, Proc.European Conference on Computer Vision, volume 2353 of LNCS, pp. 784-800. Springer, 2002.
Silverman B.W. Density Estimation for Statistics and Data Analysis. Chapman & Hall, New York, 1986.
Sminchisescu C. and Jepson A. Generative modelling for continuous non-linearly embedded visual inference. In Proc.International Conference on Machine Learn-ing, 2004.
Sminchisescu C., Kanaujia A., Li Z., and Metaxas D. Discriminative density propagation for 3D human motion estimation. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 390-397, 2005.
Sminchisescu C., Kanaujia A., and Metaxas D. Learning joint top-down and bottom-up processes for 3D visual inference. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 1743-1752, 2006.
Sminchisescu C. and Triggs B. Estimating articulated human motion with co-variance scaled sampling. International Journal of Robotics Research, 22(6):371-391,2003.
Sommer G., editor. Geometric Computing with Clifford Algebra: Theoreti-cal Foundations and Applications in Computer Vision and Robotics. Berlin, Springer, 2001.
Tsai A., Yezzi A., and Willsky A. Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolationand magnification. IEEE Transactions on Image Processing, 10(8):1169-1186, 2001.
Urtasun R., Fleet D.J., and Fua P. 3D people tracking with Gaussian process dynamical models. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 238-245. IEEE Computer Society Press, 2006.
Zhu S.-C. and Yuille A. Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9):884-900, Sept. 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer
About this chapter
Cite this chapter
Brox, T., Rosenhahn, B., Cremers, D. (2008). Contours, Optic Flow, and Prior Knowledge: Cues for Capturing 3D Human Motion in Videos. In: Rosenhahn, B., Klette, R., Metaxas, D. (eds) Human Motion. Computational Imaging and Vision, vol 36. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-6693-1_11
Download citation
DOI: https://doi.org/10.1007/978-1-4020-6693-1_11
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-6692-4
Online ISBN: 978-1-4020-6693-1
eBook Packages: Computer ScienceComputer Science (R0)