Abstract
We present an algorithm for 3D reconstruction of dynamic articulated structures, such as humans, from uncalibrated multiple views. The reconstruction exploits constraints associated with a dynamic articulated structure, specifically the conservation over time of length between rotational joints. These constraints admit reconstruction of metric structure from at least two different images in each of two uncalibrated parallel projection cameras. As a by product, the calibration of the cameras can also be computed. The algorithm is based on a stratified approach, starting with affine reconstruction from factorization, followed by rectification to metric structure using the articulated structure constraints. The exploitation of these specific constraints admits reconstruction and self-calibration with fewer feature points and views compared to standard self-calibration. The method is extended to pairs of cameras that are zooming, where calibration of the cameras allows compensation for the changing scale factor in a scaled orthographic camera. Results are presented in the form of stick figures and animated 3D reconstructions using pairs of sequences from broadcast television. The technique shows promise as a means of creating 3D animations of dynamic activities such as sports events.
Similar content being viewed by others
References
Bregler, C. and Malik, J. 1998. Tracking people with twists and exponential maps. In Proc. IEEE Conference on Computer Vision and Pattern Recognition.
Faugeras, O. 1992. What can be seen in three dimensions with an uncalibrated stereo rig? In Proc. European Conference on Computer Vision, LNCS, vol. 588, Springer-Verlag: Berlin, pp. 563– 578.
Faugeras, O.D. 1995. Stratification of three-dimensional vision: Projective, affine, and metric representation. Journal of the Optical Society of America A, 12:465–484.
Faugeras, O.D. and Maybank, S.J. 1990. Motion from point matches: Multiplicity of solutions. International Journal of Computer Vision, 4:225–246.
Hartley, R.I. 1992. Estimation of relative camera positions for uncalibrated cameras. In Proc. European Conference on Computer Vision, LNCS, vol. 588, Springer-Verlag: Berlin, pp. 579– 587.
Hartley, R.I. and Zisserman, A. 2000. Multiple View Geometry in Computer Vision. Cambridge University Press; Cambridge, UK.
Koenderink, J.J. and van Doorn, A.J. 1991. Affine structure from motion. J. Opt. Soc. Am. A, 8(2):377–385.
Kreysig, E. 1993. Advanced Engineering Mathematics. John Wiley and Sons: New York.
Leventon, M.E. and Freeman, W.T. 1998. Bayesian estimation of 3-D human motion from an image sequence. Technical Report TR-98-06, MERL.
Liebowitz, D. and Zisserman, A. 1998. Metric rectification for perspective images of planes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 482–488.
Liebowitz, D. and Zisserman, A. 1999. Combining scene and autocalibration constraints. In Proc. 7th International Conference on Computer Vision, Kerkyra, Greece.
Maybank, S. and Faugeras, O.D. 1992. A theory of self-calibration of a moving camera. International Journal of Computer Vision, 8(2):123–151.
Mundy, J. and Zisserman, A. 1992. Geometric Invariance in Computer Vision. MIT Press: Cambridge, MA.
Poelman, C. and Kanade, T. 1994. A paraperspective factorization method for shape and motion recovery. In Proc. 3rd European Conference on Computer Vision, Stockholm, vol. 2, pp. 97–108.
Pollefeys, M., Van Gool, L., and Oosterlinck, A. 1996. The modulus constraint: A new constraint for self-calibration. In Proc. International Conference on Pattern Recognition, pp. 31–42.
Quan, L. 1996. Self-calibration of an affine camera from multiple views. International Journal of Computer Vision, 19(1):93–105.
Reid, I.D. and Murray, D.W. 1996. Active tracking of foveated feature clusters using affine structure. International Journal of Computer Vision, 18(1):41–60.
Shapiro, L., Zisserman, A., and Brady, M. 1994. Motion from point matches using affine epipolar geometry. In Proc. European Conference on Computer Vision, LNCS, vol. 800/801, Springer-Verlag: Berlin.
Shapiro, L.S., Zisserman, A., and Brady, M. 1995. 3D motion recovery via affine epipolar geometry. International Journal of Computer Vision, 16(2):147–182.
Taylor, C.J. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. In Proc. IEEE Conference on Computer Vision and Pattern Recognition.
Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization approach. International Journal of Computer Vision, 9(2):137–154.
Webb, J.A. and Aggarwal, J.K. 1983. Structure from motion of rigid and jointed object. Artificial Intelligence, 19:107–130.
Weinshall, D. and Tomasi, C. 1993. Linear and incremental acquisition of invariant shape models from image sequences. In Proc. 4th International Conference on Computer Vision, Berlin, IEEE Computer Society Press: Los Alamitos, CA, pp. 675–682.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Liebowitz, D., Carlsson, S. Uncalibrated Motion Capture Exploiting Articulated Structure Constraints. International Journal of Computer Vision 51, 171–187 (2003). https://doi.org/10.1023/A:1021897717694
Issue Date:
DOI: https://doi.org/10.1023/A:1021897717694