Abstract
We describe a visual communication application for a dark, theater-like interactive virtual simulation training environment. Our system visually estimates and tracks the body position, orientation and the arm-pointing direction of the trainee. This system uses a near-IR camera array to capture images of the trainee from different angles in the dim-lighted theater. Image features like silhouettes and intermediate silhouette body axis points are then segmented and extracted from image backgrounds. 3D body shape information such as 3D body skeleton points and visual hulls can be reconstructed from these 2D features in multiple calibrated images. We proposed a particle-filtering based method that fits an articulated body model to the observed image features. Currently we focus on the arm-pointing gesture of either limb. From the fitted articulated model we can derive the position on the screen the user is pointing to. We use current graphic hardware to accelerate the processing speed so the system is able to work in real-time. The system serves as part of multi-modal user-input device in the interactive simulation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arulampalam, S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for on-line non-linear/non-gaussian bayesian tracking. IEEE Transactions on Signal Processing 50(2), 174–188 (2002)
Bernier, O., Cheung Mon, P.: Real-time 3d articulated pose tracking using particle filtering and belief propagation on factor graphs. In: BMVC 2006, p. 7 (2006)
Demirdjian, D., Darrell, T.: 3-d articulated pose tracking for untethered diectic reference. In: Proceedings of ICMI 2002, Washington, DC, USA, p. 267. IEEE Computer Society Press, Los Alamitos (2002)
Deutscher, J., Blake, A., Reid, I.D.: Articulated body motion capture by annealed particle filtering. In: CVPR 2000, pp. 126–133 (2000)
Doucet, A., de Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, Heidelberg
Eberly, D.: Perspective projection of an ellipsoid
Elgammal, A., Lee, C.-S.: Inferring 3d body pose from silhouettes using activity manifold learning (2004)
Horain, P., Bomb, M.: 3d model based gesture acquisition using a single camera. In: WACV 2002. Proceedings. Sixth IEEE Workshop, pp. 158–162. IEEE, Los Alamitos (2002)
Izo, T., Grimson, W.: Simultaneous pose estimation and camera calibration from multiple views. In: Non-Rigid 2004, p. 14 (2004)
Kehl, R., Bray, M., Van Gool, L.: Full body tracking from multiple views using stochastic sampling. In: CVPR 2005, vol. 2, pp. 129–136. IEEE Computer Society Conference, Los Alamitos (2005)
Laurentini, A.: The visual hull concept for silhouette-based image understanding. PAMI 16(2), 150–162 (1994)
Lee, M., Nevatia, R.: Dynamic human pose estimation using markov chain monte carlo approach. In: Motion 2005, pp. 168–175 (2005)
Leymarie, F.F., Kimia, B.B.: Medial Representations: Mathematics, Algorithms and Applications. Kluwer Academic Publishers, Dordrecht, ch. 11, (to be Published)
Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. In: Proceedings of the 12th Eurographics Workshop on Rendering Techniques, London, UK, pp. 115–126. Springer, Heidelberg (2001)
Parameswaran, V., Chellappa, R.: View independent human body pose estimation from a single perspective image. In: CVPR 2004, pp. 16–22 (2004)
Rahman, M., Ishikawa, S.: Appearance-based representation and recognition of human motions. In: ICRA 2003. Proceedings of IEEE International Conference on Robotics and Automation, 2003, vol. 1, pp. 1410–1415. IEEE Computer Society Press, Los Alamitos (2003)
Schmidt, J., Fritsch, J., Kwolek, B.: Kernel particle filter for real-time 3d body tracking in monocular color images. In: FGR 2006, pp. 567–572 (2006)
Yoshimoto, H., Date, N., Yonemoto, S.: Vision-based real-time motion capture system using multiple cameras. In: MFI, pp. 247–251 (2003)
Zerroug, M., Nevatia, R.: Segmentation and 3-d recovery of curved-axis generalized cylinders from an intensity image. In: Computer Vision and Image Processing, Proceedings of the 12th IAPR International Conference, vol. 1, pp. 678–681 (1994)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chu, CW., Nevatia, R. (2007). Real Time Body Pose Tracking in an Immersive Training Environment. In: Lew, M., Sebe, N., Huang, T.S., Bakker, E.M. (eds) Human–Computer Interaction. HCI 2007. Lecture Notes in Computer Science, vol 4796. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75773-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-540-75773-3_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75772-6
Online ISBN: 978-3-540-75773-3
eBook Packages: Computer ScienceComputer Science (R0)