Abstract
We present an augmented reality interface with markerless human body interaction. It consists of 3D motion capture of the human body and the processing of 3D human poses for augmented reality applications. A monocular camera is used to acquire the images of the user’s motion for 3D pose estimation. In the proposed technique, a graphical 3D human model is first constructed. Its projection on a virtual image plane is then used to match the silhouettes obtained from the image sequence. By iteratively adjusting the 3D pose of the graphical 3D model with the physical and anatomic constraints of the human motion, the human pose and the associated 3D motion parameters can be uniquely identified. The obtained 3D pose information is then transferred to the reality processing subsystem and used to achieve the marker-free interaction in the augmented environment. Experimental results are presented using a head mounted display.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Zhou, F., Duh, H.B.L., Billinghurst, M.: Trends in augmented reality tracking, interaction and display: A review of ten years of ismar. In: ISMAR 2008: Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 193–202. IEEE Computer Society, Washington (2008)
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. Computer Graphics and Applications 21, 34–47 (2001)
Starner, T., Leibe, B., Minnen, D., Westyn, T., Hurst, A., Weeks, J.: The perceptive workbench: Computer-vision-based gesture tracking, object tracking, and 3d reconstruction for augmented desks. Machine Vision and Applications 14, 59–71 (2003)
Finger tracking for interaction in augmented environments. In: ISAR 2001: Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR 2001), Washington, DC, USA, p. 55. IEEE Computer Society, Los Alamitos (2001)
Comport, A.I., Marchand, E., Pressigout, M., Chaumette, F.: Real-time markerless tracking for augmented reality: The virtual visual servoing framework. IEEE Transactions on Visualization and Computer Graphics 12, 615–628 (2006)
Lee, T., Höllerer, T.: Multithreaded hybrid feature tracking for markerless augmented reality. IEEE Transactions on Visualization and Computer Graphics 15, 355–368 (2009)
Chua, P.T., Crivella, R., Daly, B., Hu, N., Schaaf, R., Ventura, D., Camill, T., Hodgins, J., Pausch, R.: Training for physical tasks in virtual environments: Tai chi. In: VR 2003: Proceedings of the IEEE Virtual Reality 2003, Washington, DC, USA, p. 87. IEEE Computer Society, Los Alamitos (2003)
Chan, J., Leung, H., Tang, K.T., Komura, T.: Immersive performance training tools using motion capture technology. In: ImmersCom 2007: Proceedings of the First International Conference on Immersive Telecommunications, pp. 1–6 (2007)
Howe, N.R.: Silhouette lookup for monocular 3d pose tracking. Image Vision Comput. 25, 331–341 (2007)
Bregler, C., Malik, J., Pullen, K.: Twist based acquisition and tracking of animal and human kinematics. Int. J. Comput. Vision 56, 179–194 (2004)
Loy, G., Eriksson, M., Sullivan, J., Carlsson, S.: Monocular 3d reconstruction of human motion in long action sequences. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 442–455. Springer, Heidelberg (2004)
Chen, Y., Lee, J., Parent, R., Machiraju, R.: Markerless monocular motion capture using image features and physical constraints. Computer Graphics International 2005, 36–43 (2005)
Poppe, R.: Vision-based human motion analysis: An overview. Comput. Vis. Image Underst. 108, 4–18 (2007)
Ning, H., Tan, T., Wang, L., Hu, W.: Kinematics-based tracking of human walking in monocular video sequences. Image Vision Comput. 22, 429–441 (2004)
Kato, H., Billinghurst, M.: Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In: Proceedings of 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR 1999), pp. 85–94 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lin, HY., Chen, TW. (2010). Augmented Reality with Human Body Interaction Based on Monocular 3D Pose Estimation. In: Blanc-Talon, J., Bone, D., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2010. Lecture Notes in Computer Science, vol 6474. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17688-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-17688-3_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17687-6
Online ISBN: 978-3-642-17688-3
eBook Packages: Computer ScienceComputer Science (R0)