Peizhao Zhang
Abstract
We present a new method for full-body motion capture that uses input data captured by three depth cameras and a pair of pressure-sensing shoes. Our system is appealing because it is low-cost, non-intrusive and fully automatic, and can accurately reconstruct both full-body kinematics and dynamics data. We first introduce a novel tracking process that automatically reconstructs 3D skeletal poses using input data captured by three Kinect cameras and wearable pressure sensors. We formulate the problem in an optimization framework and incrementally update 3D skeletal poses with observed depth data and pressure data via iterative linear solvers. The system is highly accurate because we integrate depth data from multiple depth cameras, foot pressure data, detailed full-body geometry, and environmental contact constraints into a unified framework. In addition, we develop an efficient physics-based motion reconstruction algorithm for solving internal joint torques and contact forces in the quadratic programming framework. During reconstruction, we leverage Newtonian physics, friction cone constraints, contact pressure information, and 3D kinematic poses obtained from the kinematic tracking process to reconstruct full-body dynamics data. We demonstrate the power of our approach by capturing a wide range of human movements and achieve state-of-the-art accuracy in our comparison against alternative systems.
Supplemental Material
Available for Download
Supplemental material.
- Adelsberger, R., and Tröster, G. 2013. Pimu: A wireless pressure-sensing imu. In ISSNIP, IEEE Proceedings of the 8th International Conference on Intelligent Sensors, Sensor Networks and Information Processing.Google Scholar
- Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. 22, 3 (July), 587--594. Google ScholarDigital Library
- Ascension, 2014. http://www.ascension-tech.com/.Google Scholar
- Baak, A., Müller, M., Bharaj, G., Seidel, H.-P., and Theobalt, C. 2011. A data-driven approach for real-time full body pose reconstruction from a depth camera. In IEEE 13th International Conference on Computer Vision (ICCV), 1092--1099. Google ScholarDigital Library
- Bregler, C., Malik, J., and k Pullen. 2004. Twist based acquisition and tracking of animal and human kinematics. International Journal of Computer Vision. 56(3):179--194. Google ScholarDigital Library
- Bridson, R., Marino, S., and Fedkiw, R. 2003. Simulation of clothing with folds and wrinkles. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 28--36. Google ScholarDigital Library
- Brubaker, M. A., and Fleet, D. J. 2008. The Kneed Walker for human pose tracking. In Proceedings of IEEE CVPR.Google Scholar
- Curless, B., and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '96, 303--312. Google ScholarDigital Library
- de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3 (Aug.), 98:1--98:10. Google ScholarDigital Library
- Elgammal, A., and Lee, C. 2004. Inferring 3D body pose from silhouettes using activity manifold learning. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2: 681--688. Google ScholarDigital Library
- Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM 24, 6, 381--395. Google ScholarDigital Library
- Fisher, S., and Lin, M. C. 2001. Deformed distance fields for simulation of non-penetrating flexible bodies. In Proceedings of the Eurographic Workshop on Computer Animation and Simulation, Springer-Verlag New York, Inc., New York, NY, USA, 99--111. Google ScholarDigital Library
- Grest, D., Kruger, V., and Koch, R. 2007. Single view motion tracking by depth and silhouette information. In Proceedings of the 15th Scandinavian Conference on Image Analysis (SCIA), 719--729. Google ScholarDigital Library
- Guendelman, E., Bridson, R., and Fedkiw, R. 2003. Non-convex rigid bodies with stacking. ACM Trans. Graph. 22, 3 (July), 871--878. Google ScholarDigital Library
- Ha, S., Bai, Y., and Liu, C. K. 2011. Human motion reconstruction from force sensors. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '11, 129--138. Google ScholarDigital Library
- Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. In Computer Graphics Forum (Proc. Eurographics 2008), P. Dutr'e and M. Stamminger, Eds., vol. 2.Google Scholar
- Knoop, S., Vacek, S., and Dillmann, R. 2006. Sensor fusion for 3d human body tracking with an articulated 3d body model. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 1686--1691.Google Scholar
- Kovar, L., Schreiner, J., and Gleicher, M. 2002. Footskate cleanup for motion capture editing. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '02, 97--104. Google ScholarDigital Library
- Microsoft Kinect API for Windows, 2014. http://www.microsoft.com/en-us/kinectforwindows/.Google Scholar
- Moeslund, T. B., Hilton, A., and Krüger, V. 2006. A survey of advances in vision-based human motion capture and analysis. Journal of Computer Vision and Image Understanding. 104:90--126. Google ScholarDigital Library
- Pavlović, V., Rehg, J. M., and MacCormick, J. 2000. Learning Switching Linear Models of Human Motion. In Advances in Neural Information Processing Systems 13, 981--987.Google Scholar
- Plagemann, C., Ganapathi, V., Koller, D., and Thrun, S. 2010. Realtime identification and localization of body parts from depth images. In Proceedings of International Conferences on Robotics and Automation (ICRA 2010), 3108--3113.Google Scholar
- Rosales, R., and Sclaroff, S. 2000. Specialized mappings and the estimation of human body pose from a single image. In Proceedings of the Workshop on Human Motion. 19--24. Google ScholarDigital Library
- Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. 2011. Real-time human pose recognition in parts from a single depth image. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1297--1304. Google ScholarDigital Library
- Tekscan, 2014. http://www.tekscan.com/.Google Scholar
- Urtasun, R., Fleet, D. J., Hertzmann, A., and Fua., P. 2005. Priors for people tracking from small training sets. In IEEE International Conference on Computer Vision, 403--C410. Google ScholarDigital Library
- Vicon Systems, 2014. http://www.vicon.com.Google Scholar
- Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 3 (Aug.), 97:1--97:9. Google ScholarDigital Library
- Vondrak, M., Sigal, L., and Jenkins, O. C. 2008. Physical simulation for probabilistic motion tracking. In IEEE Conference on Computer Vision and Pattern Recognition, 1--8.Google Scholar
- Vondrak, M., Sigal, L., Hodgins, J., and Jenkins, O. 2012. Video-based 3d motion capture through biped control. ACM Trans. Graph. 31, 4 (July), 27:1--27:12. Google ScholarDigital Library
- Wei, X. K., and Chai, J. 2010. Videomocap: Modeling physically realistic human motion from monocular video sequences. ACM Transactions on Graphics. 29(4): Article No. 42. Google ScholarDigital Library
- Wei, X., Zhang, P., and Chai, J. 2012. Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. 31, 6 (Nov.), 188:1--188:12. Google ScholarDigital Library
- Xsens, 2014. http://www.xsens.com.Google Scholar
- Ye, M., Wang, X., Yang, R., Ren, L., and Pollefeys, M. 2011. Accurate 3d pose estimation from a single depth image. In Proceedings of IEEE 13th International Conference on Computer Vision, 731--738. Google ScholarDigital Library
- Yin, K., and Pai, D. K. 2003. Footsee: An interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 329--338. Google ScholarDigital Library
Index Terms
- Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture
Recommendations
Accurate realtime full-body motion capture using a single depth camera
We present a fast, automatic method for accurately capturing full-body motion data using a single depth camera. At the core of our system lies a realtime registration process that accurately reconstructs 3D human poses from single monocular depth images,...
Motion capture from body-mounted cameras
Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large ...
Motion capture from body-mounted cameras
SIGGRAPH '11: ACM SIGGRAPH 2011 papersMotion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large ...
Comments