Liuyang Zhou
Hubert P. H. Shum
ABSTRACT
Recent work has shown that depth image based 3D posture estimation hardware such as Kinect has made interactive applications more popular. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. While previous research has shown that data-driven methods can be used to reconstruct the correct postures, they usually require a large posture database, which greatly limit the usability for systems with constrained hardware such as game console. To solve this problem, we present a new probabilistic framework to enhance the accuracy of the postures live captured by Kinect. We adopt the Gaussian Process model as a prior to leverage position data obtained from Kinect and marker-based motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the observed input data from Kinect when its tracking result is good, we embed joint reliability into the optimization framework. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time posture based applications such as motion-based gaming and sport training.
Supplemental Material
Available for Download
Supplemental material.
- Baak, A., Muller, M., Bharaj, G., Seidel, H.-P., and Theobalt, C. 2011. A data-driven approach for real-time full body pose reconstruction from a depth camera. In Proceedings of the 2011 International Conference on Computer Vision, ICCV '11, 1092--1099. Google Scholar
Digital Library
- Bailey, S. W., and Bodenheimer, B. 2012. A comparison of motion capture data recorded from a vicon system and a microsoft kinect sensor. In Proceedings of the ACM Symposium on Applied Perception, SAP '12, 121--121. Google ScholarDigital Library
- Chai, J., and Hodgins, J. K. 2005. Performance animation from low-dimensional control signals. In ACM SIGGRAPH 2005 Papers, SIGGRAPH '05, 686--696. Google ScholarDigital Library
- Chan, J., Leung, H., Tang, J., and Komura, T. 2011. A virtual reality dance training system using motion capture technology. Learning Technologies, IEEE Transactions on 4, 2 (April), 187--195. Google ScholarDigital Library
- Han, J., Shao, L., Xu, D., and Shotton, J. 2013. Enhanced computer vision with microsoft kinect sensor: A review. Cybernetics, IEEE Transactions on 43, 5 (Oct), 1318--1334.Google Scholar
- Helten, T., Muller, M., Seidel, H.-P., and Theobalt, C. 2013. Real-time body tracking with one depth camera and inertial sensors. In Computer Vision (ICCV), 2013 IEEE International Conference on, 1105--1112. Google ScholarDigital Library
- Hoyet, L., McDonnell, R., and O'Sullivan, C. 2012. Push it real: Perceiving causality in virtual interactions. ACM Trans. Graph. 31, 4 (July), 90:1--90:9. Google ScholarDigital Library
- Iason Oikonomidis, N. K., and Argyros, A. 2011. Efficient model-based 3d tracking of hand articulations using kinect. In Proceedings of the British Machine Vision Conference, BMVA Press, 101.1--101.11.Google Scholar
- Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., and Fitzgibbon, A. 2011. Kinectfusion: Real-time 3d reconstruction and interaction using a moving depth camera. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST '11, 559--568. Google ScholarDigital Library
- Kim, J., Seol, Y., and Lee, J. 2013. Human motion reconstruction from sparse 3d motion sensors using kernel cca-based regression. Computer Animation and Virtual Worlds 24, 6.Google ScholarDigital Library
- Lawrence, N. 2009. Gaussian process library, http://www.cs.manchester.ac.uk/neill/gp/.Google Scholar
- Liu, H., Wei, X., Chai, J., Ha, I., and Rhee, T. 2011. Realtime human motion control with a small number of inertial sensors. In Symposium on Interactive 3D Graphics and Games, I3D '11, 133--140. Google ScholarDigital Library
- Microsoft, C. 2013. Kinect for windows sdk programming guide version 1.8.Google Scholar
- Morgan, T., Jarrell, D., and Vance, J. 2014. Poster: Rapid development of natural user interaction using kinect sensors and vrpn. In 3D User Interfaces (3DUI), 2014 IEEE Symposium on, 163--164.Google Scholar
- Quiñonero Candela, J., and Rasmussen, C. E. 2005. A unifying view of sparse approximate gaussian process regression. J. Mach. Learn. Res. 6 (Dec.), 1939--1959. Google ScholarDigital Library
- Rasmussen, C. E., and Williams, C. K. I. 2005. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press. Google ScholarDigital Library
- Shen, W., Deng, K., Bai, X., Leyvand, T., Guo, B., and Tu, Z. 2012. Exemplar-based human action pose correction and tagging. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, 1784--1791. Google ScholarDigital Library
- Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. 2011. Real-time human pose recognition in parts from single depth images. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR '11, 1297--1304. Google ScholarDigital Library
- Shum, H., and Ho, E. S. 2012. Real-time physical modelling of character movements with microsoft kinect. In Proceedings of the 18th ACM symposium on Virtual reality software and technology, VRST '12, 17--24. Google ScholarDigital Library
- Shum, H. P. H., Ho, E. S. L., Jiang, Y., and Takagi, S. 2013. Real-time posture reconstruction for microsoft kinect. IEEE Transactions on Cybernetics 43, 5, 1357--1369.Google ScholarCross Ref
- Sidenbladh, H., and Black, M. 2001. Learning image statistics for bayesian tracking. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, vol. 2, 709--716 vol.2.Google Scholar
- Sigalas, M., Pateraki, M., Oikonomidis, I., and Trahanias, P. 2013. Robust model-based 3d torso pose estimation in rgb-d sequences. In Computer Vision Workshops (ICCVW), 2013 IEEE International Conference on, 315--322. Google ScholarDigital Library
- Tashev, I. 2013. Kinect development kit: A toolkit for gesture- and speech-based human-machine interaction {best of the web}. Signal Processing Magazine, IEEE 30, 5 (Sept), 129--131.Google Scholar
- Taylor, G., Sigal, L., Fleet, D., and Hinton, G. 2010. Dynamical binary latent variable models for 3d human pose tracking. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, 631--638.Google Scholar
- Vondrak, M., Sigal, L., and Jenkins, O. 2013. Dynamical simulation priors for human motion tracking. Pattern Analysis and Machine Intelligence, IEEE Transactions on 35, 1 (Jan), 52--65. Google ScholarDigital Library
- Wei, X., Zhang, P., and Chai, J. 2012. Accurate realtime fullbody motion capture using a single depth camera. ACM Trans. Graph. 31, 6 (Nov.), 188:1--188:12. Google ScholarDigital Library
- Yasin, H., Krüger, B., and Weber, A. 2013. Model based full body human motion reconstruction from video data. In Proceedings of the 6th International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications, MIRAGE '13, 1:1--1:8. Google ScholarDigital Library
Index Terms
- Posture reconstruction using Kinect with a probabilistic model
Recommendations
Optimization-based posture reconstruction for digital human models
Digital human modeling provides a valuable tool for designers when implemented early in the design process. Motion capture experiments offer a means of validation of the digital human simulation models. However, there is a gap between the motion capture ...
Smart motion reconstruction system for golf swing: a DBN model based transportable, non-intrusive and inexpensive golf swing capture and reconstruction system
In the past decade, golf has stimulated people's great interest and the number of golf players has increased significantly. Therefore, how to train a golfer to make a perfect swing has attracted extensive research attentions. Among these researches, the ...
Analysis of Human Hand Posture Reconstruction Under Constraint and Non-constraint Wrist Position
ICIRA 2015: Proceedings, Part I, of the 8th International Conference on Intelligent Robotics and Applications - Volume 9244As a compactness unit, the human hand shows high versatility and sophisticated grasp functionality. How to design a robot hand replicating the human grasp posture is a challenging task. Mechanical implementation of postural synergies provides new hope ...
Comments