Skip to main content
SpringerLink
Log in

3-D human pose recovery using nonrigid point set registration and body part tracking of depth data

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

In this paper, we present a novel approach for recovering a 3-D pose from a single human body depth silhouette using nonrigid point set registration and body part tracking. In our method, a human body depth silhouette is presented as a set of 3-D points and matched to another set of 3-D points using point correspondences. To recognize and maintain body part labels, we initialize the first set of points to corresponding human body parts, resulting in a body part-labeled map. Then, we transform the points to a sequential set of points based on point correspondences determined by nonrigid point set registration. After point registration, we utilize the information from tracked body part labels and registered points to create a human skeleton model. A 3-D human pose gets recovered by mapping joint information from the skeleton model to a 3-D synthetic human model. Quantitative and qualitative evaluation results on synthetic and real data show that complex human poses can be recovered more reliably with lower errors compared to other conventional techniques for 3-D pose recovery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Poppe, R.: Vision-based human motion analysis: an overview. Comput. Vis. Image Underst. 108(1), 4–18 (2007)

    Article  Google Scholar 

  2. Uddin, M.Z., Thang, N.D., Kim, T.S., Kim, J.T.: Human activity recognition using body joint angle features and hidden markov model. ETRI J. 33(4), 569–579 (2011)

    Article  Google Scholar 

  3. Jalal, M., Uddin, M.Z., Kim, T.S.: Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home. IEEE Trans. Consum. Electron. 58(3), 863–871 (2012)

    Article  Google Scholar 

  4. Yang, M.T., Chuang, M.W.: Fall risk assessment and early-warning for toddler behaviors at home. Sensors 13(12), 16985–17005 (2013)

    Article  Google Scholar 

  5. Chen, L., Wei, H., Ferryman, J.: A survey of human motion analysis using depth imagery. Pattern Recog. Lett. ISSN 0167–8655 (2013)

  6. Kalogerakis, E., Hertzmann, A., Singh, K.: Learning 3D mesh segmentation and labeling. ACM Trans. Graph. (TOG) 29(4), 102–114 (2010)

    Article  Google Scholar 

  7. Plagemann, C., Ganapathi, V., Koller, D., Thrun, S.: Real-time identification and localization of body parts from depth images. In: proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 3108–3113 (2010)

  8. Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N.: Human skeleton tracking from depth data using geodesic distances and optical flow. Image Vis. Comput. 30(3), 217–226 (2012)

    Article  Google Scholar 

  9. Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56(1), 116–124 (2013)

    Article  Google Scholar 

  10. Taylor, J., Shotton, J., Sharp, T., Fitzgibbon, A.: The Vitruvian manifold: inferring dense correspondences for one-shot human pose estimation. In: Proc. of IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 103–110 (2012)

  11. Kim, D., Kim, D.: A fast ICP algorithm for 3-D human body motion tracking. Signal Process. Lett. IEEE 17(4), 402–405 (2010)

    Article  Google Scholar 

  12. Mundermann, L., Corazza, S., Andriacchi, T.P.: Accurately measuring human movement using articulated ICP with soft-joint constraints and a repository of articulated models. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–6 (2007)

  13. Corazza, S., Mündermann, L., Gambaretto, E., Ferrigno, G., Andriacchi, T.P.: Markerless motion capture through visual hull, articulated ICP and subject specific model generation. Int. J. Comput. Vision 87(1–2), 156–169 (2010)

    Article  Google Scholar 

  14. Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010)

    Article  Google Scholar 

  15. Dinh, D.L., Lim, M.J., Thang, N.D., Lee, S., Kim, T.S.: Real-time 3-D human pose recovery from a single depth image using principal direction analysis. Appl. Intell. 41(2), 473–486 (2014)

    Article  Google Scholar 

  16. Tam, G.K., Cheng, Z.Q., Lai, Y.K., Langbein, F.C., Liu, Y., Marshall, D., Rosin, P.L.: Registration of 3-D point clouds and meshes. A survey from rigid to nonrigid. IEEE Trans. Vis. Comput. Graph. 19(7), 1199–1217 (2013)

    Article  Google Scholar 

  17. Yuille, A.L., Grzywacz, N.M.: A mathematical analysis of the motion coherence theory. Int. J. Comput. Vis. 3(2), 155–175 (1989)

    Article  Google Scholar 

  18. Jian, B., Vemuri, B.C.: A robust algorithm for point set registration using mixture of Gaussians. IEEE Int. Conf Comput. Vis. 2(20), 1246–1251 (2005)

    Google Scholar 

  19. Jian, B., Vemuri, B.C.: Robust point set registration using gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–1645 (2011)

    Article  Google Scholar 

  20. Sang, Q., Zhang, J., Yu, Z.: Non-rigid point set registration: a bidirectional approach. ICASSP 2012, 693–696 (2012)

    Google Scholar 

  21. Droeschel, D., Behnke, S.: 3-D body pose estimation using an adaptive person model for articulated ICP. In: Intelligent Robotics and Applications, pp. 157–167 (2011)

  22. Siddiqui, M., Medioni, G.: Human pose estimation from a single view point, real-time range sensor. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8 (2010)

  23. Brox, T., Rosenhahn, B., Gall, J., Cremers, D.: Combined region and motion-based 3-D tracking of rigid and articulated objects. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 402–415 (2010)

    Article  Google Scholar 

  24. Wei, X., Zhang, P., Chai, J.: Accurate realtime full-body motion capture using a single depth camera. ACM Transact. Graph. (TOG) 31(6), 188 (2012)

    Google Scholar 

  25. Ye, M., Wang, X., Yang, R., Ren, L., Pollefeys, M.: Accurate 3D pose estimation from a single depth image. 2011 IEEE International Conference on Computer Vision (ICCV), pp. 731–738 (2011)

  26. Baak, A., Müller, M., Bharaj, G., Seidel, H.P., Theobalt, C.: A data-driven approach for real-time full body pose reconstruction from a depth camera. In: Consumer Depth Cameras for Computer Vision, pp. 71–98 (2013)

  27. Ganapathi, V., Plagemann, C., Koller, D., Thrun, S. Real time motion capture using a single time-of-flight camera. In: CVPR, pp. 3108–3113 (2010)

  28. Ganapathi, V., Plagemann, C., Koller, D., Thrun, S.: Real-time human pose tracking from range data. In: ECCV, pp. 738–751 (2012)

  29. Helten, T., Baak, A., Bharaj, G., Muller, M., Seidel, H.P., Theobalt, C.: Personalization and evaluation of a real-time depth-based full body tracker. In: 3DV, pp. 279–286 (2013)

  30. Ye, M., Yang, R.: Real-time simultaneous pose and shape estimation for articulated objects using a single depth camera. In: CVPR, pp. 2353–2360 (2014)

  31. Fossati, A., Dimitrijevic, M., Lepetit, V., Fua, P.: From canonical poses to 3-D motion capture using a single camera. Pattern Anal. Mach. Intell. IEEE Transact. 32(7), 1165–1181 (2010)

    Article  Google Scholar 

  32. Lee, M.W., Cohen, I.: A model-based approach for estimating human 3D poses in static images. Pattern Anal. Mach. Intell. IEEE Transact. 28(6), 905–916 (2006)

    Article  Google Scholar 

  33. Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  34. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. PAMI 24(5), 1–5 (2002)

    Article  Google Scholar 

  35. CMU motion capture database. http://mocap.cs.cmu.edu

  36. Autodesk 3ds Max, 2012

Download references

Acknowledgments

This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program supervised by the NIPA (National IT Industry Promotion Agency (NIPA-2013-(H0301-13-2001)). This work was also supported by the Industrial Core Technology Development Program (10049079, Development of Mining core technology exploiting personal big data) funded by the Ministry of Trade, Industry and Energy (MOTIE, Korea).

Author information

Authors and Affiliations

  1. Department of Information Technology, Nha Trang University, 2 Nguyen Dinh Chieu, Nha Trang, Vietnam

    Dong-Luong Dinh

  2. Department of Computer Engineering, Kyung Hee University, 1 Seocheon-dong, Giheung-gu, Yongin-Si, Gyeonggi-do, Republic of Korea

    Sungyoung Lee

  3. Department of Biomedical Engineering, Kyung Hee University, 1 Seocheon-dong, Giheung-gu, Yongin-Si, Gyeonggi-do, Republic of Korea

    Tae-Seong Kim

Authors
  1. Dong-Luong Dinh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sungyoung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tae-Seong Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Sungyoung Lee or Tae-Seong Kim.

Additional information

Communicated by B. Huet.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dinh, DL., Lee, S. & Kim, TS. 3-D human pose recovery using nonrigid point set registration and body part tracking of depth data. Multimedia Systems 23, 369–380 (2017). https://doi.org/10.1007/s00530-015-0497-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-015-0497-y

Keywords

Search

Navigation