Skip to main content
SpringerLink
Log in

3D human modeling from a single depth image dealing with self-occlusion

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a 2D to 3D conversion scheme to generate a 3D human model using a single depth image with several color images. In building a complete 3D model, no prior knowledge such as a pre-computed scene structure and photometric and geometric calibrations is required since the depth camera can directly acquire the calibrated geometric and color information in real time. The proposed method deals with a self-occlusion problem which often occurs in images captured by a monocular camera. When an image is obtained from a fixed view, it may not have data for a certain part of an object due to occlusion. The proposed method consists of following steps to resolve this problem. First, the noise in a depth image is reduced by using a series of image processing techniques. Second, a 3D mesh surface is constructed using the proposed depth image-based modeling method. Third, the occlusion problem is resolved by removing the unwanted triangles in the occlusion region and filling the corresponding hole. Finally, textures are extracted and mapped to the 3D surface of the model to provide photo-realistic appearance. Comparison results with the related work demonstrate the efficiency of our method in terms of visual quality and computation time. It can be utilized in creating 3D human models in many 3D applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Allen B, Curless B, Popovic Z (2003) The space of human body shapes: reconstruction and parameterization from range images. In: Proceedings of ACM SIGGRAPH 2003, pp 587–594

  2. Atiqur RA, Tan JK, Kim HS, Ishikawa S (2008) Solutions to motion self-occlusion problem in human activity analysis. In: Proceedings of the 11th international conference on computer and information technology (ICCIT), Article no. 4803095. Khulna, Bangladesh, pp 201–206

  3. Bhasin SS, Chaudhuri S (2001) Depth from defocus in presence of partial self occlusion. In: Proceedings of the international conference on computer vision, vol 2. Vancouver, Canada, pp 488–493

  4. Bouguet JY (2000) Pyramidal implementation of the Lucas Kanade feature tracker. Intel Corporation, Microprocessor Research Labs

  5. Carranza J, Theobalt C, Magnor MA, Seidel HP (2003) Free-viewpoint video of human actors. ACM Trans Graph 22(3):569–577

    Article  Google Scholar 

  6. Cyberware. http://www.cyberware.com. Accessed 22 January 2011

  7. D’Apuzzo N, Plankers R, Fua P, Gruen A, Thalmann D (1999) Modeling human bodies from video sequences. In: Proceedings of the SPIE videometrics VI, vol 3461. San Jose, USA, pp 36–47

  8. Dockstader SL, Tekalp AM (2001) Multiple camera tracking of interacting and occluded human motion. Proc IEEE 89(10):1441–1455

    Article  Google Scholar 

  9. Domiter V (2004) Constrained Delaunay triangulation using plane subdivision. In: Proceedings of the 8th central European seminar on computer graphics. Budmerice, pp 105–110

  10. Garland M, Wilmott A, Heckbert PS (2001) Hierarchical face clustering on polygonal surfaces. In: Proceedings of symposium on interactive 3D graphics, pp 49–58

  11. Guan L, Sinha S, Franco JS, Pollefeys M (2006) Visualhull construction in the presence of partial occlusion. In: Proceedings of the third international symposium on 3D data processing, visualization, and transmission (3DPVT06). Chapel Hill, USA, pp 413–420

  12. Hilsmann A, Eisert P (2008) Tracking deformable surfaces with optical flow in the presence of self occlusion in monocular image sequences. In: Proceedings of CVPR workshop on non-rigid shape analysis and deformable image alignment. Anchorage, USA, pp 1–6

  13. Hilton A, Beresford D, Gentils T, Smith R, Sun W (1999) Virtual people: capturing human models to populate virtual worlds. In: Proceedings of the IEEE international conference on computer animation. Geneva, pp 174–185

  14. Iddan GJ, Yahav G (2001) 3D imaging in the studio and elsewhere.... In: Proceedings of the SPIE videometrics and optical methods for 3D shape measurements. San Jose, CA, USA, pp 48–55

  15. Kim SM, Cha J, Ryu J, Lee KH (2006) Depth video enhancement for haptice interaction using a smooth surface reconstruction. IEICE Trans Inf Syst E89-D(1):37–44

    Article  Google Scholar 

  16. Liu YK, Zalik B (2005) An efficient chain code with Huffman coding. Pattern Recog 38(4):553–557

    Article  Google Scholar 

  17. Mikic I, Trivedi M, Hunter E, Cosman P (2003) Human body model acquisition and tracking using voxel data. Int J Comput Vis 53:199–223

    Article  Google Scholar 

  18. OpenCV Library. http://www.intel.com/research/mrl/research/opencv/. Accessed 22 January 2011

  19. Otsu N (1979) A threshold selection method from gray-level histogram. IEEE Trans Syst Man Cybern SMC-8:62–66

    Google Scholar 

  20. Park JC, Kim SM, Lee KH (2006) 3D mesh construction from depth images with occlusion. In: Proceedings of the Pacific conference on multimedia (PCM). LNCS, vol 4261, pp 770–778

  21. Peng E, Li L (2004) 3D Human model acquisition from uncalibrated monocular video. In: Proceedings of the computer vision and graphics international conference, ICCVG 2004. Warsaw, Poland, pp 1018–1023

  22. Peter L (2003) Filling holes in meshes. In: Proceedings of the Eurographics symposium on geometry processing, Aachen, Germany, pp 200–206

  23. Plankers R, Fua P (2001) Tracking and modeling people in video sequences. Comput Vis Image Underst 81:285–302

    Article  Google Scholar 

  24. Remondino F, Roditakis A (2003) Human figure reconstruction and modeling from single image or monocular video sequence. In: Proceddings of the 4th international conference on 3-D digital imaging and modeling (3DIM). Banff, Canada, pp 79–86

  25. Sappa A, Aifanti N, Malassiotis S, Strintzis GM (2003) Monocular 3D human body reconstruction towards depth augmentation of television sequences. In: Proceddings of IEEE int. conf. on image processing. Barcelona, Spain

  26. Schmaltz C, Rosenhahn B, Brox T, Weickert J, Wietzke L, Sommer G (2008) Dealing with self-occlusion in region based motion capture by means of internal regions. In: Articulated motion and deformable objects (AMDO). LNCS, vol 5098. Springer, Heidelberg, pp 102–111

    Chapter  Google Scholar 

  27. Taubin G (1995) A signal processing approach to fair surface design. In: Proceedings of the SIGGRAPH, pp 351–358

  28. Vitronic. http://www.vitronic.de/bodyscannen/. Accessed 22 January 2011

Download references

Acknowledgements

This research was supported by the MKE (The Ministry of Knowledge Economy), Korea, under the ITRC (Information Technology Research Center) support program supervised by the NIPA (National IT Industry Promotion Agency) (NIPA-2010-(C1090-1011-0003)) and also by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 20100018897).

Author information

Authors and Affiliations

  1. 213, Department of Mechatronics, Gwangju Institute of Science and Technology (GIST), 216, Cheomdan-gwagiro, Buk-gu, Gwangju, 500-712, South Korea

    In Yeop Jang, Ji-Ho Cho & Kwan H. Lee

Authors
  1. In Yeop Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ji-Ho Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kwan H. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kwan H. Lee.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jang, I.Y., Cho, JH. & Lee, K.H. 3D human modeling from a single depth image dealing with self-occlusion. Multimed Tools Appl 58, 267–288 (2012). https://doi.org/10.1007/s11042-010-0719-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0719-4

Keywords

Search

Navigation