Abstract
This paper presents a 2D to 3D conversion scheme to generate a 3D human model using a single depth image with several color images. In building a complete 3D model, no prior knowledge such as a pre-computed scene structure and photometric and geometric calibrations is required since the depth camera can directly acquire the calibrated geometric and color information in real time. The proposed method deals with a self-occlusion problem which often occurs in images captured by a monocular camera. When an image is obtained from a fixed view, it may not have data for a certain part of an object due to occlusion. The proposed method consists of following steps to resolve this problem. First, the noise in a depth image is reduced by using a series of image processing techniques. Second, a 3D mesh surface is constructed using the proposed depth image-based modeling method. Third, the occlusion problem is resolved by removing the unwanted triangles in the occlusion region and filling the corresponding hole. Finally, textures are extracted and mapped to the 3D surface of the model to provide photo-realistic appearance. Comparison results with the related work demonstrate the efficiency of our method in terms of visual quality and computation time. It can be utilized in creating 3D human models in many 3D applications.


















Similar content being viewed by others
References
Allen B, Curless B, Popovic Z (2003) The space of human body shapes: reconstruction and parameterization from range images. In: Proceedings of ACM SIGGRAPH 2003, pp 587–594
Atiqur RA, Tan JK, Kim HS, Ishikawa S (2008) Solutions to motion self-occlusion problem in human activity analysis. In: Proceedings of the 11th international conference on computer and information technology (ICCIT), Article no. 4803095. Khulna, Bangladesh, pp 201–206
Bhasin SS, Chaudhuri S (2001) Depth from defocus in presence of partial self occlusion. In: Proceedings of the international conference on computer vision, vol 2. Vancouver, Canada, pp 488–493
Bouguet JY (2000) Pyramidal implementation of the Lucas Kanade feature tracker. Intel Corporation, Microprocessor Research Labs
Carranza J, Theobalt C, Magnor MA, Seidel HP (2003) Free-viewpoint video of human actors. ACM Trans Graph 22(3):569–577
Cyberware. http://www.cyberware.com. Accessed 22 January 2011
D’Apuzzo N, Plankers R, Fua P, Gruen A, Thalmann D (1999) Modeling human bodies from video sequences. In: Proceedings of the SPIE videometrics VI, vol 3461. San Jose, USA, pp 36–47
Dockstader SL, Tekalp AM (2001) Multiple camera tracking of interacting and occluded human motion. Proc IEEE 89(10):1441–1455
Domiter V (2004) Constrained Delaunay triangulation using plane subdivision. In: Proceedings of the 8th central European seminar on computer graphics. Budmerice, pp 105–110
Garland M, Wilmott A, Heckbert PS (2001) Hierarchical face clustering on polygonal surfaces. In: Proceedings of symposium on interactive 3D graphics, pp 49–58
Guan L, Sinha S, Franco JS, Pollefeys M (2006) Visualhull construction in the presence of partial occlusion. In: Proceedings of the third international symposium on 3D data processing, visualization, and transmission (3DPVT06). Chapel Hill, USA, pp 413–420
Hilsmann A, Eisert P (2008) Tracking deformable surfaces with optical flow in the presence of self occlusion in monocular image sequences. In: Proceedings of CVPR workshop on non-rigid shape analysis and deformable image alignment. Anchorage, USA, pp 1–6
Hilton A, Beresford D, Gentils T, Smith R, Sun W (1999) Virtual people: capturing human models to populate virtual worlds. In: Proceedings of the IEEE international conference on computer animation. Geneva, pp 174–185
Iddan GJ, Yahav G (2001) 3D imaging in the studio and elsewhere.... In: Proceedings of the SPIE videometrics and optical methods for 3D shape measurements. San Jose, CA, USA, pp 48–55
Kim SM, Cha J, Ryu J, Lee KH (2006) Depth video enhancement for haptice interaction using a smooth surface reconstruction. IEICE Trans Inf Syst E89-D(1):37–44
Liu YK, Zalik B (2005) An efficient chain code with Huffman coding. Pattern Recog 38(4):553–557
Mikic I, Trivedi M, Hunter E, Cosman P (2003) Human body model acquisition and tracking using voxel data. Int J Comput Vis 53:199–223
OpenCV Library. http://www.intel.com/research/mrl/research/opencv/. Accessed 22 January 2011
Otsu N (1979) A threshold selection method from gray-level histogram. IEEE Trans Syst Man Cybern SMC-8:62–66
Park JC, Kim SM, Lee KH (2006) 3D mesh construction from depth images with occlusion. In: Proceedings of the Pacific conference on multimedia (PCM). LNCS, vol 4261, pp 770–778
Peng E, Li L (2004) 3D Human model acquisition from uncalibrated monocular video. In: Proceedings of the computer vision and graphics international conference, ICCVG 2004. Warsaw, Poland, pp 1018–1023
Peter L (2003) Filling holes in meshes. In: Proceedings of the Eurographics symposium on geometry processing, Aachen, Germany, pp 200–206
Plankers R, Fua P (2001) Tracking and modeling people in video sequences. Comput Vis Image Underst 81:285–302
Remondino F, Roditakis A (2003) Human figure reconstruction and modeling from single image or monocular video sequence. In: Proceddings of the 4th international conference on 3-D digital imaging and modeling (3DIM). Banff, Canada, pp 79–86
Sappa A, Aifanti N, Malassiotis S, Strintzis GM (2003) Monocular 3D human body reconstruction towards depth augmentation of television sequences. In: Proceddings of IEEE int. conf. on image processing. Barcelona, Spain
Schmaltz C, Rosenhahn B, Brox T, Weickert J, Wietzke L, Sommer G (2008) Dealing with self-occlusion in region based motion capture by means of internal regions. In: Articulated motion and deformable objects (AMDO). LNCS, vol 5098. Springer, Heidelberg, pp 102–111
Taubin G (1995) A signal processing approach to fair surface design. In: Proceedings of the SIGGRAPH, pp 351–358
Vitronic. http://www.vitronic.de/bodyscannen/. Accessed 22 January 2011
Acknowledgements
This research was supported by the MKE (The Ministry of Knowledge Economy), Korea, under the ITRC (Information Technology Research Center) support program supervised by the NIPA (National IT Industry Promotion Agency) (NIPA-2010-(C1090-1011-0003)) and also by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 20100018897).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jang, I.Y., Cho, JH. & Lee, K.H. 3D human modeling from a single depth image dealing with self-occlusion. Multimed Tools Appl 58, 267–288 (2012). https://doi.org/10.1007/s11042-010-0719-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0719-4