Abstract
Single-view 3D clothed human reconstruction is a challenging task, not only because of the need to infer the complex global topology of human body but also due to the requirement to recover delicate surface details. In this paper, a method named HEI-Human is proposed to hybridize an explicit model and an implicit model for 3D clothed human reconstruction. In the explicit model, the SMPL model is voxelized and then integrated into a 3D hourglass network to supervise the global geometric aligned features extraction. In the implicit model, 2D aligned features are first extracted by a 2D hourglass network, and then an implicit surface function is employed to construct the occupancy field of human body using the hybrid 2D and 3D aligned features. As the explicit model and implicit model are mutually beneficial, our HEI-Human method not only generates reconstructions with plausible global topology but also recovers rich and accurate surface details. The HEI-Human is evaluated on the current largest publicly available dataset, and the experimental results demonstrate that our method outperforms the state-of-the-art methods including DeepHuman, PIFu, and GeoPIFu.
This work was supported by the National Natural Science Foundation of China (62077026, 61937001), the Fundamental Research Funds for the Central Universities (CCNU19ZN004), and the Research Project of Graduate Teaching Reform of CCNU (2019JG01).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Pons-Moll, G., Pujades, S., Hu, S., et al.: ClothCap: Seamless 4D clothing capture and retargetting. ACM Trans. Graph. (TOG) 36(4), 1–15 (2017)
Cha, W., Price, T., Wei, Z., et al.: Towards fully mobile 3D face, body, and environment capture using only head-worn cameras. IEEE Trans. Visual. Comput. Graph. 24(11), 2993–3004 (2018)
Ji, M., Gall, J., Zheng, H., et al.: Surfacenet: an end-to-end 3D neural network for multiview stereopsis. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2307–2315. IEEE, Italy (2017)
Yu, T., Zheng, Z., Guo, K., et al.: Doublefusion: real-time capture of human performances with inner body shapes from a single depth sensor. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7287–7296. IEEE, Salt Lake City (2018)
Zheng, Z., Yu, T., Wei, Y., et al.: Deephuman: 3D human reconstruction from a single image. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7739–7749. IEEE, Korea (2019)
Saito, S., Huang, Z., Natsume, R., et al.: PIFu: pixel-aligned implicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2304–2314. IEEE, Korea (2019)
He, T., Collomosse, J., Jin, H., et al.: Geo-PIFu: geometry and pixel aligned implicit functions for single-view human reconstruction. In: Advances in Neural Information Processing Systems, pp. 9276–9287, NIPS (2020)
Loper, M., Mahmood, N., Romero, J., et al.: SMPL: a skinned multi-person linear model. ACM Trans. Graph. (TOG) 34(6), 1–16 (2015)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Proceedings of the IEEE Conference on European Conference on Computer Vision, pp. 483–499. IEEE, Holland (2016)
Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 605–613. IEEE, U.S.A (2017)
Kanazawa, A., Black, J., Jacobs, W., et al.: End-to-end recovery of human shape and pose. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7122–7131. IEEE, U.S.A (2018)
Kolotouros, N., Pavlakos, G., Black, M.J., et al.: Learning to reconstruct 3D human pose and shape via model-fitting in the loop. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2252–2261. IEEE, Korea (2019)
Zhang, H., Cao, J., Lu, G., et al.: Learning 3D human shape and pose from dense body parts. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Kolotouros, N., Pavlakos, G., Daniilidis, K.: Convolutional mesh regression for single-image human shape reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4501–4510. IEEE, U.S.A (2019)
Alldieck, T., Pons-Moll, G., Theobalt, C., et al.: Tex2shape: detailed full human body geometry from a single image. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2293–2303. IEEE, Korea (2019)
Varol, G., Ceylan, D., Russell, B., et al.: Bodynet: volumetric inference of 3D human body shapes. In: Proceedings of the European Conference on Computer Vision, pp. 20–36. IEEE, Germany (2018)
Onizuka, H., Hayirci, Z., Thomas, D., et al.: TetraTSDF: 3D human reconstruction from a single image with a tetrahedral outer shell. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 6011–6020. IEEE(2020)
Jackson, S., Manafas, C., Tzimiropoulos, G.: 3D human body reconstruction from a single image via volumetric regression. In: Proceedings of the European Conference on Computer Vision Workshops. IEEE, Germany (2018)
Saito, S., Simon, T., Saragih, J., et al.: PIFuHD: multi-level pixel-aligned implicit function for high-resolution 3D human digitization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 84–93. IEEE (2020)
Huang, Z., Xu, Y., Lassner, C., et al.: Arch: animatable reconstruction of clothed humans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3093–3102. IEEE (2020)
Natsume, R., Saito, S., Huang, Z., et al.: Siclope: silhouette-based clothed people. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4480–4490. IEEE, U.S.A (2019)
Lorensen, E., Cline, E.: Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput. Graph. 21(4), 163–169 (1987)
Bogo, F., Kanazawa, A., Lassner, C., et al.: Keep it SMPL: automatic estimation of 3D human pose and shape from a single image. In: Proceedings of the IEEE International Conference on European Conference on Computer Vision, pp. 561–578. IEEE, Holland (2016)
Jackson, S., Bulat, A., Argyriou, V., et al.: Large pose 3D face reconstruction from a single image via direct volumetric CNN regression. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1031–1039. IEEE, Italy (2017)
Paszke, A., Gross, S., Massa, F., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, pp. 8024–8035. NIPS, Canada (2019)
Liu, L., Ke, Z., Huo, J., et al.: Head pose estimation through keypoints matching between reconstructed 3D face model and 2D image. Sensors 21(5), 1841 (2021)
Liu, L., Zhang, L., Chen, J.: Progressive pose normalization generative adversarial network for frontal face synthesis and face recognition under large pose. In: Proceedings of the IEEE International Conference on Image Processing, pp. 4434–4438. IEEE, Taibei, China (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, L., Sun, J., Gao, Y., Chen, J. (2021). HEI-Human: A Hybrid Explicit and Implicit Method for Single-View 3D Clothed Human Reconstruction. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13020. Springer, Cham. https://doi.org/10.1007/978-3-030-88007-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-88007-1_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88006-4
Online ISBN: 978-3-030-88007-1
eBook Packages: Computer ScienceComputer Science (R0)