Abstract
Restoring the color of the 3D human body is a critical issue when reconstructing a 3D human body model from a single-view image. Nonetheless, previous methods have not made use of the reconstructed 3D human body features to aid color restoration, leading to less-than-optimal results. We propose a new method called CHMF in this paper. Unlike previous works, this method fully exploits the inherent features of the 3D human body for color restoration. To achieve this, we first design a deep fix network to extract structural features of the 3D human body to repair color features extracted from single-view image. Then, we utilize a mesh encoding network to extract the shape features of the 3D human body, combining the repaired color features with the transformed shape features to compute the final color. The 3D human body reconstructed by this method not only preserves excellent texture details in terms of color but also eliminates color errors caused by limb occlusion due to human movement, a limitation in existing methods. We evaluate our method on the Thuman2.0 dataset, and extensive experiments have shown that our approach outperforms previous methods both qualitatively and quantitatively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Feng, Y., Feng, Y., You, H., Zhao, X., Gao, Y.: MeshNet: mesh neural network for 3d shape representation. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, pp. 8279–8286 (2019)
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: SMPL: a skinned multiperson linear model. In: Seminal Graphics Papers: Pushing the Boundaries, vol. 2, pp. 851–866 (2023)
Maron, H., et al.: Convolutional neural networks on surfaces via seamless toric covers. ACM Trans. Graph. 36(4), 71–81 (2017)
Oechsle, M., Mescheder, L., Niemeyer, M., Strauss, T., Geiger, A.: Texture fields: learning texture representations in function space. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4531–4540 (2019)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)
Saito, S., Huang, Z., Natsume, R., Morishima, S., Kanazawa, A., Li, H.: PIFU: pixel-aligned implicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2304–2314 (2019)
Singh, V.V., Sheshappanavar, S.V., Kambhamettu, C.: Meshnet++: a network with a face. In: ACM Multimedia. pp. 4883–4891 (2021)
Wang, J., Zhong, Y., Li, Y., Zhang, C., Wei, Y.: Re-identification supervised texture generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11846–11856 (2019)
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graphics (tog) 38(5), 1–12 (2019)
Wiersma, R., Nasikun, A., Eisemann, E., Hildebrandt, K.: Deltaconv: anisotropic operators for geometric deep learning on point clouds. ACM Trans. Graphics (TOG) 41(4), 1–10 (2022)
Xiu, Y., Yang, J., Tzionas, D., Black, M.J.: ICON: implicit clothed humans obtained from normals. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13286–13296. IEEE (2022)
Xu, X., Chen, H., Moreno-Noguer, F., Jeni, L.A., De la Torre, F.: 3D human pose, shape and texture from low-resolution images and videos. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 4490–4504 (2021)
Yu, T., Zheng, Z., Guo, K., Liu, P., Dai, Q., Liu, Y.: Function4d: real-time human volumetric capture from very sparse consumer RGBD sensors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5746–5756 (2021)
Zhang, H., Tian, Y., Zhou, X., Ouyang, W., Liu, Y., Wang, L., Sun, Z.: PyMAF: 3D human pose and shape regression with pyramidal mesh alignment feedback loop. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11446–11456 (2021)
Zhao, F., Liao, S., Zhang, K., Shao, L.: Human parsing based texture transfer from single image to 3D human via cross-view consistency. Adv. Neural. Inf. Process. Syst. 33, 14326–14337 (2020)
Zheng, Z., Yu, T., Liu, Y., Dai, Q.: PaMir: parametric model-conditioned implicit representation for image-based human reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 44(6), 3170–3184 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wan, Y., Yu, J., Yao, L. (2024). CHMF: Colorful Human Reconstruction Based on Mesh Features. In: Huang, DS., Pan, Y., Guo, J. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14873. Springer, Singapore. https://doi.org/10.1007/978-981-97-5615-5_17
Download citation
DOI: https://doi.org/10.1007/978-981-97-5615-5_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5614-8
Online ISBN: 978-981-97-5615-5
eBook Packages: Computer ScienceComputer Science (R0)