Abstract
Automatically retrieving 3D human shapes from a single 2D image is a challenging problem. The 2D nature of one image makes it difficult to infer depth, pose and style. We propose a novel method for 3D human shape retrieval based on a single image. We present a single-network approach for keypoints detection, which entails simultaneous localization of internal region keypoints and the outer contour keypoints. The network is trained by using multi-task learning, which can handle scale differences between body/foot and face/hand keypoints through an improved architecture. Based on the keypoints, we can estimate the 3D pose, which is used for 3D pose retrieval. From the outer contour keypoints, the 2D closed boundary curve can be automatically generated. We formulate the 2D curve to 3D human shapes similarity calculation as an energy minimization problem for more sophisticated retrieval. Experimental results show that our method can achieve satisfactory retrieval performance on the two benchmark datasets.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chang, A.: Shapenet: An Information-Rich 3D Model Repository. Technical report, Stanford University-Princeton University-Toyota Technological Institute at Chicago (2015)
Dyke, R., Stride, C., Lai, Y., Rosin, P.: SHREC-19: shape correspondence with isometric and non-isometric deformations. In: Eurographics Workshop on 3D Object Retrieval. Genova, Italy, 5–6 May (2019)
Ameesh, M., Kostas, D.: Spherical correlation of visual representations for 3D model retrieval. Int. J. Comput. Vis. 89, 193–210 (2010)
Chao, M., Yulan, G., Jungang, Y., Wei, A.: Learning multi-view representation with LSTM for 3-D shape recognition and retrieval. IEEE Trans. Multimed. 21(5), 1169–1182 (2019)
Xun, Y., Yongsheng, G., Jun, Z.: Sparse 3D directional vertices vs continuous 3D curves: Efficient 3D surface matching and its application for single model face recognition. Pattern Recogn. 65, 296–306 (2017)
Groueix, T., Fisher, M., Kim, V.G., Russell B.C., Aubry, M.: 3D-CODED: 3D correspondences by deep deformation. In: The European Conference on Computer Vision (ECCV) (2018)
Juefei, Y., et al.: SHREC’18 Track: 2D scene sketch-based 3D scene retrieval. In: Eurographics Workshop on 3D Object Retrieval. Delft, The Netherlands (2018)
Jiantao, P., Karthik, R.: A 3D model retrieval method using 2D freehand sketches. In: International Conference on Computational Science. pp. 343–346, (2005)
Zorah, L., et al.: Efficient globally optimal 2D-to-3D deformable shape matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Pickup, D., Sun, X., Rosin, P., Martin, R.: Skeleton-based canonical forms for non-rigid 3D shape retrieval. Comput. Visual Media 2(3), 231–243 (2016). https://doi.org/10.1007/s41095-016-0045-5
Jin, X., Guoxian, D., Fan, Z., Ling, S., Yi, F.: Deep nonlinear metric learning for 3-D shape retrieval. IEEE Trans. Cybernet. 48 (1), 412–422, (2018)
Kocabas, M., Salih, K., Emre, A.: MultiPoseNet: fast multi-person pose estimation using pose residual network. In: European Conference on Computer Vision (ECCV) (2018)
Alexander, G., Peter, M.R., Vincent, L.: 3D pose estimation and 3D model retrieval for objects in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 99–110 (2018)
Nhat Vu, B.S.: Manjunath.: shape prior segmentation of multiple objects with graph cuts. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2008)
Pengjie, L., Huadong, M., Anlong, M.: Non-rigid 3D model retrieval using multi-scale local features. In: ACM Multimedia. pp. 1425–1428 (2011)
Mehta, D., et al.: Vnect: Real-time 3d human pose estimation with a single rgb camera. ACM Trans. Graph. (TOG). 36(4), 1–14 (2017)
Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grant No. 11901037.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, P., Shen, Y. (2020). An Image-Based Method for 3D Human Shapes Retrieval. In: Yu, Z., Becker, C., Xing, G. (eds) Green, Pervasive, and Cloud Computing. GPC 2020. Lecture Notes in Computer Science(), vol 12398. Springer, Cham. https://doi.org/10.1007/978-3-030-64243-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-64243-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64242-6
Online ISBN: 978-3-030-64243-3
eBook Packages: Computer ScienceComputer Science (R0)