Skip to main content

Advertisement

SpringerLink
Log in

Consistent 3D human body segmentation based on combinatorial descriptor in spectral domain

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Consistent 3D human body segmentation plays a vital role in many human-oriented applications. Recently research used supervised methods to achieve state-of-the-art performance. However, requiring massive labelled data and tedious training is a costly process. Unsupervised methods do not need labelling and training but struggle to achieve consistent segmentation for a non-rigid deformable mesh. Moreover, the segmentation style is also fixed. In the paper, we aim to achieve high-performance, consistent 3D human mesh segmentation avoiding fully-labelled data and time-consuming training. Specifically, this paper designs a Laplacian operator by incorporating mesh saliency, in which a face-level filter is proposed to improve the detection of concave vertices. Accordingly, we construct a combinatorial descriptor by explicitly employing the global and local attributes derived from the spectrum of the proposed saliency Laplacian operator to achieve consistent segmentation in the spectral domain. An automatic determination mechanism is adopted to determine the number of segments. Extensive experimental results demonstrate that the presented method is effective and efficient for many 3D meshes, especially for the human body shape. The segmentation results are comparable to other state-of-the-art performances without requiring time-consuming labelling and training on large-scale datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Data availability

Princeton Segmentation Benchmark (PSB) is downloaded from https://segeval.cs.princeton.edu/, and TOSCA dataset is downloaded from http://tosca.cs.technion.ac.il/.

References

  1. Au OK-C, Tai C-L, Chu H-K, Cohen-Or D, Lee T-Y (2008) Skeleton extraction by mesh contraction. ACM Trans Graph 27(3):1–10. https://doi.org/10.1145/1360612.1360643

    Article  Google Scholar 

  2. Benjamin W, Polk AW, Vishwanathan SVN, Ramani K (2011) Heat walk: robust salient segmentation of non-rigid shapes. Comput Graph Forum 30(7):2097–2106. https://doi.org/10.1111/j.1467-8659.2011.02060.x

    Article  Google Scholar 

  3. Benzian MY, Benamrane N (2020) 3D Mesh Segmentation by Region growing based on discrete curvature. In: 2020 1st International Conference on Communications, Control Systems and Signal Processing (CCSSP), May 2020, pp 271–276. https://doi.org/10.1109/CCSSP49278.2020.9151831

    Chapter  Google Scholar 

  4. Bronstein AM, Bronstein MM, Kimmel R (2006) Efficient computation of isometry-invariant distances between surfaces. SIAM J Sci Comput 28(5):1812–1836. https://doi.org/10.1137/050639296

    Article  MATH  MathSciNet  Google Scholar 

  5. Chang AX et al (2015) Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012

  6. Chen X, Golovinskiy A, Funkhouser T (2009) A benchmark for 3D mesh segmentation. ACM Trans. Graph 28(3):73:1–73:12. https://doi.org/10.1145/1531326.1531379

    Article  Google Scholar 

  7. Fang Y, Sun M, Kim M, Ramani K (2011) Heat-mapping: a robust approach toward perceptually consistent mesh segmentation. In: CVPR 2011, pp 2145–2152. https://doi.org/10.1109/CVPR.2011.5995695

  8. Gauthier S, Puech W, Bénière R, Subsol G (2017) Digitized 3D mesh segmentation based on curvature analysis. Electron Imaging 2017(20):33–38. https://doi.org/10.2352/ISSN.2470-1173.2017.20.3DIPM-005

    Article  Google Scholar 

  9. George D, Xie X, Tam GK (2018) 3D mesh segmentation via multi-branch 1D convolutional neural networks. Graph Model 96:1–10. https://doi.org/10.1016/j.gmod.2018.01.001

    Article  MathSciNet  Google Scholar 

  10. Golovinskiy A, Funkhouser T (2008) Randomized cuts for 3D mesh analysis. In: ACM SIGGRAPH Asia 2008 papers, New York, NY, USA, pp 1–12. https://doi.org/10.1145/1457515.1409098

    Chapter  Google Scholar 

  11. Guo K, Zou D, Chen X (2016) 3D mesh labeling via deep convolutional neural networks. ACM Trans Graph 35(1):3:1–3:12. https://doi.org/10.1145/2835487

    Article  Google Scholar 

  12. Hanocka R, Hertz A, Fish N, Giryes R, Fleishman S, Cohen-Or D (2019) MeshCNN: a network with an edge. ACM Trans Graph 38(4):90:1–90:12. https://doi.org/10.1145/3306346.3322959

    Article  Google Scholar 

  13. He C, Wang C (2018) A survey on segmentation of 3D models. Wireless Pers Commun 102(4):3835–3842. https://doi.org/10.1007/s11277-018-5414-1

    Article  Google Scholar 

  14. He Y et al (2021) Deep learning based 3D segmentation: a survey. arXiv:2103.05423 [cs], Mar. 2021, [Online]. Available: http://arxiv.org/abs/2103.05423. Accessed 26 July 2021

  15. Hoffman DD, Singh M (1997) Salience of visual parts. Cognition 63(1):29–78

  16. Hou Y, Zhao Y (2021) A robust segmentation algorithm for for 3D complex meshes. J Phys Conf Ser 1802(3):032045. https://doi.org/10.1088/1742-6596/1802/3/032045

    Article  Google Scholar 

  17. Hou Y, Zhao Y, Shan X (2021) 3D mesh segmentation via L0-constrained random walks. Multimed Tools Appl 80(16):24885–24899. https://doi.org/10.1007/s11042-021-10816-0

    Article  Google Scholar 

  18. Huang J, Kwok T-H, Zhou C (2019) Parametric design for human body modeling by wireframe-assisted deep learning. Comput Aided Des 108:19–29

    Article  Google Scholar 

  19. Ji Z, Liu L, Chen Z, Wang G (2006) Easy mesh cutting. Comput Graph Forum 25(3):283–291. https://doi.org/10.1111/j.1467-8659.2006.00947.x

    Article  Google Scholar 

  20. Jiao X, Wu T, Qin X (2017) Mesh segmentation by combining mesh saliency with spectral clustering. J Comput Appl Math, [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0377042717302431. Accessed 16 July 2017

  21. Jung JY, Chee S, Sul IH (2021) Automatic segmentation and 3D printing of A-shaped Manikins using a bounding box and body-feature points. Fashion Textiles 8(1):13. https://doi.org/10.1186/s40691-021-00255-8

    Article  Google Scholar 

  22. Kalogerakis E, Hertzmann A, Singh K (2010) Learning 3D mesh segmentation and labeling. ACM SIGGRAPH 2010 papers, New York, NY, USA, pp 1–12. https://doi.org/10.1145/1833349.1778839

  23. Kin-Chung Au O, Zheng Y, Chen M, Xu P, Tai C-L (2012) Mesh segmentation with concavity-aware fields. IEEE Trans Vis Comput Graph 18(7):1125–1134. https://doi.org/10.1109/TVCG.2011.131

    Article  Google Scholar 

  24. Le T, Bui G, Duan Y (2017) A multi-view recurrent neural network for 3D mesh segmentation. Comput Graph 66:103–112. https://doi.org/10.1016/j.cag.2017.05.011

    Article  Google Scholar 

  25. Lee Y, Lee S, Shamir A, Cohen-Or D, Seidel HP (2004) Intelligent mesh scissoring using 3D snakes. In: 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings, Oct. 2004, pp 279–287. https://doi.org/10.1109/PCCGA.2004.1348358

  26. Lee CH, Varshney A, Jacobs DW (2005) Mesh saliency. In: ACM SIGGRAPH, New York, NY, USA, pp 659–666. https://doi.org/10.1145/1186822.1073244

  27. Levy B (2006) Laplace-Beltrami eigenfunctions towards an algorithm that ‘understands’ geometry. In: IEEE international conference on shape modeling and applications 2006 (SMI’06), p 13. https://doi.org/10.1109/SMI.2006.21

    Chapter  Google Scholar 

  28. Lv J, Chen X, Huang J, Bao H (2012) Semi-supervised mesh segmentation and labeling. Comput Graph Forum 31(7):2241–2248. https://doi.org/10.1111/j.1467-8659.2012.03217.x

    Article  Google Scholar 

  29. Mejia D, Ruiz-Salguero O, Cadavid CA (2017) Spectral-based mesh segmentation. Int J Interact Des Manuf (IJIDeM) 11(3):503–514. https://doi.org/10.1007/s12008-016-0300-0

    Article  Google Scholar 

  30. Pal P (2012) Fast freeform hybrid reconstruction with manual mesh segmentation. Int J Adv Manuf Technol 63(9):1205–1215. https://doi.org/10.1007/s00170-012-3986-6

    Article  Google Scholar 

  31. Rodrigues RSV, Morgado JFM, Gomes AJP (2018) Part-based mesh segmentation: a survey. Comput Graph Forum 37(6):235–274. https://doi.org/10.1111/cgf.13323

    Article  Google Scholar 

  32. Rustamov RM (2007) Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In: Proceedings of the fifth Eurographics symposium on Geometry processing, Goslar, DEU, pp 225–233

  33. Shu Z, Yang S, Wu H, Xin S, Pang C, Kavan L, Liu L (2022) 3D shape segmentation using soft density peak clustering and semi-supervised learning. Comput Aided Des 145:103181. https://doi.org/10.1016/j.cad.2021.103181

    Article  MathSciNet  Google Scholar 

  34. Song R, Liu Y, Martin RR, Rosin PL (2014) Mesh saliency via spectral processing. ACM Trans Graph 33(1):6:1–6:17. https://doi.org/10.1145/2530691

    Article  MATH  Google Scholar 

  35. Sun J, Ovsjanikov M, Guibas L (2009) A concise and provably informative multi-scale signature based on heat diffusion. Comput Graph Forum 28(5):1383–1392. https://doi.org/10.1111/j.1467-8659.2009.01515.x

    Article  Google Scholar 

  36. Tao S, Huang Z, Ma L, Guo S, Wang S, Xie Y (2013) Partial retrieval of CAD models based on local surface region decomposition. Comput Aided Des 45(11):1239–1252. https://doi.org/10.1016/j.cad.2013.05.008

    Article  Google Scholar 

  37. Theologou P, Pratikakis I, Theoharis T (2017) Unsupervised spectral mesh segmentation driven by heterogeneous graphs. IEEE Trans Pattern Anal Mach Intell 39(2):397–410. https://doi.org/10.1109/TPAMI.2016.2544311

    Article  Google Scholar 

  38. Tong W, Yang X, Pan M, Chen F (2020) Spectral mesh segmentation via L0 gradient minimization. IEEE Trans Vis Comput Graph 26(4):1807–1820. https://doi.org/10.1109/TVCG.2018.2882212

    Article  Google Scholar 

  39. Vasilakis AA, Fudos I (2014) Pose partitioning for multi-resolution segmentation of arbitrary mesh animations. Comput Graph Forum 33(2):293–302. https://doi.org/10.1111/cgf.12327

    Article  Google Scholar 

  40. Wang H, Lu T, Au OK-C, Tai C-L (2014) Spectral 3D mesh segmentation with a novel single segmentation field. Graph Model 76(5):440–456. https://doi.org/10.1016/j.gmod.2014.04.009

    Article  Google Scholar 

  41. Wang P, Gan Y, Shui P, Yu F, Zhang Y, Chen S, Sun Z (2018) 3D shape segmentation via shape fully convolutional networks. Comput Graph 70:128–139

    Article  Google Scholar 

  42. Yamauchi H, Gumhold S, Zayer R, Seidel H-P (2005) Mesh segmentation driven by Gaussian curvature. Vis Comput 21(8):659–668. https://doi.org/10.1007/s00371-005-0319-x

    Article  Google Scholar 

  43. Yang S, Wang R, Zhao W, Ke Y (2020) 3D intelligent scissors for dental mesh segmentation. Comput Math Methods Med 2020:e1394231. https://doi.org/10.1155/2020/1394231

    Article  MATH  Google Scholar 

  44. Yi B, Liu Z, Tan J, Cheng F, Duan G, Liu L (2014) Shape recognition of CAD models via iterative slippage analysis. Comput Aided Des 55:13–25. https://doi.org/10.1016/j.cad.2014.04.008

    Article  Google Scholar 

  45. Zhang L, Guo J, Xiao J, Zhang X, Yan D-M (2020) Blending surface segmentation and editing for 3D models. IEEE Trans Vis Comput Graph 28:1. https://doi.org/10.1109/TVCG.2020.3045450

    Article  Google Scholar 

  46. Zhong Y, Li D, Wu G, Hu P (2018) Automatic body measurement based on slicing loops. Int J Cloth Sci Technol 30(3):380–397. https://doi.org/10.1108/IJCST-06-2017-0086

    Article  Google Scholar 

  47. Zhou Y, Huang Z (2004) Decomposing polygon meshes by means of critical points. In: 10th International Multimedia Modelling Conference, 2004. Proceedings, Jan. 2004, pp 187–195. https://doi.org/10.1109/MULMM.2004.1264985

Download references

Funding

This work is supported by the Key Science and Technology Program of Henan Province, China [Grant No. 222102210124], National Natural Science Foundation of China [Grant No. 61572124], Key Program of Higher Education Teaching Reform Research and Practice, Henan, China [Grant No. 2021SJGLX167].

Author information

Authors and Affiliations

  1. College of Information Engineering, North China University of Water Resources and Electric Power, Zhengzhou, 450046, China

    Haoyang Xie

  2. College of Textiles, Donghua University, Shanghai, 201620, China

    Yueqi Zhong

  3. Key Laboratory of Textile Science & Technology of Ministry of Education, Donghua University, Shanghai, 201620, China

    Yueqi Zhong

Authors
  1. Haoyang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yueqi Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haoyang Xie.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, H., Zhong, Y. Consistent 3D human body segmentation based on combinatorial descriptor in spectral domain. Multimed Tools Appl 82, 27927–27947 (2023). https://doi.org/10.1007/s11042-023-14729-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14729-y

Keywords

Search

Navigation