Skip to main content
SpringerLink
Log in

A fast and efficient 3D reflection symmetry detector based on neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Determining the 3D reflection symmetry planes from 3D models is very difficult and time-consuming. In this paper, we formulate the symmetry detection as a per-point classification problem and present a deep neural network based method to solve it. During the training procedure, we firstly collect a lot of CAD mesh models with reflection symmetry as the training data, and then convert each mesh model into a dense point cloud with points located on the symmetry planes labeled as positive. Based on the PointNet++ architecture, we train a multi-scale deep neural network to capture the reflection symmetry property from the point cloud automatically. In addition, a novel weighted cross-entropy loss function is adopted to balance the positive and the negative samples. During the inference procedure, we firstly feed the down-sampled point cloud into the trained neural network. Then, the output per-point classification result is used to calculate an initial symmetry plane equation with RANSAC strategy and the least square method. Finally, iterative closest point algorithm is performed to optimize the fitted symmetry plane. Experimental results on both the synthetic and the real data demonstrate the efficiency, robustness and flexibility of our approach. Our method is pretty fast and generates comparable or better results than the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) Tensorflow: a system for large-scale machine learning. In: OSDI

  2. Atadjanov IR, Lee S (2016) Reflection symmetry detection via appearance of structure descriptor. In: ECCV

  3. Besl PJ, McKay ND, et al. (1992) A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256

    Article  Google Scholar 

  4. Bokeloh M, Berner A, Wand M, Seidel HP, Schilling A (2009) Symmetry detection using line features. Comput Graph Forum 28(2):697–706

    Article  Google Scholar 

  5. Choi S, Zhou QY, Koltun V (2015) Robust reconstruction of indoor scenes. In: CVPR

  6. Ecins A, Fermüller C, Aloimonos Y (2017) Detecting reflectional symmetries in 3d data through symmetrical fitting. In: ICCV Workshop

  7. Ecins A, Fermüller C, Aloimonos Y (2016) Cluttered scene segmentation using the symmetry constraint. In: IEEE International conference on robotics and automation, pp 2271–2278

  8. Funk C, Lee S, Oswald MR, Tsogkas S, Shen W, Cohen A, Dickinson S, Liu Y (2017) 2017 iccv challenge: Detecting symmetry in the wild. In: ICCV Workshop

  9. Funk C, Liu Y (2017) Beyond planar symmetry: Modeling human perception of reflection and rotation symmetries in the wild. In: ICCV

  10. Ivan S, Robert G, Tobias S (2015) Approximate symmetry detection in partial 3d meshes. Comput Graph Forum 33(7):131–140

    Google Scholar 

  11. Kazhdan M, Funkhouser T, Rusinkiewicz S (2004) Symmetry descriptors and 3d shape matching. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing

  12. Kim VG, Lipman Y, Chen X, Funkhouser T (2010) Mbius transformations for global intrinsic symmetry analysis. Comput Graph Forum 29(5):1689–1700

    Article  Google Scholar 

  13. Korman S, Litman R, Avidan S, Bronstein A (2015) Probably approximately symmetric: Fast rigid symmetry detection with global guarantees. Computer Graphics Forum 34(1):2–13

    Article  Google Scholar 

  14. Lan X, Ma AJ, Yuen PC, Chellappa R (2015) Joint sparse representation and robust feature-level fusion for multi-cue visual tracking. IEEE Trans Image Process 24(12):5826–5841

    Article  MathSciNet  MATH  Google Scholar 

  15. Lan X, Ye M, Zhang S, Zhou H, Yuen PC (2018) Modality-correlation-aware sparse representation for rgb-infrared object tracking. Pattern Recognition Letters

  16. Lan X, Zhang S, Yuen PC, Chellappa R (2018) Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans Image Process 27(4):2022– 2037

    Article  MathSciNet  MATH  Google Scholar 

  17. Lan X, Ye M, Shao R, Zhong B, Yuen PC, Zhou H (2019) Learning modality-consistency feature templates: a robust rgb-infrared tracking system. IEEE Trans Ind Electron:1–1

  18. Li B, Johan H, Ye Y, Lu Y (2016) Efficient 3d reflection symmetry detection: a view-based approach. Graph Model 83:2–14

    Article  MathSciNet  Google Scholar 

  19. Li Y, Bu R, Sun M, Wu W, Di X, Chen B (2018) Pointcnn: Convolution on x-transformed points. In: NIPS

  20. Liu Y, Hel-Or H, Kaplan CS, Gool LJV (2010) Computational symmetry in computer vision and computer graphics. Found Trends Comput Graph Vis 5:1–199

    Article  MATH  Google Scholar 

  21. Loy G, Eklundh JO (2006) Detecting symmetry and symmetric constellations of features. In: ECCV

  22. Maron H, Galun M, Aigerman N, Trope M, Kim VG, Kim VG, Kim VG, Lipman Y (2017) Convolutional neural networks on surfaces via seamless toric covers. ACM Trans Graph 36(4):1–10

    Article  Google Scholar 

  23. Martin D, Fowlkes C, Tal D, Malik J (2002) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV

  24. Mitra NJ, Guibas L, Pauly M (2006) Partial and approximate symmetry detection for 3d geometry. ACM Trans Graph 25(3):560–568

    Article  Google Scholar 

  25. Mitra NJ, Pauly M, Wand M, Ceylan D (2013) Symmetry in 3d geometry: Extraction and applications. Comput Graph Forum 32(6):1–23

    Article  Google Scholar 

  26. Oord Avd, Kalchbrenner N, Kavukcuoglu K (2016) Pixel recurrent neural networks. ICML

  27. Ovsjanikov M, Sun J, Guibas L (2008) Global intrinsic symmetries of shapes. In: Symposium on geometry processing

  28. Pollefeys M, Sinha SN, Zach C, Cohen A (2012) Discovering and exploiting 3d symmetries in structure from motion. In: CVPR

  29. Qi CR, Su H, Mo K, Guibas L (2017) Pointnet: Deep learning on point sets for 3d classification and segmentation. In: CVPR

  30. Qi CR, Yi L, Su H, Guibas L (2017) Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: NIPS

  31. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2019) Hybridsn: Exploring 3d-2d cnn feature hierarchy for hyperspectral image classification. Arxiv

  32. Shen W, Zhao K, Jiang Y, Wang Y, Zhang Z, Bai X (2016) Object skeleton extraction in natural images by fusing scale-associated deep side outputs. In: CVPR

  33. Shi B, Bai S, Zhou Z, Bai X (2015) Deeppano: Deep panoramic representation for 3-d shape recognition. IEEE Signal Process Lett 22(12):2339–2343

    Article  Google Scholar 

  34. Speciale P, Oswald MR, Cohen A, Pollefeys M (2016) A symmetry prior for convex variational 3d reconstruction. In: ECCV

  35. Su H, Maji S, Kalogerakis E, Learned-miller EG (2015) Multi-view convolutional neural networks for 3d shape recognition. In: ICCV, pp 945–953

  36. Sun C, Sherrah J (1997) 3d symmetry detection using the extended gaussian image. IEEE Trans Pattern Anal Mach Intell 19(2):164–168

    Article  Google Scholar 

  37. Telea A, Van Wijk JJ (2002) An augmented fast marching method for computing skeletons and centerlines. In: Proceedings of the Symposium on Data Visualisation

  38. Tsogkas S, Kokkinos I (2012) Learning-based Symmetry Detection in Natural Images, vol 7578. Springer, Berlin

    Google Scholar 

  39. Wang PS, Liu Y, Guo YX, Sun CY, Tong X (2017) O-cnn: octree-based convolutional neural networks for 3d shape analysis. ACM Trans Graph 36(4):1–11

    Google Scholar 

  40. Wang L, Qian X, Zhang Y, Shen J, Cao X (2019) Enhancing sketch-based image retrieval by cnn semantic re-ranking. IEEE Transactions on Cybernetics:1–13

  41. Wen Y, Gao Y, Hong R, Luan H, Liu Q, Shen J, Ji R (2012) View-based 3d object retrieval by bipartite graph matching. In: ACM Multimedia

  42. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2014) 3d shapenets: a deep representation for volumetric shapes. In: CVPR, pp 1912–1920

  43. Xiao J, Feng Y, Ji M, Zhuang Y (2016) Fast view-based 3d model retrieval via unsupervised multiple feature fusion and online projection learning. Signal Process 120:702–713

    Article  Google Scholar 

  44. Xie S, Tu Z (2015) Holistically-nested edge detection. Int J Comput Vis 125 (1-3):3–18

    Article  MathSciNet  Google Scholar 

  45. Xie J, Zhu F, Dai G, Shao L, Fang Y (2017) Progressive shape-distribution-encoder for learning 3d shape representation. IEEE Trans Image Process 26(3):1231–1242

    Article  MathSciNet  MATH  Google Scholar 

  46. Xu K, Zhang H, Tagliasacchi A, Liu L, Li G, Meng M, Xiong Y (2009) Partial intrinsic reflectional symmetry of 3d shapes. ACM Trans Graph 28 (5):1–10

    Google Scholar 

  47. Xu K, Zhang H, Jiang W, Dyer R, Cheng Z, Liu L, Chen B (2012) Multi-scale partial intrinsic symmetry detection. ACM Trans Graph 31(6):1–11

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their valuable comments. This work was supported by the National Natural Science Foundation of China under Grants Nos. 61872317

Author information

Authors and Affiliations

  1. The State Key Lab of CAD, CG, Zhejiang University, Hangzhou, China

    Penglei Ji & Xinguo Liu

Authors
  1. Penglei Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinguo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinguo Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, P., Liu, X. A fast and efficient 3D reflection symmetry detector based on neural networks. Multimed Tools Appl 78, 35471–35492 (2019). https://doi.org/10.1007/s11042-019-08043-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08043-9

Keywords

Search

Navigation