Skip to main content
SpringerLink
Log in

Object restoration based on extrinsic reflective symmetry plane detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Object restoration is applied in multitudinous fields. It is common that the models that need to be restored have extrinsic reflective symmetry features (ERSF), the detections of which are often applied to their restorations. However, most of these symmetry detection approaches have limitations, especially for the incompletion models with a greater area of missing part and much more complex and obvious features. Therefore, this research proposes a novel object restoration method (ORM-ERSPD) based on extrinsic reflective symmetry plane (ERSP) detection, which can be divided into two steps: extrinsic reflective symmetry plane detection (ERSPD) and object restoration (OR). During ERSPD, the reflected mesh can be computed based on an initial ERSP and aligned to the selected mesh by applying the iterative closest point algorithm. Thus, the reflective middle point set can be obtained between the selected mesh and the aligned reflected mesh to fit for the final ERSP. In OR, the selected mesh is first mirrored, aligned and deformed to the missing part whose boundary is reflected on the complete part for selecting and expanding the mesh. Then, the Boolean operation between the reflected mesh and the input mesh is conducted. Finally, the proposed ORM-ERSPD is applied to a set of incompletion models with global and local ERSF. The results of this research demonstrate that ORM-ERSPD can extract ERSP effectively and robustly and, thus, complete OR successfully.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chen, C.Y., Cheng, K.Y.: A sharpness-dependent filter for recovering sharp features in repaired 3D mesh models. IEEE Trans. Vis. Comput. Graph. 14(1), 200–212 (2008)

    Article  Google Scholar 

  2. Hu, T., Han, Z.Z., Shrivastava A., Zwicker M.: Render4Completion: synthesizing multi-view depth maps for 3D shape completion. In: Proceedings of ICCV 2019 Workshop on Geometry Meets Deep Learning, pp. 4114–4122 (2019)

  3. Foti, S., Koo, B., Dowrick, T., Ramalhinho, J., llam, M., Davidson, B., Stoyanov, D., Clarkson, M.J.: Intraoperative Liver Surface completion with graph convolutional VAE. In: Proceedings of Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis, pp.198–207(2020)

  4. Stutz, D., Geiger, A.: Learning 3D shape completion under weak supervision. Int. J. Comput. Vis. 128(2), 1162–1181 (2020)

    Article  MATH  Google Scholar 

  5. Gregor, R., Sipiran, I., Papaioannou, G., Schreck, T., Andreadis, A.A.T., Mavridis, P.: Towards automated 3D reconstruction of cultural heritage objects. In: Proceedings of Eurographics Workshop on Graphics and Cultural Heritage, pp. 135–144 (2014)

  6. Papaioannou, G., Schreck, T., Andreadis, A., Mavridis, P., Gregor, R., Sipiran, I., Vardis, K.: From reassembly to object completion: a complete systems pipeline. J. Comput. Cult. Herit. (JOCCH) 10(2), 1–22 (2017)

    Article  Google Scholar 

  7. Harary, G., Tal, A.E., Grinspun, E.: Context-based coherent surface completion. ACM Trans. Graph. 33(1), 1–12 (2005)

    Article  MATH  Google Scholar 

  8. Pauly, M., Mitra, N.J., Giesen, J.: Gross M.: Example-based 3D scan completion. In: Proceedings of Eurographics Symposium on Geometry Processing, pp. 23–32 (2005)

  9. Huang, H., Gong, M.L., Cohen-Or, D., Ouyang, Y.B., Tan, F.W., Zhang, H.: Field-guided registration for feature conforming shape composition. ACM Trans. Graph. 31(6), 1–11 (2012)

    Google Scholar 

  10. Li, H., Adams, B., Guibas, L.J., Pauly, M.: Robust single-view geometry and motion reconstruction. ACM Trans. Graph. 28(5), 1–10 (2009)

    Article  Google Scholar 

  11. Siang, C.V., Mohamed, F., Sunar, M.S.B., Selamat, A.B.: Using modified butterfly interpolation scheme for hole-filling in 3D data reconstruction. In: Proceedings of IEEE Conference on Graphics and Media, pp.7–12 (2019)

  12. Enkhbayar, A., Oyundolgor, K., Katsutsugu, M., Kouichi, K.: Complex hole-filling algorithm for 3D models. In: Proceedings of the Computer Graphics International Conference, pp.10:1–10:6 (2017)

  13. Ju, T.: Robust repair of polygonal models. ACM Trans. Graph. 23(3), 888–895 (2004)

    Article  Google Scholar 

  14. Harary, G., Tal, A., Grinspun, E.: Feature preserving surface completion using four points. Comput. Graph. Forum. 33(5), 45–54 (2014)

    Article  Google Scholar 

  15. Bischoff, S., Pavic, D., Kobbelt, L.: Automatic restoration of polygon models. ACM Trans. Graph. 24(4), 1332–1352 (2005)

    Article  Google Scholar 

  16. Xiao, C.X., Zheng, W.T., Miao, Y.W., Zhao, Y.: A unified method for appearance and geometry completion of point set surfaces. Vis. Comput. 23(6), 433–443 (2007)

    Article  Google Scholar 

  17. Mavridis, P., Sipiran, I., Andreadis, A., Papaioannou, G.: Object completion using k-sparse optimization. Comput. Graph. Forum. 34(7), 13–21 (2015)

    Article  Google Scholar 

  18. Sipiran, I., Gregor, R., Schreck, T.: Approximate symmetry detection in partial 3D meshes. Comput. Graph. Forum. 33(7), 131–140 (2014)

    Article  Google Scholar 

  19. Thrun, S., Wegbreit, B.: Shape from symmetry. In: Proceedings of IEEE International Conference of Computer Vision, pp. 1824–1831 (2005)

  20. Wu, Y., Pan, G., Wu, Z.H.: Face authentication based on multiple profiles extracted from range data. In: Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication, pp. 515–522 (2003)

  21. Allen, B., Curless, B., Popovic, Z.: The space of human body shapes: reconstruction and parameterization from range scan. ACM Trans Graph. 22(3), 587–594 (2003)

    Article  Google Scholar 

  22. Guo, X., Xiao, J., Wang, Y.: A survey on algorithms of hole filling in 3D surface reconstruction. Vis. Comput. 34(4), 93–103 (2016)

    Google Scholar 

  23. Sun, J., Ovsjanikov, M., Guibas, L.J.: A concise and provably informative multi-scale signature based on heat diffusion. Comput. Graph. Forum. 28(5), 1383–1392 (2009)

    Article  Google Scholar 

  24. Richard, A., Cherabier, I., Oswald, M.R., Pollefeys, M., Schindler, K.: KAPLAN: A 3D point descriptor for shape completion. arXiv: 2008.00096 (2020)

  25. Dang, Q.V., Mouysset, S., Géraldine, M.: Symmetry-based alignment for 3D model retrieval. In: International Workshop on Content-Based Multimedia Indexing, pp. 1–6 (2014)

  26. Cheng, L., Wu, Y., Chen, S., Zong, W., Yuan, Y., Sun, Y.: A symmetry-based method for lidar point registration. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 11(1), 285–299 (2018)

    Article  Google Scholar 

  27. Avetisyan, A., Dai, A., Nießner, M.: End-to-end CAD model retrieval and 9DoF alignment in 3D scans. In: Proceedings of International Conference on Computer Vision, pp. 2551–2560 (2019)

  28. Reniers, D., Telea, A.: Hierarchical part-type segmentation using voxel-based curve skeletons. Vis. Comput. 24(6), 383–395 (2008)

    Article  Google Scholar 

  29. Teo, C.L., Fermüller, C., Aloimonos, Y.: Detection and segmentation of 2D curved reflection symmetric structures. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1644–1652 (2015)

  30. Nagar, R., Raman, S.: SymmSLIC: symmetry aware superpixel segmentation. In: Proceedings of IEEE International Conference on Computer Vision Workshop IEEE Computer Society, pp. 1764–1773 (2017)

  31. Kermi, A., Andjouh, K., Zidane, F.: Fully automated brain tumour segmentation system in 3D-MRI using symmetry analysis of brain and level sets. IET Image Proc. 12(11), 1964–1971 (2018)

    Article  Google Scholar 

  32. Martinek, M., Grosso, R., Greiner, G.: Interactive partial 3D shape matching with geometric distance optimization. Vis. Comput. 31(2), 223–233 (2015)

    Article  Google Scholar 

  33. Sfikas, K., Pratikakis, I., Theoharis, T.: Ensemble of PANORAMA-based convolutional neural networks for 3D model classification and retrieval. Comput. Graph. 71(4), 208–218 (2017)

    Google Scholar 

  34. Canul-Ku, M., Hasimoto-Beltran, R., Jimenez-Badillo, D., Roman-Rangel, E.F.: Classification of 3D archaeological objects using multi-view curvature structure signatures. IEEE Access. 7, 3298–3313 (2019)

    Article  Google Scholar 

  35. Passalis, G., Theoharis, T., Kakadiaris, I.A.: PTK: a novel depth buffer-based shape descriptor for three-dimensional object retrieval. Vis. Comput. 23(1), 5–14 (2007)

    Article  Google Scholar 

  36. Sfikas, K., Theoharis, T., Pratikakis, I.: Pose normalization of 3D models via reflective symmetry on panoramic views. Vis. Comput. 30(11), 1261–1274 (2014)

    Article  Google Scholar 

  37. Yasseen, Z., Verroust-Blondet, A., Nasri, A.: View selection for sketch-based 3D model retrieval using visual part shape description. Vis. Comput. 33, 565–583 (2017)

    Article  Google Scholar 

  38. Mao, D.H., Hao, Z.H.: A novel sketch-based three-dimensional shape retrieval method using multi-view convolutional neural network. Symmetry 11(5), 703–715 (2019)

    Article  Google Scholar 

  39. Martinet, A., Soler, C., Holzschuch, N., Sillion, F.X.: Accurate detection of symmetries in 3D shapes. ACM Trans. Graph. 25(2), 439–464 (2006)

    Article  Google Scholar 

  40. Kazhdan, M., Funkhouser, T., Rusinkiewicz, S.: Symmetry descriptors and 3d shape matching. In: Proceedings of Eurographics SIGGRAPH 2004 Posters, pp. 115–123 (2004)

  41. Kakarala, R., Kaliamoorthi, P.: Premachandran, V.: Three-dimensional bilateral symmetry plane estimation in the phase domain. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 249–256 (2013)

  42. Lipman, Y., Chen, X.B., Daubechies, I., Funkhouser, T.: Symmetry factored embedding and distance. ACM Trans. Graph. 29(4), 1–12 (2010)

    Article  Google Scholar 

  43. Podolak, J., Shilane, P., Golovinskiy, A., Rusinkiewicz, S., Funkhouser, T.A.: A planar-reflective symmetry transform for 3d shapes. ACM Trans. Graph. 25(3), 549–559 (2006)

    Article  Google Scholar 

  44. Mitra, N.J., Guibas, L.J., Pauly, M.: Partial and approximate symmetry detection for 3D geometry. ACM Trans. Graph. 25(3), 560–568 (2006)

    Article  Google Scholar 

  45. Korman, S., Litman, R., Avidan, S., Bronstein, A.: Probably approximately symmetric: fast rigid symmetry detection with global guarantees. Comput. Graph. Forum. 34(1), 2–13 (2015)

    Article  Google Scholar 

  46. Ji, P., Liu, X.: A fast and efficient 3D reflection symmetry detector based on neural networks. Multimed. Tools Appl. 78(24), 35471–35492 (2019)

    Article  Google Scholar 

  47. Gao, L., Zhang, L.X., Meng, H.Y., Ren, Y.H., Lai, Y.K., Kobbelt, L.: PRS-Net: Planar Reflective Symmetry Detection Net for 3D Models. IEEE Trans. Vis. Comput. Graph. 1(6), 1–12 (2020)

    Google Scholar 

  48. Shi, Y., Huang, J., Zhang, H., Xu, X., Rusinkiewicz, S., Xu, K.: SymmetryNet: learning to predict reflectional and rotational symmetries of 3D shapes from single-view RGB-D images. ACM Trans. Graph. (SIGGRAPH Asia) 1(1), 1–15 (2020)

    Google Scholar 

  49. Speciale, P., Oswald, M.R., Cohen, A., Pollefeys, M.: A symmetry prior for convex variational 3D reconstruction. In: Proceedings of European Conference on Computer Vision, pp. 313–328 (2016)

  50. Dvorák, J., Hruda, L.: Estimating approximate plane of symmetry of 3D triangle meshes. In: Proceedings of Central European Seminar on Computer Graphics (2017)

  51. Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001)

  52. Nealen, A., Igarashi, T., Sorkine, O., Alexa, M.: Laplacian mesh optimization. In: Proceedings of International Conference on Computer graphics and interactive techniques in Australasia and Southeast Asia, pp. 381–389 (2006)

  53. Liepa, P.: Filling holes in meshes. In: Proceedings of Eurographics Symposium on Geometry Processing, pp. 200–206 (2003)

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant No. 51575196) and the Pilot Project of Fujian Province (Grant No. 2020H0015).

Author information

Authors and Affiliations

  1. Fujian Provincial Key Laboratory of Special Energy Manufacturing, Xiamen Key Laboratory of Digital Vision Measurement, Huaqiao University, Xiamen, 361021, China

    Jianwen Xu, Wei Cao, Bin Liu & Kaiyong Jiang

Authors
  1. Jianwen Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kaiyong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kaiyong Jiang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, J., Cao, W., Liu, B. et al. Object restoration based on extrinsic reflective symmetry plane detection. Vis Comput 38, 3627–3642 (2022). https://doi.org/10.1007/s00371-021-02192-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02192-5

Keywords

Search

Navigation