Skip to main content
SpringerLink
Log in

Data-Driven Restoration of Digital Archaeological Pottery with Point Cloud Analysis

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The Josefina Ramos de Cox museum in Lima, Peru, decided to digitize hundreds of archaeological pieces from pre-Colombian cultures to support further research and create virtual educational environments. However, the 3D scanning procedure led to imperfections in the objects’ surface, mainly due to the difficulty of manipulating the fragile objects during the acquisition. The problem was that many of the scanned artifacts do not contain the base because the contact surface during acquisition was not visible to the scanner. This paper proposes a method to repair the digital objects’ surface using a data-driven approach. We design and train a point cloud neural network that learns to synthesize the missing geometry in an end-to-end manner. Our model consists of a novel architecture and training protocol that addresses the problem of point cloud completion. We propose an end-to-end neural network architecture that focuses on calculating the missing geometry and merging the known input and the predicted point cloud. Our method is composed of two neural networks: the missing part prediction network and the merging-refinement network. The first module focuses on extracting information from the incomplete input to infer the missing geometry. The second module merges both point clouds and improves the distribution of the points. Our approach is effective in repairing pottery objects with large imperfections during the scanning. Besides, our experiments on ShapeNet and Completion3D datasets show that our method is effective in a general setting for shape completion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://github.com/ivansipiran/Data-driven-cultural-heritage.

References

  • Achlioptas, P., Diamanti, O., Mitliagkas, I., & Guibas, L. J. (2018). Learning representations and generative models for 3D point clouds. In J. G. Dy, & A. Krause (Eds.), Proceedings of the 35th international conference on machine learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10–15, 2018, PMLR, Proceedings of machine learning research (Vol. 80, pp. 40–49).

  • Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., & Xiao, J. (2015). Shapenet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012

  • Charles R. Q., Su, H., Kaichun, M., & Guibas L. J. (2017). PointNet: Deep learning on point sets for 3D classification and segmentation. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 77–85).

  • Dai, A., Qi, C. R., & Nießner, M. (2017). Shape completion using 3D-encoder-predictor CNNs and shape synthesis. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 6545–6554).

  • Fan, H., Su, H., & Guibas, L. (2017) A point set generation network for 3D object reconstruction from a single image. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2463–2471).

  • Gropp, A., Yariv, L., Haim, N., Atzmon, M., & Lipman, Y. (2020). Implicit geometric regularization for learning shapes. In Proceedings of the 37th international conference on machine learning, ICML 2020, 13–18 July 2020, Virtual Event, PMLR, proceedings of machine learning research (Vol. 119, pp. 3789–3799).

  • Groueix, T., Fisher, M., Kim, V. G., Russell, B. C., & Aubry, M. (2018). A Papier-Mache approach to learning 3d surface generation. In 2018 IEEE/CVF conference on computer vision and pattern recognition (pp. 216–224).

  • Han, X., Li, Z., Huang, H., Kalogerakis, E., & Yu, Y. (2017). High-resolution shape completion using deep neural networks for global structure and local geometry inference. In 2017 IEEE international conference on computer vision (ICCV) (pp. 85–93).

  • Hanocka, R., Hertz, A., Fish, N., Giryes, R., Fleishman, S., & Cohen-Or, D. (2019) MeshCNN: A network with an edge. ACM Transactions on Graphics, 38(4).

  • Harary, G., Tal, A., & Grinspun, E. (2014a) Context-based coherent surface completion. ACM Transactions on Graphics, 33(1), 5:1–5:12.

  • Harary, G., Tal, A., & Grinspun, E. (2014b). Feature-preserving surface completion using four points. Computer Graphics Forum, 33(5), 45–54.

  • Hermoza, R., & Sipiran, I. (2018). 3D reconstruction of incomplete archaeological objects using a generative adversarial network. In Proceedings of computer graphics international 2018, CGI 2018 (pp. 5–11). Association for Computing Machinery.

  • Hu, T., Han, Z., Shrivastava, A., & Zwicker, M. (2019). Render4completion: Synthesizing multi-view depth maps for 3D shape completion. In 2019 IEEE/CVF international conference on computer vision workshop (ICCVW) (pp. 4114–4122).

  • Hu, T., Han, Z., & Zwicker, M. (2020). 3D shape completion with multi-view consistent inference. In The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, The tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020 (pp. 10997–11004). AAAI Press.

  • Huang, H., Gong, M., Cohen-Or, D., Ouyang, Y., Tan, F., & Zhang, H. (2012). Field-guided registration for feature-conforming shape composition. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2012), 31, 171:1–171:11.

  • Huang, Z., Yu, Y., Xu, J., Ni, F., & Le, X. (2020). PF-Net: Point fractal network for 3D point cloud completion. In 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 7659–7667).

  • Jiang, W., Xu, K., Cheng, Z. Q., & Zhang, H. (2013). Skeleton-based intrinsic symmetry detection on point clouds. Graphical Models, 75(4), 177–188.

    Article  Google Scholar 

  • Koutsoudis, A., Pavlidis, G., Liami, V., Tsiafakis, D., & Chamzas, C. (2010). 3D pottery content-based retrieval based on pose normalisation and segmentation. Journal of Cultural Heritage, 11(3), 329–338.

    Article  Google Scholar 

  • Li, E., Zhang, X., & Chen, Y. (2014). Symmetry based Chinese ancient architecture reconstruction from incomplete point cloud. In 2014 5th International conference on digital home (pp. 157–161).

  • Liu, F., Tran, L., & Liu, X. (2019). 3D face modeling from diverse raw scan data. In 2019 IEEE/CVF international conference on computer vision (ICCV) (pp. 9407–9417).

  • Liu, M., Sheng, L., Yang, S., Shao, J., & Hu, S. (2020). Morphing and sampling network for dense point cloud completion. In The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, the tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020 (pp. 11596–11603). AAAI Press.

  • Maturana, D., & Scherer, S. (2015). VoxNet: A 3D convolutional neural network for real-time object recognition. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 922–928).

  • Mavridis, P., Sipiran, I., Andreadis, A., & Papaioannou, G. (2015). Object completion using k-sparse optimization. Computer Graphics Forum, 34(7), 13–21.

    Article  Google Scholar 

  • Pan, L., Chen, X., Cai, Z., Zhang, J., Zhao, H., Yi, S., & Liu, Z. (2021). Variational relational point completion network. In 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 8520–8529). IEEE Computer Society.

  • Papaioannou, G., Schreck, T., Andreadis, A., Mavridis, P., Gregor, R., Sipiran, I., & Vardis, K. (2017). From reassembly to object completion: A complete systems pipeline. Journal on Computing and Cultural Heritage (JOCCH), 10(2).

  • Pauly, M., Mitra, N. J., Giesen, J., Gross, M., & Guibas, L. J. (2005). Example-based 3D scan completion. In Proceedings of the third eurographics symposium on geometry processing, SGP’05. Eurographics Association.

  • Pratikakis, I., Savelonas, M., Mavridis, P., Papaioannou, G., Sfikas, K., Arnaoutoglou, F., & Rieke-Zapp, D. (2018). Predictive digitisation of cultural heritage objects. Multimedia Tools and Applications, 77(10), 12991–13021.

    Article  Google Scholar 

  • Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 30, pp. 5099–5108). Curran Associates, Inc.

  • Qi, C. R., Chen, X., Litany, O., & Guibas, L. J. (2020). Imvotenet: Boosting 3D object detection in point clouds with image votes. In 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 4403–4412).

  • Sipiran, I. (2017). Analysis of partial axial symmetry on 3D surfaces and its application in the restoration of cultural heritage objects. In 2017 IEEE international conference on computer vision workshops (ICCVW) (pp. 2925–2933).

  • Sipiran, I. (2018). Completion of cultural heritage objects with rotational symmetry. In Proceedings of the 11th eurographics workshop on 3D object retrieval, eurographics association, Goslar, DEU, 3DOR’18 (pp. 87–93).

  • Sipiran, I., Gregor, R., & Schreck, T. (2014). Approximate symmetry detection in partial 3D meshes. Computer Graphics Forum, 33(7), 131–140.

    Article  Google Scholar 

  • Son, K., Almeida, E. B., & Cooper, D. B. (2013). Axially symmetric 3D pots configuration system using axis of symmetry and break curve. In 2013 IEEE conference on computer vision and pattern recognition (pp. 257–264).

  • Su, H., Maji, S., Kalogerakis, E., & Learned-Miller, E. (2015). Multi-view convolutional neural networks for 3D shape recognition. In 2015 IEEE international conference on computer vision (ICCV) (pp. 945–953).

  • Sun, Y., Wang, Y., Liu, Z., Siegel, J. E., & Sarma, S. E. (2020). Pointgrow: Autoregressively learned point cloud generation with self-attention. In 2020 IEEE winter conference on applications of computer vision (WACV) (pp 61–70).

  • Tchapmi, L. P., Kosaraju, V., Rezatofighi, H., Reid, I., & Savarese, S. (2019). Topnet: Structural point cloud decoder. In 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 383–392).

  • Thrun, S., & Wegbreit, B. (2005). Shape from symmetry. In Tenth IEEE international conference on computer vision, 2005. ICCV 2005 (Vol. 2, pp. 1824–1831).

  • Wang, X., Ang, M. H., & Lee, G. H. (2020). Cascaded refinement network for point cloud completion. In 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 787–796).

  • Wen, X., Li, T., Han, Z., & Liu, Y. S. (2020). Point cloud completion by skip-attention network with hierarchical folding. In 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 1936–1945).

  • Wen, X., Han, Z., Cao, Y. P., Wan, P., Zheng, W., & Liu, Y. S. (2021a). Cycle4completion: Unpaired point cloud completion using cycle transformation with missing region coding. In 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 13075–13084).

  • Wen, X., Xiang, P., Han, Z., Cao, Y. P., Wan, P., Zheng, W., & Liu, Y. S. (2021b). PMP-Net: Point cloud completion by learning multi-step point moving paths. In 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 7439–7448).

  • Xiang, P., Wen, X., Liu, Y. S., Cao, Y. P., Wan, P., Zheng, W., & Han, Z. (2021). Snowflakenet: Point cloud completion by snowflake point deconvolution with skip-transformer. In 2021 IEEE/CVF international conference on computer vision (ICCV) (pp. 5479–5489).

  • Xie, H., Yao, H., Zhou, S., Mao, J., Zhang, S., & Sun, W. (2020). Grnet: Gridding residual network for dense point cloud completion. In A. Vedaldi, H. Bischof, T. Brox, & J. M. Frahm (Eds.), Computer vision—ECCV 2020 (pp. 365–381). Springer.

  • Xu, K., Zhang, H., Tagliasacchi, A., Liu, L., Li, G., Meng, M., & Xiong, Y. (2009). Partial intrinsic reflectional symmetry of 3D shapes. ACM Transactions on Graphics, 28(5), 138:1–138:10.

  • Yang, Y., Feng, C., Shen, Y., & Tian, D. (2018). Foldingnet: Point cloud auto-encoder via deep grid deformation. In 2018 IEEE/CVF conference on computer vision and pattern recognition (pp. 206–215).

  • Yi, L., Kim, V. G., Ceylan, D., Shen, I. C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A., & Guibas, L. (2016). A scalable active framework for region annotation in 3D shape collections. ACM Transactions on Graphics, 35(6).

  • Yu, X., Rao, Y., Wang, Z., Liu, Z., Lu, J., & Zhou, J. (2021). Pointr: Diverse point cloud completion with geometry-aware transformers. In 2021 IEEE/CVF international conference on computer vision (ICCV) (pp. 12478–12487).

  • Yuan, W., Khot, T., Held, D., Mertz, C., & Hebert, M. (2018). PCN: Point completion network. In 2018 International conference on 3D vision (3DV) (pp. 728–737).

  • Zheng, Q., Sharf, A., Wan, G., Li, Y., Mitra, N. J., Cohen-Or, D., & Chen, B. (2010). Non-local scan consolidation for 3D urban scenes. ACM Transactions on Graphics, 29(4), 94:1–94:9.

  • Zhirong, W., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015). 3D shapenets: A deep representation for volumetric shapes. In 2015 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1912–1920).

Download references

Acknowledgements

The work of Ivan Sipiran has been partially supported by ANID Chile—Research Initiation Program—Grant N\(^{\circ }\) 11220211. The work of Cristian Lopez has been supported by Proyecto de Mejoramiento y Ampliacion de los Servicios del Sistema Nacional de Ciencia Tecnologia e Innovacion Tecnologica(Banco Mundial, Concytec-Peru), Nr. Grant 062-2018-FONDECYT-BM-IADT-AV. Thanks to the Josefina Ramos de Cox Museum for the continuous support during the realization of this project.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Chile, Santiago, Chile

    Ivan Sipiran

  2. Universidad Nacional de San Agustin, Arequipa, Peru

    Alexis Mendoza & Alexander Apaza

  3. Universidad de Ingenieria y Tecnologia, UTEC, Barranco, Peru

    Cristian Lopez

Authors
  1. Ivan Sipiran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexis Mendoza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Apaza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cristian Lopez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ivan Sipiran.

Additional information

Communicated by Rei Kawakami.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sipiran, I., Mendoza, A., Apaza, A. et al. Data-Driven Restoration of Digital Archaeological Pottery with Point Cloud Analysis. Int J Comput Vis 130, 2149–2165 (2022). https://doi.org/10.1007/s11263-022-01637-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-022-01637-1

Keywords

Search

Navigation