Abstract
The quality of a 3D model depends on the object digitization process, which is usually characterized by a tradeoff between volume resolution and scanning speed, i.e., higher resolution scans require longer scanning times. Aiming to improve the quality of lower resolution 3D models, this paper proposes a novel approach to 3D model reconstruction from an initially coarse point cloud (PC) representation of an object. The main contribution of this paper is the introduction of a novel periodic activation function, named Wave-shaping Neural Activation (WNA), in the context of implicit neural representations (INRs). The use of the WNA function in a multilayer perceptron (MLP) can enhance the learning of continuous functions describing object surfaces given their coarse 3D representation. Then, the trained MLP can be regarded as a continuous implicit representation of the 3D representation of the object, and it can be used to reconstruct the originally coarse 3D model with higher detail. The proposed methodology is experimentally evaluated by two case studies in different application domains: a) reconstruction of complex human tissue structures for medical applications; b) reconstruction of ancient artifacts for cultural heritage applications. The experimental evaluation, which includes comparisons with state-of-the-art approaches, verifies the effectiveness and improved performance of the WNA-based INR for 3D object reconstruction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.: Learning representations and generative models for 3D point clouds. In: International Conference on Machine Learning. PMLR, pp. 40–49 (2018)
Bagautdinov, T., Wu, C., Saragih, J., Fua, P., Sheikh, Y.: Modeling facial geometry using compositional VAEs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3877–3886 (2018)
Balashova, E., Wang, J., Singh, V., Georgescu, B., Teixeira, B., Kapoor, A.: 3D organ shape reconstruction from Topogram images. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 347–359. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_26
Ballarin, M., Balletti, C., Vernier, P.: Replicas in cultural heritage: 3D printing and the museum experience. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 42, 55–62 (2018)
Chabra, R., et al.: Deep local shapes: learning local SDF priors for detailed 3D reconstruction. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12374, pp. 608–625. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58526-6_36
Chen, X., et al.: A fast reconstruction method of the dense point-cloud model for cultural heritage artifacts based on compressed sensing and sparse auto-encoder. Opt. Quant. Electron. 51, 1–16 (2019)
Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5939–5948 (2019)
Chibane, J., et al.: Neural unsigned distance fields for implicit function learning. In: Advances in Neural Information Processing Systems, vol. 33, pp. 21638–21652 (2020)
Clark, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26, 1045–1057 (2013)
Dai, A., Ruizhongtai Qi, C., Nießner, M.: Shape completion using 3D-encoder-predictor CNNs and shape synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5868–5877 (2017)
Deng, Z., Yao, Y., Deng, B., Zhang, J.: A robust loss for point cloud registration. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6138–6147 (2021)
Garcia Carrizosa, H., Sheehy, K., Rix, J., Seale, J., Hayhoe, S.: Designing technologies for museums: accessibility and participation issues. J. Enabl. Technol. 14, 31–39 (2020)
Gómez-Rodrguez, J.J., Lamarca, J., Morlana, J., Tardós, J.D., Montiel, J.M.: SD-DefSLAM: Semi-direct monocular SLAM for deformable and intracorporeal scenes. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp 5170–5177. IEEE (2021)
Gropp, A., Yariv, L., Haim, N., Atzmon, M., Lipman, Y.: Implicit geometric regularization for learning shapes. arXiv preprint arXiv:200210099 (2020)
Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: A papier-mâché approach to learning 3D surface generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 216–224 (2018)
Hu, M., Penney, G., Edwards, P., Figl, M., Hawkes, D.J.: 3D reconstruction of internal organ surfaces for minimal invasive surgery. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007. LNCS, vol. 4791, pp. 68–77. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75757-3_9
Huovilainen, A.: Non-linear digital implementation of the Moog ladder filter. In: Proceedings of the International Conference on Digital Audio Effects (DAFx-04), pp 61–64 (2004)
Kalozoumis, P.G., Marino, M., Carniel, E.L., Iakovidis, D.K.: Towards the development of a digital twin for endoscopic medical device testing. In: Hassanien, A.E., Darwish, A., Snasel, V. (eds.) Digital Twins for Digital Transformation: Innovation in Industry. Studies in Systems, Decision and Control, vol. 423, pp. 113–145. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96802-1_7
Kaneda, A., Nakagawa, T., Tamura, K., Noshita, K., Nakao, H.: A proposal of a new automated method for SfM/MVS 3D reconstruction through comparisons of 3D data by SfM/MVS and handheld laser scanners. PLoS ONE 17, e0270660 (2022)
Kazhdan, M., Hoppe, H.: Screened Poisson surface reconstruction. ACM Trans. Graph. (ToG) 32, 1–13 (2013)
Lamarca, J., Parashar, S., Bartoli, A., Montiel, J.: DefSLAM: tracking and mapping of deforming scenes from monocular sequences. IEEE Trans. Rob. 37, 291–303 (2020)
Lazzarini, V., Timoney, J.: New perspectives on distortion synthesis for virtual Analog oscillators. Comput. Music. J. 34, 28–40 (2010)
Levina, E., Bickel, P.: The earth mover’s distance is the mallows distance: some insights from statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, pp 251–256. IEEE (2001)
Lewiner, T., Lopes, H., Vieira, A.W., Tavares, G.: Efficient implementation of marching cubes’ cases with topological guarantees. J. Graph. Tools 8, 1–15 (2003)
Ma, B., Han, Z., Liu, Y.-S., Zwicker, M.: Neural-pull: learning signed distance functions from point clouds by learning to pull space onto surfaces. arXiv preprint arXiv:201113495 (2020)
Makantasis, K., Doulamis, A., Doulamis, N., Ioannides, M.: In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction. Multimed. Tools Appl. 75, 3593–3629 (2016)
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4460–4470 (2019)
Osher, S., Fedkiw, R.: Signed distance functions. In: Level Set Methods and Dynamic Implicit Surfaces, pp 17–22. Springer (2003)
Pakarinen, J., Yeh, D.T.: A review of digital techniques for modeling vacuum-tube guitar amplifiers. Comput. Music. J. 33, 85–100 (2009)
Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 165–174 (2019)
Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., Geiger, A.: Convolutional occupancy networks. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12348, pp. 523–540. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58580-8_31
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 652–660 (2017)
Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), pp 519–528. IEEE (2006)
Sengupta, A., Bartoli, A.: Colonoscopic 3D reconstruction by tubular non-rigid structure-from-motion. Int. J. Comput. Assist. Radiol. Surg. 16, 1237–1241 (2021)
Ben-Shabat, Y., Koneputugodage, C.H., Gould, S.: DiGS: divergence guided shape implicit neural representation for unoriented point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 19323–19332 (2022)
Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. In: Advances in Neural Information Processing Systems, vol. 33, pp. 7462–7473 (2020)
Vaz, R., Freitas, D., Coelho, A.: Blind and visually impaired visitors’ experiences in museums: increasing accessibility through assistive technologies. Int. J. Inclusive Mus. 13, 57 (2020)
Wang, Z., et al.: A Deep Learning based Fast Signed Distance Map Generation. arXiv preprint arXiv:200512662 (2020)
Wilson, P.F., Stott, J., Warnett, J.M., Attridge, A., Smith, M.P., Williams, M.A.: Evaluation of touchable 3D-printed replicas in museums. Curator Mus. J. 60, 445–465 (2017)
Wu, Z., et al.: 3D shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1912–1920 (2015)
Xu, Z., Xu, C., Hu, J., Meng, Z.: Robust resistance to noise and outliers: screened Poisson surface reconstruction using adaptive kernel density estimation. Comput. Graph. 97, 19–27 (2021)
Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M.: PCN: point completion network. In: 2018 International Conference on 3D Vision (3DV), pp 728–737. IEEE (2018)
Zhang, S., Zhao, L., Huang, S., Ma, R., Hu, B., Hao, Q.: 3D reconstruction of deformable colon structures based on preoperative model and deep neural network. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp 1875–1881. IEEE (2021)
Zhou, L., Sun, G., Li, Y., Li, W., Su, Z.: Point cloud denoising review: from classical to deep learning-based approaches. Graph. Models 121, 101140 (2022)
Acknowledgement
We acknowledge support of this work by the project “Smart Tourist” (MIS 5047243) which is implemented under the Action “Reinforcement of the Research and Innovation Infrastructure”, funded by the Operational Programme "Competitiveness, Entrepreneurship and Innovation" (NSRF 2014–2020) and co-financed by Greece and the European Union (European Regional Development Fund).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Triantafyllou, G., Dimas, G., Kalozoumis, P.G., Iakovidis, D.K. (2023). Wave-Shaping Neural Activation for Improved 3D Model Reconstruction from Sparse Point Clouds. In: Blanc-Talon, J., Delmas, P., Philips, W., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2023. Lecture Notes in Computer Science, vol 14124. Springer, Cham. https://doi.org/10.1007/978-3-031-45382-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-45382-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45381-6
Online ISBN: 978-3-031-45382-3
eBook Packages: Computer ScienceComputer Science (R0)