Abstract
Upsampling and inpainting sparse and incomplete point clouds is a challenging task. Existing methods mainly focus on upsampling low-resolution point clouds or inpainting incomplete point clouds. In this paper, we propose a unified framework for upsampling and inpainting point clouds simultaneously. Specifically, we develop a point cloud upsampling and inpainting network called PUI-Net, which consists of the attention convolution unit and non-local feature expansion unit. In the attention convolution unit, the channel attention mechanism is adopted to extract discriminative features of point clouds. With the extracted discriminative features, we employ the non-local feature expansion unit to generate dense feature maps of high-resolution point clouds. Furthermore, we formulate a novel inpainting loss to train the PUI-Net so that the holes in the incomplete clouds can be completed. Quantitative and qualitative comparisons demonstrate that our proposed PUI-Net can yield good performance in terms of the upsampling and inpainting tasks on the incomplete point clouds.
This work was supported by the National Science Fund of China (Grant No. 61876084).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yu, L., Li, X., Fu, C.-W., Cohen-Or, D., Heng, P.-A.: EC-Net: an edge-aware point set consolidation network. In: ECCV, pp. 386–402 (2018)
Yu, L., Li, X., Fu, C.-W., Cohen-Or, D., Heng, P.-A.: Pu-Net: point cloud upsampling network. In: CVPR, pp. 2790–2799 (2018)
Yifan, W., Wu, S., Huang, H., Cohen-Or, D., Sorkine-Hornung, O.: Patch-based progressive 3D point set upsampling. In: CVPR, pp. 5958–5967 (2019)
Li, R., Li, X., Fu, C.-W., Cohen-Or, D., Heng, P.-A.: EC-Net: Pu-gan: a point cloud upsampling adversarial network. In: ICCV, pp. 7203–7212 (2019)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: CVPR, pp. 652–660 (2017)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. Adv. Neural Inf, Process. Syst. 38(5), 146 (2019)
Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., Silva, C.T.: Computing and rendering point set surfaces. IEEE Trans. Visual Comput. Graphics 9(1), 3–15 (2003)
Lipman, Y., Cohen-Or, D., Levin, D., Tal-Ezer, H.: Parameterization-free projection for geometry reconstruction. ACM Trans. Graphics 26(3), 22 (2007)
Huang, H., Wu, S., Gong, M., Cohen-Or, D., Ascher, U., Zhang, H.R.: Edge-aware point set resampling. ACM Trans. Graphics 32(1), 9 (2013)
Wu, S., Huang, H., Gong, M., Zwicker, M., Cohen-Or, D.: Deep points consolidation. ACM Trans. Graphics 34(6), 176 (2015)
Sahay, P., Rajagopalan, A.N.: Geometric in painting of 3D structures. In: CVPR Workshops, pp. 1–7 (2015)
Dinesh, C., Bajić, I.V., Cheung, G.: Exemplar-based framework for 3D point cloud hole filling. In: 2017 IEEE Visual Communications and Image Processing, pp. 1–4 (2017)
Quinsat, Y., et al.: Filling holes in digitized point cloud using a morphing-based approach to preserve volume characteristics. Int. J. Adv. Manuf. Technol. 81(1–4), 411–421 (2015)
Wang, J., Oliveira, M.M.: Filling holes on locally smooth surfaces reconstructed from point clouds. Image Vis. Comput. 25(1), 103–113 (2007)
Muraki, Y., Nishio, K., Kanaya, T., Kobori, K,-I.: An automatic hole filling method of point cloud for 3D scanning. Václav Skala-UNION Agency (2017)
Lozes, F., Elmoataz, A., Lézoray, O.: PDE-based graph signal processing for 3-D color point clouds: opportunities for cultural heritage. IEEE Signal Process. Mag. 32(4), 103–111 (2015)
Fu, Z., Hu, W., Guo, Z.: Point cloud in painting on graphs from non-local self-similarity. In: 2018 25th IEEE International Conference on Image Processing, pp. 2137–2141 (2018)
Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: CVPR, pp. 7794–7803 (2018)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: ECCV, pp. 286–301 (2018)
Bogo, F., Romero, J., Loper, M., Black, M.J.: FAUST: dataset and evaluation for 3D mesh registration. In: CVPR, pp. 3794–3801 (2014)
Mao, X., Shen, C., Yang, Y.-B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in Neural Information Processing Systems, pp. 2802–2810 (2016)
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image in painting with contextual attention. In: CVPR, pp. 5505–5514 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, Y., Xie, J., Qian, J., Yang, J. (2020). PUI-Net: A Point Cloud Upsampling and Inpainting Network. In: Peng, Y., et al. Pattern Recognition and Computer Vision. PRCV 2020. Lecture Notes in Computer Science(), vol 12305. Springer, Cham. https://doi.org/10.1007/978-3-030-60633-6_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-60633-6_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60632-9
Online ISBN: 978-3-030-60633-6
eBook Packages: Computer ScienceComputer Science (R0)