Skip to main content
SpringerLink
Log in

PLGP: point cloud inpainting by patch-based local geometric propagating

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The booming of LiDAR technologies has made the point cloud become a prevailing data format for 3D object representation. However, point cloud usually exhibits holes of data loss mainly due to occurrence of noise, occlusion or the surface material of the object, which is a serious problem affects the target expression of point cloud. Point cloud inpainting is the key solution for holes problem. In this paper, we propose a patch-based local geometric propagating (PLGP) method to automatically fill the lost data obtained by three-dimensional scanning. Different from typical methods transforming the point cloud into range image to conduct the hole-detection or filling the missing region with a whole best match, this work tends to detect the hole directly in 3D space and inpaint it by iteratively searching for the context with local similarity and making it propagate appropriately along the occlusion’s local geometric structure. The experimental results with comparisons demonstrate its competitive effectiveness with a F-score as high as 0.89 and a 23.45 dB average gain in GPSNR with consumed time reduced by up to 60%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Guo, X., Xiao, J., Wang, Y.: A survey on algorithms of hole filling in 3D surface reconstruction. Vis. Comput. 34, 93–103 (2018)

    Article  Google Scholar 

  2. Li, X., Shen, H., Zhang, L., Zhang, H., Yuan, Q., Yang, G.: Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning. IEEE Trans. Geosci. Remote Sens. 52(11), 7086–7098 (2014)

    Article  Google Scholar 

  3. Dinesh, C., Bajic, I.V., Cheung, G.: Adaptive nonrigid inpainting of three-dimensional point cloud geometry. IEEE Signal Process. Lett. 25(6), 878–882 (2018)

    Article  Google Scholar 

  4. Doria, D., Radke, R. J.: Filling large holes in LiDAR data by inpainting depth gradients. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 65–72 (2012)

  5. Pauly, M., Mitra, N.J., Giesen, J., Gross, M.H., Guibas, L.J.: Example-based 3D scan completion. In: Symposium on Geometry Processing, pp. 23–32 (2005)

  6. Sahay, P., Rajagopalan, A.N.: Geometric inpainting of 3D structures. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–7 (2015)

  7. Wang, J., Oliveira, M.M.: Filling holes on locally smooth surfaces reconstructed from point clouds. Image Vis. Comput. 25(1), 103–113 (2007)

    Article  Google Scholar 

  8. Quinsat, Y., Lartigue, C.: Filling holes in digitized point cloud using a morphing-based approach to preserve volume characteristics. Int. J. Adv. Manuf. Technol. 81(1–4), 411–421 (2015)

    Article  Google Scholar 

  9. Xiao, C., Zheng, W., Miao, Y., Zhao, Y., Peng, Q.: A unified method for appearance and geometry completion of point set surfaces. Vis. Comput. 23, 433–443 (2007)

    Article  Google Scholar 

  10. Hu, W., Fu, Z., Guo, Z.: Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting. IEEE Trans. Image Process. 28(8), 4087–4100 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  11. Fu, Z., Hu, W., Guo, Z.: Point cloud inpainting on graphs from non-local self-similarity. In: IEEE International Conference on Image Processing (ICIP) (2018)

  12. Lai, P.J., Huang, Y.L., Chien, S.Y.: Surface-based background completion in 3D scene. In: IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 1218–1222 (2016)

  13. Fu, Z., Hu, W., Guo, Z.: 3d Dynamic Point Cloud Inpainting Via Temporal Consistency On Graphs. In: IEEE International Conference on Multimedia and Expo (ICME) (2020)

  14. Fu, Z., Hu, W.: Dynamic point cloud inpainting via spatial-temporal graph learning. IEEE Trans. Multimedia. 23, 3022–3034 (2021)

    Article  Google Scholar 

  15. Sun, H., Liu, X., Deng, Q., Jiang, W., Ha, Y.: Efficient FPGA implementation of K-nearest-neighbor search algorithm for 3D LIDAR localization and mapping in smart vehicles. IEEE Trans. Circuits Syst. II Exp. Briefs. 67(9), 1644–1648 (2020)

    Google Scholar 

  16. Tachella, J., Altmann, Y., Mclaughlin, S., Tourneret, J.Y.: Real-Time 3D Color Imaging with Single-Photon Lidar Data. In: IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), pp. 206–210 (2019)

  17. Aranjuelo, N., Engels, G., Unzueta, L., Arganda-Carreras, I., Otaegui, O.: Robust 3D Object Detection from LiDAR Point Cloud Data with Spatial Information Aggregation. In: International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO), pp. 813–823 (2020)

  18. Cai, Z., Wang, C., Wen, C., Li, J.: Occluded boundary detection for small-footprint groundborne LIDAR point cloud guided by last echo. IEEE Geosci. Remote Sens. Lett. 12(11), 2272–2276 (2015)

    Article  Google Scholar 

  19. Goyal, P., Challa, J.S., Kumar, D., et al.: Grid-R-tree: a data structure for efficient neighborhood and nearest neighbor queries in data mining. Int. J. Data Sci. Anal. 10(10), 25–47 (2020)

    Article  Google Scholar 

  20. Wexler, Y., Shechtman, E., Irani, M.: Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 463–476 (2007)

    Article  Google Scholar 

  21. Fedorov, V., Facciolo, G., Arias, P.: Variational framework for non-local inpainting. Image Process. Line. 5, 362–386 (2015)

    Article  MathSciNet  Google Scholar 

  22. Levoy, M., Gerth, J., Curless, B., Pull, K.: The Stanford 3D scanning repository. [Online] https://graphics.stanford.edu/data/3Dscanrep/.

  23. https://jpeg.org/plenodb/.

  24. https://npm3d.fr/paris-lille-3d.

  25. http://vcl.iti.gr/dataset/reconstruction/.

  26. Tian, D., Ochimizu, H., Feng, C., Cohen R., Vetro, A.: Geometric distortion metrics for point cloud compression. In IEEE International Conference on Image Processing (ICIP), pp. 3460–3464 (2017)

Download references

Funding

This study was funded by National Key R&D Program of China (Grant 2019YFB1802904). National Key R&D Program of China, 2019YFB1802904, Chuanchuan Yang.

Author information

Authors and Affiliations

  1. State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronics, Peking University, Beijing, 100871, China

    Yan Huang, Chuanchuan Yang & Zhangyuan Chen

  2. Department of Electronics, Peking University, Beijing, 100871, China

    Yu Shi

  3. Sense Future Technologies Co., Ltd, Beijing, 100025, China

    Hao Chen & Weizhen Yan

Authors
  1. Yan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chuanchuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weizhen Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhangyuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuanchuan Yang.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Y., Yang, C., Shi, Y. et al. PLGP: point cloud inpainting by patch-based local geometric propagating. Vis Comput 39, 723–732 (2023). https://doi.org/10.1007/s00371-021-02370-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02370-5

Keywords

Search

Navigation