Skip to main content
SpringerLink
Log in

Point Cloud Segmentation with Guided Sampling and Continuous Interpolation

  • Conference paper
  • First Online:
Computational Visual Media (CVM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14592))

Included in the following conference series:

  • 170 Accesses

Abstract

Sampling and interpolation are pivotal in the design of 3D neural networks. Presently, farthest point sampling and \(k\)-NN interpolation are the predominant techniques. Nonetheless, the former can lead to information loss in feature-rich regions, while the latter might introduce noticeable discontinuities, compromising neural network performance. In this research, we address information loss with a novel method, DistrFPS, that considers the input information distribution during the farthest point sampling. Leveraging DistrFPS, we introduce a guided sampling module to retain crucial information for subsequent network layers. We also propose a continuous interpolation module grounded in barycentric interpolation to ensure spatial coherent feature propagation to higher resolution network layers. Our approach’s efficacy in preserving information is demonstrated empirically through signal reconstruction in both 2D and 3D realms. Comprehensive experiments on S3DIS, ScanNet, and ShapeNetPart affirm the advantages of our technique for point-based networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Supplementary material at: https://doi.org/10.5281/zenodo.10494633.

  2. 2.

    https://github.com/yanx27/PointASNL.

References

  1. Armeni, I., et al.: 3D semantic parsing of large-scale indoor spaces. In: CVPR (2016)

    Google Scholar 

  2. Cao, T.T., Nanjappa, A., Gao, M., Tan, T.S.: A GPU accelerated algorithm for 3D Delaunay triangulation. In: I3D (2014)

    Google Scholar 

  3. Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3D object detection network for autonomous driving. In: CVPR (2017)

    Google Scholar 

  4. Chiang, H.Y., Lin, Y.L., Liu, Y.C., Hsu, W.H.: A unified point-based framework for 3D segmentation. In: 3DV (2019)

    Google Scholar 

  5. Choy, C., Gwak, J., Savarese, S.: 4D spatio-temporal convnets: Minkowski convolutional neural networks. In: CVPR (2019)

    Google Scholar 

  6. Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: Scannet: richly-annotated 3D reconstructions of indoor scenes. In: CVPR (2017)

    Google Scholar 

  7. Dovrat, O., Lang, I., Avidan, S.: Learning to sample. In: CVPR (2019)

    Google Scholar 

  8. Eldar, Y., Lindenbaum, M., Porat, M., Zeevi, Y.Y.: The farthest point strategy for progressive image sampling. TIP 6(9), 1305–1315 (1997)

    Google Scholar 

  9. Fan, S., Dong, Q., Zhu, F., Lv, Y., Ye, P., Wang, F.Y.: SCF-net: learning spatial contextual features for large-scale point cloud segmentation. In: CVPR (2021)

    Google Scholar 

  10. Gouraud, H.: Continuous shading of curved surfaces. TC (1971)

    Google Scholar 

  11. Graham, B., Engelcke, M., Van Der Maaten, L.: 3D semantic segmentation with submanifold sparse convolutional networks. In: CVPR (2018)

    Google Scholar 

  12. Guibas, L., Stolfi, J.: Primitives for the manipulation of general subdivisions and the computation of Voronoi. TOG 4(2), 74–123 (1985)

    Article  Google Scholar 

  13. Guo, M.H., Cai, J.X., Liu, Z.N., Mu, T.J., Martin, R.R., Hu, S.M.: PCT: point cloud transformer. CVM 7, 187–199 (2021)

    Google Scholar 

  14. Hu, Q., et al.: RandLA-net: efficient semantic segmentation of large-scale point clouds. In: CVPR (2020)

    Google Scholar 

  15. Huang, Q., Wang, W., Neumann, U.: Recurrent slice networks for 3D segmentation of point clouds. In: CVPR (2018)

    Google Scholar 

  16. Kanezaki, A., Matsushita, Y., Nishida, Y.: Rotationnet: joint object categorization and pose estimation using multiviews from unsupervised viewpoints. In: CVPR (2018)

    Google Scholar 

  17. Lai, X., et al.: Stratified transformer for 3D point cloud segmentation. In: CVPR (2022)

    Google Scholar 

  18. Landrieu, L., Simonovsky, M.: Large-scale point cloud semantic segmentation with superpoint graphs. In: CVPR (2018)

    Google Scholar 

  19. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: CVPR (2019)

    Google Scholar 

  20. Lang, I., Manor, A., Avidan, S.: Samplenet: differentiable point cloud sampling. In: CVPR (2020)

    Google Scholar 

  21. Li, B.: 3D fully convolutional network for vehicle detection in point cloud. In: IROS (2017)

    Google Scholar 

  22. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on x-transformed points. In: NeurIPS (2018)

    Google Scholar 

  23. Lin, Y., Chen, L., Huang, H., Ma, C., Han, X., Cui, S.: Task-aware sampling layer for point-wise analysis. TVCG (2022)

    Google Scholar 

  24. Liu, Y., Fan, B., Xiang, S., Pan, C.: Relation-shape convolutional neural network for point cloud analysis. In: CVPR (2019)

    Google Scholar 

  25. Ma, X., Qin, C., You, H., Ran, H., Fu, Y.: Rethinking network design and local geometry in point cloud: a simple residual MLP framework. In: ICLR (2022)

    Google Scholar 

  26. Maturana, D., Scherer, S.: VoxNet: a 3D convolutional neural network for real-time object recognition. In: IROS (2015)

    Google Scholar 

  27. Nekrasov, A., Schult, J., Litany, O., Leibe, B., Engelmann, F.: Mix3D: Out-of-Context Data Augmentation for 3D Scenes (2021)

    Google Scholar 

  28. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS (2019)

    Google Scholar 

  29. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: CVPR (2017)

    Google Scholar 

  30. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: NeurIPS (2017)

    Google Scholar 

  31. Qian, G., Hammoud, H., Li, G., Thabet, A., Ghanem, B.: ASSANet: an anisotropic separable set abstraction for efficient point cloud representation learning. In: NeurIPS (2021)

    Google Scholar 

  32. Qian, G., et al.: PointNeXt: revisiting PointNet++ with improved training and scaling strategies. In: NeurIPS (2022)

    Google Scholar 

  33. Ran, H., Liu, J., Wang, C.: Surface representation for point clouds. In: CVPR (2022)

    Google Scholar 

  34. Ran, H., Zhuo, W., Liu, J., Lu, L.: Learning inner-group relations on point clouds. In: ICCV (2021)

    Google Scholar 

  35. Riegler, G., Osman Ulusoy, A., Geiger, A.: Octnet: learning deep 3D representations at high resolutions. In: ICCV (2017)

    Google Scholar 

  36. Robert, D., Vallet, B., Landrieu, L.: Learning multi-view aggregation in the wild for large-scale 3D semantic segmentation. In: CVPR (2022)

    Google Scholar 

  37. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: MICCAI (2015)

    Google Scholar 

  38. Rosu, R.A., Schütt, P., Quenzel, J., Behnke, S.: LatticeNet: fast spatio-temporal point cloud segmentation using permutohedral lattices. Auton. Robots 46(1), 45–60 (2022)

    Article  Google Scholar 

  39. Song, S., Yu, F., Zeng, A., Chang, A.X., Savva, M., Funkhouser, T.: Semantic scene completion from a single depth image. In: CVPR (2017)

    Google Scholar 

  40. Su, H., et al.: Splatnet: sparse lattice networks for point cloud processing. In: CVPR (2018)

    Google Scholar 

  41. Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3D shape recognition. In: ICCV (2015)

    Google Scholar 

  42. Tang, L., Zhan, Y., Chen, Z., Yu, B., Tao, D.: Contrastive boundary learning for point cloud segmentation. In: CVPR (2022)

    Google Scholar 

  43. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.: KPConv: flexible and deformable convolution for point clouds. In: ICCV (2019)

    Google Scholar 

  44. Virtanen, P., et al.: SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods 17(3), 261–272 (2020)

    Google Scholar 

  45. Wang, P.S., Liu, Y., Guo, Y.X., Sun, C.Y., Tong, X.: O-CNN: octree-based convolutional neural networks for 3D shape analysis. TOG 36(4), 1–11 (2017)

    Google Scholar 

  46. Wang, Y., Ji, R., Chang, S.F.: Label propagation from ImageNet to 3D point clouds. In: CVPR (2013)

    Google Scholar 

  47. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. TOG 38(5), 1–12 (2019)

    Article  Google Scholar 

  48. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. TIP 13(4), 600–612 (2004)

    Google Scholar 

  49. Wu, X., Lao, Y., Jiang, L., Liu, X., Zhao, H.: Point transformer V2: grouped vector attention and partition-based pooling. In: NeurIPS (2022)

    Google Scholar 

  50. Wu, Z., et al.: 3D shapenets: a deep representation for volumetric shapes. In: CVPR (2015)

    Google Scholar 

  51. Xiang, T., Zhang, C., Song, Y., Yu, J., Cai, W.: Walk in the cloud: learning curves for point clouds shape analysis. In: ICCV (2021)

    Google Scholar 

  52. Xu, M., Ding, R., Zhao, H., Qi, X.: PAConv: position adaptive convolution with dynamic kernel assembling on point clouds. In: CVPR (2021)

    Google Scholar 

  53. Xu, Y., Fan, T., Xu, M., Zeng, L., Qiao, Y.: Spidercnn: deep learning on point sets with parameterized convolutional filters. In: ECCV (2018)

    Google Scholar 

  54. Yan, X., Zheng, C., Li, Z., Wang, S., Cui, S.: PointASNL: robust point clouds processing using nonlocal neural networks with adaptive sampling. In: CVPR (2020)

    Google Scholar 

  55. Yang, J., et al.: Modeling point clouds with self-attention and gumbel subset sampling. In: CVPR (2019)

    Google Scholar 

  56. Yi, L., et al.: A scalable active framework for region annotation in 3D shape collections. TOG 35(6), 1–12 (2016)

    Article  Google Scholar 

  57. Zhang, C., Wan, H., Shen, X., Wu, Z.: PatchFormer: an efficient point transformer with patch attention. In: CVPR (2022)

    Google Scholar 

  58. Zhang, Z., Hua, B.S., Yeung, S.K.: Shellnet: efficient point cloud convolutional neural networks using concentric shells statistics. In: ICCV (2019)

    Google Scholar 

  59. Zhao, H., Jiang, L., Fu, C.W., Jia, J.: Pointweb: enhancing local neighborhood features for point cloud processing. In: CVPR (2019)

    Google Scholar 

  60. Zhao, H., Jiang, L., Jia, J., Torr, P.H., Koltun, V.: Point transformer. In: ICCV (2021)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the CVM 2024 reviewers and program chairs for their valuable feedback. This work is supported by National Natural Science Foundation of China (No. 62032011).

Author information

Authors and Affiliations

  1. State Key Lab of CAD &CG, Zhejiang University, Hangzhou, China

    Gaoyang Zhang & Xinguo Liu

Authors
  1. Gaoyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinguo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinguo Liu .

Editor information

Editors and Affiliations

  1. Victoria University of Wellington, Wellington, New Zealand

    Fang-Lue Zhang

  2. Ben-Gurion University, Be'er Sheva, Israel

    Andrei Sharf

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, G., Liu, X. (2024). Point Cloud Segmentation with Guided Sampling and Continuous Interpolation. In: Zhang, FL., Sharf, A. (eds) Computational Visual Media. CVM 2024. Lecture Notes in Computer Science, vol 14592. Springer, Singapore. https://doi.org/10.1007/978-981-97-2095-8_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-2095-8_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-2094-1

  • Online ISBN: 978-981-97-2095-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation