Skip to main content
SpringerLink
Log in

Structure Guided Proposal Completion for 3D Object Detection

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13841))

Included in the following conference series:

  • 435 Accesses

Abstract

3D object detection from point clouds is one of the key components in autonomous driving. Current two-stage detectors generate a small number of proposals, and then refine them in the second RCNN procedure. However, due to the inherent sparsity of point clouds, the first stage may predict some low quality proposals with incomplete structure and inaccurate localization. These low quality proposals fail to obtain adequate and precise proposal features which are essential for the following refinement, inevitably degrading the overall detection performance. To alleviate this problem, we propose Structure guided Proposal Completion (SPC) for 3D object detection from point clouds. Specifically, two completion strategies are developed to obtain high quality proposals: one is Structure Completion, in which a group of structural proposals are obtained by traversing most structures, and thus at least one proposal with ground truth similar structure can be guaranteed. The other is RoI Feature Completion, which is used to fill the empty area of proposals with virtual points under structure-aware manner. With the proposed SPC, high quality proposals with clearer structure and more precise localization can be obtained, and further promote the RCNN to perceive adequate proposal features. Extensive experiments on KITTI benchmark demonstrate the effectiveness of our proposed method, especially for hard setting objects with fewer LiDAR points.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bansal, M., Krizhevsky, A., Ogale, A.: ChauffeurNet: learning to drive by imitating the best and synthesizing the worst. arXiv preprint arXiv:1812.03079 (2018)

  2. Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3D object detection network for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1907–1915 (2017)

    Google Scholar 

  3. Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., Li, H.: Voxel r-CNN: towards high performance voxel-based 3D object detection. arXiv preprint arXiv:2012.15712 1(2), 4 (2020)

  4. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the kitti dataset. The Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

  5. Grigorescu, S., Trasnea, B., Cocias, T., Macesanu, G.: A survey of deep learning techniques for autonomous driving. Int. J. Robot. Res. 37(3), 362–386 (2020)

    Google Scholar 

  6. He, C., Zeng, H., Huang, J., Hua, X.S., Zhang, L.: Structure aware single-stage 3D object detection from point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11873–11882 (2020)

    Google Scholar 

  7. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  8. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12697–12705 (2019)

    Google Scholar 

  9. Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.W., Heng, P.A.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)

    Article  Google Scholar 

  10. Li, Z., Wang, F., Wang, N.: Lidar r-CNN: an efficient and universal 3d object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7546–7555 (2021)

    Google Scholar 

  11. Liang, M., Yang, B., Chen, Y., Hu, R., Urtasun, R.: Multi-task multi-sensor fusion for 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7345–7353 (2019)

    Google Scholar 

  12. Liang, M., Yang, B., Wang, S., Urtasun, R.: Deep continuous fusion for multi-sensor 3d object detection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 663–678. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_39

    Chapter  Google Scholar 

  13. Liu, Z., Zhao, X., Huang, T., Hu, R., Zhou, Y., Bai, X.: TaNet: robust 3D object detection from point clouds with triple attention. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 11677–11684 (2020)

    Google Scholar 

  14. Ma, X., et al.: Delving into localization errors for monocular 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4721–4730 (2021)

    Google Scholar 

  15. Mao, J., Niu, M., Bai, H., Liang, X., Xu, H., Xu, C.: Pyramid r-CNN: towards better performance and adaptability for 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2723–2732 (2021)

    Google Scholar 

  16. Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum pointnets for 3D object detection from RGB-d data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 918–927 (2018)

    Google Scholar 

  17. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)

    Google Scholar 

  18. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: 30th Proceedings of Conference on Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  19. Qian, X., Wang, L., Zhu, Y., Zhang, L., Fu, Y., Xue, X.: Impdet: exploring implicit fields for 3D object detection. arXiv preprint arXiv:2203.17240 (2022)

  20. Sheng, H., et al.: Improving 3d object detection with channel-wise transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2743–2752 (2021)

    Google Scholar 

  21. Shi, S., et al.: PV-RCNN: point-voxel feature set abstraction for 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10529–10538 (2020)

    Google Scholar 

  22. Shi, S., Wang, X., Li, H.: Point RCNN: 3D object proposal generation and detection from point cloud. In: Proceedings of the IEEE/CVF Conference On Computer Vision and Pattern Recognition, pp. 770–779 (2019)

    Google Scholar 

  23. Shi, S., Wang, Z., Wang, X., Li, H.: Part-a\(\hat{\,}\)2 Net: 3D part-aware and aggregation neural network for object detection from point cloud. arXiv preprint arXiv:1907.03670 2(3) (2019)

  24. Vora, S., Lang, A.H., Helou, B., Beijbom, O.: Pointpainting: sequential fusion for 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4604–4612 (2020)

    Google Scholar 

  25. Wu, X., et al.: Sparse fuse dense: towards high quality 3D detection with depth completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5418–5427 (2022)

    Google Scholar 

  26. Xie, L., et al.: Pi-RCNN: an efficient multi-sensor 3D object detector with point-based attentive CONT-conv fusion module. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12460–12467 (2020)

    Google Scholar 

  27. Xu, Q., Zhong, Y., Neumann, U.: Behind the curtain: learning occluded shapes for 3D object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 2893–2901 (2022)

    Google Scholar 

  28. Yan, Y., Mao, Y., Li, B.: Second: sparsely embedded convolutional detection. Sensors 18(10), 3337 (2018)

    Article  Google Scholar 

  29. Yang, Z., Sun, Y., Liu, S., Jia, J.: 3DSSD: point-based 3D single stage object detector. In: Proceedings of the IEEE/CVF Conference On Computer Vision and Pattern Recognition, pp. 11040–11048 (2020)

    Google Scholar 

  30. Yang, Z., Sun, Y., Liu, S., Shen, X., Jia, J.: STD: sparse-to-dense 3D object detector for point cloud. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1951–1960 (2019)

    Google Scholar 

  31. Yin, T., Zhou, X., Krähenbühl, P.: Multimodal virtual point 3D detection. In: 34th Proceedings of Advances in Neural Information Processing Systems (2021)

    Google Scholar 

  32. Yu, L., Li, X., Fu, C.W., Cohen-Or, D., Heng, P.A.: Pu-Net: point cloud upsampling network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2790–2799 (2018)

    Google Scholar 

  33. Zheng, W., Tang, W., Chen, S., Jiang, L., Fu, C.W.: CIA-SSD: confident IOU-aware single-stage object detector from point cloud. arXiv preprint arXiv:2012.03015 (2020)

  34. Zheng, W., Tang, W., Jiang, L., Fu, C.W.: SE-SSD: self-ensembling single-stage object detector from point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14494–14503 (2021)

    Google Scholar 

  35. Zhou, Y., Tuzel, O.: VoxelNet: end-to-end learning for point cloud based 3D object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4490–4499 (2018)

    Google Scholar 

Download references

Acknowledgments

This work was partly funded by NSFC(No.61971281), Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102), and STCSM(18DZ2270700).

Author information

Authors and Affiliations

  1. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Chao Shi, Chongyang Zhang & Yan Luo

  2. Shanghai Key Laboratory of Digital Media Processing and Transmission, Shanghai, 200240, China

    Chongyang Zhang

Authors
  1. Chao Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chongyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chongyang Zhang .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shi, C., Zhang, C., Luo, Y. (2023). Structure Guided Proposal Completion for 3D Object Detection. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13841. Springer, Cham. https://doi.org/10.1007/978-3-031-26319-4_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26319-4_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26318-7

  • Online ISBN: 978-3-031-26319-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation