Skip to main content
SpringerLink
Log in

FuseNet: 3D Object Detection Network with Fused Information for Lidar Point Clouds

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

3D object detection from lidar point cloud has an important role in the environment sensing system of autonomous driving vehicles. In this paper, we propose two modules for object detection works by more detailed voxel initial information extraction and full fusion of context information. Additionally, we extract density information as the initial feature of the voxels and fully confuse the coordinate and density information with a point-based method to reduce the loss of original data caused by voxelization. Second, we extract the voxel features with a backbone neural network based on 3D sparse convolution. We propose a Cross-connected Region Proposal Network to integrate multiscale and multidepth regional features and to obtain high-quality 3D proposal regions. In addition, we extend the target generation strategy in the anchor-based 3D object detection algorithm, which stabilizes the network performance for multiple objections. Our modules can be flexibly applied to state-of-the-art models and effectively improves the network performance, which proves the effectiveness of the modules that we proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Xiaozhi C, Huimin M, Ji W, Bo L, Tian X (2017) Multi-view 3D object detection network for autonomous driving, Spotlight 4–2B—YouTube. Comput Videos 1907–1915. Available: https://www.youtube.com/watch?v=ChkgSvxAvMg.

  2. Ku J, Mozifian M, Lee J, Harakeh A, Waslander SL (2018) Joint 3D proposal generation and object detection from view aggregation. IEEE Int Conf Intell Robot Syst. https://doi.org/10.1109/IROS.2018.8594049

    Article  Google Scholar 

  3. Girshick R (2015) Fast R-CNN. Proc IEEE Int Conf Comput Vis. https://doi.org/10.1109/ICCV.2015.169

    Article  Google Scholar 

  4. Deng J, Shi S, Li P, Zhou W, Zhang Y, Li H (2020) Voxel R-CNN: towards high performance voxel-based 3D object detection. Available: http://arxiv.org/abs/2012.15712.

  5. Yan Y, Mao Y, Li B (2018) Second: Sparsely embedded convolutional detection. Sensors (Switzerland) 18(10):1–17. https://doi.org/10.3390/s18103337

    Article  Google Scholar 

  6. Shi S, Wang Z, Shi J, Wang X, Li H (2020) From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/tpami.2020.2977026

    Article  Google Scholar 

  7. Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the KITTI vision benchmark suite. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. https://doi.org/10.1109/CVPR.2012.6248074

    Article  Google Scholar 

  8. Qi CR, Su H, Mo K, Guibas LJ (2017) Pointnet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 652-660). https://doi.org/10.1109/CVPR.2017.16

  9. Qi CR, Yi L, Su H, Guibas LJ (2017) PointNet++: Deep hierarchical feature learning on point sets in a metric space. Adv Neural Inf Process Syst 30:5100–5109

    Google Scholar 

  10. Qi CR, Liu W, Wu C, Su H, Guibas LJ (2018) Frustum pointnets for 3D object detection from Rgb-D data. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 918-927). https://doi.org/10.1109/CVPR.2018.00102.

  11. Jaritz M, Gu J, Su H (2019) Multi-view pointnet for 3d scene understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019 (pp. 0-0). https://doi.org/10.1109/ICCVW.2019.00494.

  12. Yang Z, Sun Y, Liu S, Shen X, Jia J (2019) Std: Sparse-to-dense 3d object detector for point cloud. In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2019 (pp. 1951-1960). https://doi.org/10.1109/ICCV.2019.00204.

  13. Liang M, Yang B, Wang S, Urtasun R (2018) Deep continuous fusion for multi-sensor 3D object detection. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). https://doi.org/10.1007/978-3-030-01270-0_39.

  14. He C, Zeng H, Huang J, Hua XS, Zhang L (2020) Structure aware single-stage 3D object detection from point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11873-11882). https://doi.org/10.1109/CVPR42600.2020.01189.

  15. Shi S, et al. (2020) Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10529-10538)., https://doi.org/10.1109/CVPR42600.2020.01054.

  16. Lang AH, Vora S, Caesar H, Zhou L, Yang J, Beijbom O (2019) Pointpillars: Fast encoders for object detection from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12697-12705). https://doi.org/10.1109/CVPR.2019.01298.

  17. Shi S, Wang X, Li H (2019) Pointrcnn: 3D object proposal generation and detection from point cloud. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 770-779). https://doi.org/10.1109/CVPR.2019.00086.

  18. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  19. Zheng W, Tang W, Jiang L, Fu C-W (2021) SE-SSD: Self-ensembling single-stage object detector from point cloud (pp. 14494–14503). Available: http://arxiv.org/abs/2104.09804.s

  20. Simon M, et al. (2018) Complex-YOLO: Real-time 3D object detection on point clouds.

  21. Yang B, Luo W, Urtasun R (2019) PIXOR: Real-time 3D object detection from point clouds. In: 2018 IEEE/CVF conference on computer vision and pattern recognition IEEE.

  22. Li C et al (2021) HDNet: Hybrid distance network for semantic segmentation. Neurocomputing 447:129–144

    Article  Google Scholar 

  23. Zhang S, Wen L, Bian X, Lei Z, Li SZ (2018) RefinementNet. In: Proceeding IEEE conference computer visual pattern recognition (pp. 4203–4212)

  24. Li Z, Yao Y, Quan Z, Yang W, Xie J (2021) SIENet: spatial information enhancement network for 3D object detection from point cloud (pp. 32–34). Available: http://arxiv.org/abs/2103.15396.

  25. Tan M, Pang R, Le QV (2020) Efficientdet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10781-10790). https://doi.org/10.1109/CVPR42600.2020.01079.

  26. Yang Z, Sun Y, Liu S, Jia J (2020) 3DSSD: Point-based 3d single stage object detector. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11040-11048). https://doi.org/10.1109/CVPR42600.2020.01105.

  27. Liu Z, Zhao X, Huang T, Hu R, Zhou Y, Bai X (2020) Tanet: Robust 3d object detection from point clouds with triple attention. In Proceedings of the AAAI Conference on Artificial Intelligence 34(07): 11677-11684. https://doi.org/10.1609/aaai.v34i07.6837

  28. Chen Q, Sun L, Wang Z, Jia K, Yuille A (2020) Object as hotspots: an anchor-free 3D object detection approach via firing of hotspots. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). https://doi.org/10.1007/978-3-030-58589-1_5

  29. Wang Z, Jia K (2019) Frustum convnet: Sliding frustums to aggregate local point-wise features for amodal 3D object detection. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1742-1749). IEEE. https://doi.org/10.1109/IROS40897.2019.8968513.

  30. Shi W, Rajkumar R (2020) Point-gnn: Graph neural network for 3d object detection in a point cloud. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 1711-1719). https://doi.org/10.1109/CVPR42600.2020.00178.

Download references

Acknowledgements

This work was supported by the Fundamental Research Funds for the Central Universities (2021JBM024) and Beijing Natural Science Foundation (L201021).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Beijing Jiaotong University, Beijing, 100044, China

    Biao Liu, Bihao Tian, Hengyang Wang & Junchao Qiao

  2. School of Science, Beijing Jiaotong University, Beijing, 100044, China

    Zhi Wang

Authors
  1. Biao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bihao Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hengyang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junchao Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bihao Tian.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, B., Tian, B., Wang, H. et al. FuseNet: 3D Object Detection Network with Fused Information for Lidar Point Clouds. Neural Process Lett 54, 5063–5078 (2022). https://doi.org/10.1007/s11063-022-10848-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10848-z

Keywords

Search

Navigation