Abstract
LiDAR-based 3D object detection is efficient for autonomous driving because high accuracy LiDAR information is extremely useful for 3D proposals generation and 3D boxes regression. However, some background and foreground objects may have similar appearances in point clouds. Therefore the accuracy of LiDAR-based 3D object detection is hard to be improved. In this paper, we propose a three-stage 3D object detection method called RGB3D to reinforce LiDAR-based 3D object detection by using an RGB-D classifier with a 3D classifier in parallel. We also apply proper training method to improve the performance of the added classifiers. The 3D classifier is trained by using higher IoU threshold with refined 3D information, and the RGB-D classifier is trained with resized 2D RoIs projected from refined 3D boxes. Extensive experiments are conducted on the KITTI object detection benchmark. The results show that the proposed method is effective.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cai, Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detection. In: CVPR (2018)
Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3D object detection network for autonomous driving. In: IEEE Conference on Computer Vision & Pattern Recognition (2017)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. (2017)
Jiang, M., Wu, Y., Zhao, T., Zhao, Z., Lu, C.: PointSIFT: a sift-like network module for 3D point cloud semantic segmentation (2018)
Ku, J., Mozifian, M., Lee, J., Harakeh, A., Waslander, S.: Joint 3D proposal generation and object detection from view aggregation. In: IROS (2018)
Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: PointPillars: fast encoders for object detection from point clouds. In: CVPR (2019)
Li, P., Chen, X., Shen, S.: Stereo R-CNN based 3D object detection for autonomous driving. In: CVPR (2019)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. Arxiv 79, November 2014
Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum pointNets for 3D object detection from RGB-D data. arXiv preprint arXiv:1711.08488 (2017)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. arXiv preprint arXiv:1612.00593 (2016)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413 (2017)
Qin, Z., Wang, J., Lu, Y.: MonoGRNet: a geometric reasoning network for 3D object localization. In: The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) (2019)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Computer Vision & Pattern Recognition (2016)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)
Shi, S., Wang, X., Li, H.: PointRCNN: 3D object proposal generation and detection from point cloud. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
Shin, K., Kwon, Y.P., Tomizuka, M.: RoarNet: a robust 3D object detection based on region approximation refinement (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014)
Wang, Y., Chao, W.L., Garg, D., Hariharan, B., Campbell, M., Weinberger, K.: Pseudo-LiDAR from visual depth estimation: bridging the gap in 3D object detection for autonomous driving. In: CVPR (2019)
Wang, Z., Jia, K.: Frustum convNet: sliding frustums to aggregate local point-wise features for amodal 3D object detection (2019)
Yan, Y., Mao, Y., Li, B.: Second: sparsely embedded convolutional detection. Sensors 18, 3337 (2018). https://doi.org/10.3390/s18103337
Yin, Z., Tuzel, O.: VoxelNet: end-to-end learning for point cloud based 3D object detection (2017)
Acknowledgment
This work is supported by Shanghai Automotive Industry Sci-Tech Development Foundation (No. 1823).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, W., Zhou, Y. (2019). Reinforcing LiDAR-Based 3D Object Detection with RGB and 3D Information. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11954. Springer, Cham. https://doi.org/10.1007/978-3-030-36711-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-36711-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36710-7
Online ISBN: 978-3-030-36711-4
eBook Packages: Computer ScienceComputer Science (R0)