Abstract
Lidar-based 3D object detectors utilize point clouds to detect objects in autonomous driving. However, the point clouds are sparse and incomplete, which affects the detectors’ learning of shape knowledge and limits the 3D detection performance. Previous works improve performance through completing object shape at the point level or representation level, such as voxel. The former increases computational burden, while the latter has poor generalization ability to point-based detectors. In this paper, we present an approach, namely Occluded Keypoint Generation and Refinement (OKGR), which is effective to improve 3D detection performance by completing object features at the keypoint level. Specifically, Occluded Keypoint Generation (OKG) generates occluded keypoints to densify raw keypoints and learns the offsets between the generated keypoints and prototypes, while retaining the raw keypoints unchanged. Occluded Keypoint Refinement (OKR) assigns weights to the generated keypoints and conducts these weights to features to obtain high-quality complete features for detection. We apply our approach to two representative detectors, PV-RCNN++ and PDV, and evaluate the detectors on KITTI and Waymo Open Dataset. The experiments show significant performance improvement. Particularly, our OKGR applied on PV-RCNN++ achieves improvements of Pedestrian and Cyclist of +3.19%, +2.53% AP on average difficulty levels on KITTI, and +2.18%, +2.29% mAPH on Waymo Open Dataset. For more information, the supplementary material and code are available at https://github.com/Mingqj/OKGR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Shi, S., Wang, X., Li, H.: PointRCNN: 3D object proposal generation and detection from point cloud. In: CVPR (2019)
Yang, Z., Sun, Y., Liu, S., Shen, X., Jia, J.: IPOD: intensive point-based object detector for point cloud. arXiv:1812.05276 (2018)
Yang, Z., Sun, Y., Liu, S., Shen, X., Jia, J.: STD: sparse-to-dense 3D object detector for point cloud. In: ICCV (2019)
Ngiam, J., et al.: StarNet: targeted computation for object detection in point clouds. arXiv:1908.11069 (2019)
Zhou, Y., Tuzel, O.: VoxelNet: end-to-end learning for point cloud based 3D object detection. In: CVPR (2018)
Yan, Y., Mao, Y., Li, B.: Second: sparsely embedded convolutional detection. Sensors (2018)
Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: PointPillars: fast encoders for object detection from point clouds. In: CVPR (2019)
Li, Z., Yao, Y., Quan, Z., Xie, J., Yang, W.: Spatial information enhancement network for 3D object detection from point cloud. PR (2022)
Xu, Q., Zhou, Y., Wang, W., Qi, C.R., Anguelov, D.: SPG: unsupervised domain adaptation for 3D object detection via semantic point generation. In: ICCV (2021)
Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M.: PCN: point completion network. In: 3DV (2018)
Shi, S., et al.: PV-RCNN++: point-voxel feature set abstraction with local vector representation for 3D object detection. IJCV (2022)
Hu, J.S., Kuai, T., Waslander, S.L.: Point density-aware voxels for lidar 3D object detection. In: CVPR (2022)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: CVPR (2012)
Sun, P., et al.: Scalability in perception for autonomous driving: waymo open dataset. In: CVPR (2020)
Mao, J., Shi, S., Wang, X., Li, H.: 3D object detection for autonomous driving: a review and new outlooks. arXiv:2206.09474 (2022)
Shi, W., Rajkumar, R.: Point-GNN: graph neural network for 3D object detection in a point cloud. In: CVPR (2020)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: NeurIPS (2017)
Shi, S., Wang, Z., Shi, J., Wang, X., Li, H.: From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network. TPAMI (2020)
Yin, T., Zhou, X., Krahenbuhl, P.: Center-based 3D object detection. In: CVPR (2021)
Mao, J., et al.: Voxel transformer for 3D object detection. In: ICCV (2021)
Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., Li, H.: Voxel R-CNN: towards high performance voxel-based 3D object detection. In: AAAI (2021)
Graham, B., Engelcke, M., Van Der Maaten, L.: 3D semantic segmentation with submanifold sparse convolutional networks. In: CVPR (2018)
Shi, S., et al.: PV-RCNN: point-voxel feature set abstraction for 3D object detection. In: CVPR (2020)
Li, Z., Wang, F., Wang, N.: Lidar R-CNN: an efficient and universal 3D object detector. In: CVPR (2021)
Mao, J., Niu, M., Bai, H., Liang, X., Xu, H., Xu, C.: Pyramid R-CNN: towards better performance and adaptability for 3D object detection. In: ICCV (2021)
Eldar, Y., Lindenbaum, M., Porat, M., Zeevi, Y.Y.: The farthest point strategy for progressive image sampling. TIP (1997)
Zhu, X., Ma, Y., Wang, T., Xu, Y., Shi, J., Lin, D.: SSN: shape signature networks for multi-class object detection from point clouds. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12370, pp. 581–597. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58595-2_35
Xu, Q., Zhong, Y., Neumann, U.: Behind the curtain: learning occluded shapes for 3D object detection. In: AAAI (2022)
Wang, T., Hu, X., Liu, Z., Fu, C.W.: Sparse2dense: learning to densify 3D features for 3D object detection. In: NeurIPS (2022)
Wen, X., et al.: PMP-Net: point cloud completion by learning multi-step point moving paths. In: CVPR (2021)
Wen, X., et al.: PMP-Net++: point cloud completion by transformer-enhanced multi-step point moving paths. TPAMI (2022)
Rosenblatt, M.: Remarks on some nonparametric estimates of a density function. Ann. Math. Stat. (1956)
Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Comput. (2000)
Acknowledgement
This work is supported by the National Natural Science Foundation of China (Grant No. 62322602, Grant No. 62172225), CAAI-Huawei MindSpore Open Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ji, M., Yang, J., Zhang, S. (2024). OKGR: Occluded Keypoint Generation and Refinement for 3D Object Detection. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14436. Springer, Singapore. https://doi.org/10.1007/978-981-99-8555-5_1
Download citation
DOI: https://doi.org/10.1007/978-981-99-8555-5_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8554-8
Online ISBN: 978-981-99-8555-5
eBook Packages: Computer ScienceComputer Science (R0)