Abstract
Recent works have shown remarkable success in 3D object tracking in point clouds. However, these methods may fail when tracking distant objects or objects interfered by similar geometries in point clouds. We aim to use high-resolution images with rich textures to help point cloud based tracking to deal with the above-mentioned failures. In this paper, we propose an end-to-end framework, which effectively uses both image semantic features and geometric features to facilitate tracking in point clouds. Specifically, we design a fusion module to establish the correspondence between image and point cloud features in a point-to-point manner and use attention-weighted image features to enhance point features. In addition, we utilize geometric transformation to convert 2D image geometric features inferred by deep layer aggregation network (DLA) to 3D as extra tracking clues for 3D voting. Quantitative and qualitative comparisons on the KITTI tracking dataset demonstrate the advantages of our model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3D object detection network for autonomous driving. In: CVPR, pp. 6526–6534 (2017)
Fang, Z., Zhou, S., Cui, Y., Scherer, S.: 3D-siamrpn: an end-to-end learning method for real-time 3D single object tracking using raw point cloud. IEEE Sens. J. 21, 4995–5011 (2021)
Feng, T., Jiao, L., Zhu, H., Sun, L.: A novel object re-track framework for 3D point clouds. In: ACM MM, pp. 3118–3126 (2020)
Giancola, S., Zarzar, J., Ghanem, B.: Leveraging shape completion for 3D siamese tracking. In: CVPR, pp. 1359–1368 (2019)
Gordon, N., Ristic, B., Arulampalam, S.: Beyond the Kalman Filter: Particle Filters for Tracking Applications, pp. 1–4. Artech House, London (2004)
Han, W., Dong, X., Khan, F., Shao, L., Shen, J.: Learning to fuse asymmetric feature maps in siamese trackers. In: CVPR, pp. 16570–16580 (2021)
Huang, T., Liu, Z., Chen, X., Bai, X.: Epnet: enhancing point features with image semantics for 3D object detection. In: ECCV, pp. 35–52 (2020)
Kart, U., Kämäräinen, J., Matas, J.: How to make an rgbd tracker? In: ECCV Workshops (2018)
Kart, U., Lukezic, A., Kristan, M., Kämäräinen, J., Matas, J.: Object tracking by reconstruction with view-specific discriminative correlation filters. In: CVPR, pp. 1339–1348 (2019)
Klein, G., Murray, D.: Parallel tracking and mapping for small ar workspaces. In: ISMAR, pp. 225–234 (2007)
Ku, J.T., Mozifian, M., Lee, J., Harakeh, A., Waslander, S.L.: Joint 3D proposal generation and object detection from view aggregation. In: IROS, pp. 1–8 (2018)
Liu, Y., Jing, X., Nie, J., Gao, H., Liu, J., Jiang, G.P.: Context-aware three-dimensional mean-shift with occlusion handling for robust object tracking in rgb-d videos. IEEE Trans. Multimedia 21, 664–677 (2019)
Qi, C., Chen, X., Litany, O., Guibas, L.: Imvotenet: boosting 3D object detection in point clouds with image votes. In: CVPR, pp. 4403–4412 (2020)
Qi, C., Litany, O., He, K., Guibas, L.: Deep hough voting for 3D object detection in point clouds. In: ICCV, pp. 9276–9285 (2019)
Qi, C., Yi, L., Su, H., Guibas, L.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: NIPS (2017)
Qi, H., Feng, C., Cao, Z., Zhao, F., Xiao, Y.: P2b: point-to-box network for 3D object tracking in point clouds. In: CVPR, pp. 6328–6337 (2020)
Simon, M., et al.: Complexer-yolo: real-time 3D object detection and tracking on semantic point clouds. In: CVPR Workshops, pp. 1190–1199 (2019)
Wang, W., Tran, D., Feiszli, M.: What makes training multi-modal classification networks hard? In: CVPR, pp. 12692–12702 (2020)
Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: CVPR, pp. 2411–2418 (2013)
Yan, B., Zhang, X., Wang, D., Lu, H., Yang, X.: Alpha-refine: boosting tracking performance by precise bounding box estimation. In: CVPR, pp. 5289–5298 (2021)
Yu, F., Wang, D., Darrell, T.: Deep layer aggregation. In: CVPR, pp. 2403–2412 (2018)
Zhang, W., Zhou, H., Sun, S., Wang, Z., Shi, J., Loy, C.C.: Robust multi-modality multi-object tracking. In: ICCV, pp. 2365–2374 (2019)
Zhang, Y., Wang, C., Wang, X., Zeng, W., Liu, W.: A simple baseline for multi-object tracking, p. 6 (2020). arXiv preprint arXiv:2004.01888
Zhang, Z., Peng, H.: Ocean: object-aware anchor-free tracking. In: ECCV, pp. 771–787 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, L., Hui, L., Xie, J. (2021). Facilitating 3D Object Tracking in Point Clouds with Image Semantics and Geometry. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13019. Springer, Cham. https://doi.org/10.1007/978-3-030-88004-0_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-88004-0_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88003-3
Online ISBN: 978-3-030-88004-0
eBook Packages: Computer ScienceComputer Science (R0)