Abstract
Obstacle detection is a significant and fundamental issue in autonomous driving and robotics. This paper proposes a novel method to locate obstacles in the scene by comprehensively utilizing the sparse point clouds captured by a Lidar and the natural image taken from a camera. We do this because Lidar is able to capture the data accurately, while the object details can be perfectly preserved by an image. To establish the depth map, the proposed method firstly uses cross-calibration to align the point clouds with reference image. Then we introduce the common U-disparity map in the stereo vision to deal with this depth map and extract all the points belonging to obstacles. By employing 3D point coordinates and the pairwise image pixels as input, the features for density based clustering technique are learned from a specific sparse regression model. After adopting the clustering technique, the corresponding obstacles can be localized by a subset of the relevant points. Quantitative and qualitative experimental results on KITTI object detection benchmark reveal that our proposed method achieves very encouraging performances in various practical environments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. Trans Pattern Anal Mach Intell 34(11):2274–2282
Bewley A, Guizilini V, Ramos F, Upcroft B (2014) Online self-supervised multi-instance segmentation of dynamic objects. In: International conference on robotics and automation, pp. 1296–1303. IEEE
Cai JF, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982
Chen X, Kundu K, Zhu Y, Ma H, Fidler S, Urtasun R (2018) 3d object proposals using stereo imagery for accurate object class detection. IEEE Trans Pattern Anal Mach Intell 40(5):1259–1272
Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the kitti dataset. Int J Robot Res 32(11):1231–1237
Held D, Levinson J, Thrun S (2013) Precision tracking with sparse 3d and dense color 2d data. In: International conference on robotics and automation. IEEE, pp 1138–1145
Held D, Levinson J, Thrun S, Savarese S (2014) Combining 3D shape, color, and motion for robust anytime tracking. In: Proceedings of robotics: science and systems, Berkeley, CA
Himmelsbach M, Hundelshausen FV, Wuensche HJ (2010) Fast segmentation of 3d point clouds for ground vehicles. In: Intelligent Vehicles Symposium (IV). IEEE, pp 560–565
Hu Z, Lamosa F, Uchimura K (2005) A complete UV-disparity study for stereovision based 3d driving environment analysis. In: Fifth international conference on 3-D digital imaging and modeling. IEEE, pp 204–211
Kramm S, Bensrhair A (2012) Obstacle detection using sparse stereovision and clustering techniques. In: Intelligent vehicles symposium. IEEE, pp 760–765
Chen Liang, Kong Hui, Y J (2017) A fast lidar-guided scene analysis framework for autonomous driving[c]. In: IEEE international conference on robotics and automation (ICRA)
Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055
Moosmann F, Pink O, Stiller C (2009) Segmentation of 3d lidar data in non-flat urban environments using a local convexity criterion. In: Intelligent vehicles symposium. IEEE, pp 215–220
Pfaff P, Triebel R, Burgard W (2007) An efficient extension to elevation maps for outdoor terrain mapping and loop closing. Int J Robot Res 26(2):217–230
Portilla J, Mancera L (2007) L0-based sparse approximation: two alternative methods and some applications. In: Optical Engineering + Applications. International Society for Optics and Photonics, pp 67011Z–67011Z
Rodriguez A, Laio A (2014) Clustering by fast search and find of density peaks. Science 344(6191):1492–1496
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905
Siciliano B, Khatib O (2016) Springer handbook of robotics. Springer, New York
Sivaraman S, Trivedi MM (2013) Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE Trans Intell Transp Syst 14(4):1773–1795
Thrun S, Montemerlo M, Dahlkamp H, Stavens D, Aron A, Diebel J, Fong P, Gale J, Halpenny M, Hoffmann G et al (2006) Stanley: the robot that won the darpa grand challenge. J Field Robot 23(9):661–692
Vu TD, Aycard O, Tango F (2014) Object perception for intelligent vehicle applications: a multi-sensor fusion approach. In: Intelligent vehicles symposium proceedings. IEEE, pp 774–780
Wang B, Fremont V, Rodriguez SA (2014) Color-based road detection and its evaluation on the kitti road benchmark. In: Intelligent vehicles symposium proceedings. IEEE, pp 31–36
Acknowledgements
The authors would like to thank the editor and anonymous reviewers for their critical and constructive comments and suggestions. This work was supported by the NSF of China under Grant Nos. U1713208, 61472187, 61602246 and 61502235, the 973 Program No.2014CB349303, Program for Changjiang Scholars, NSF of Jiangsu Province No. BK20171430, the Fundamental Research Funds for the Central Universities No. 30918011319 and the Open Project Program of Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, Minjiang University under Grant MJUKF201723.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wei, Y., Yang, J., Gong, C. et al. Obstacle Detection by Fusing Point Clouds and Monocular Image. Neural Process Lett 49, 1007–1019 (2019). https://doi.org/10.1007/s11063-018-9861-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-018-9861-1