Skip to main content
SpringerLink
Log in

Semantic scan context: a novel semantic-based loop-closure method for LiDAR SLAM

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

As one of the key technologies of SLAM, loop-closure detection can help eliminate the cumulative errors of the odometry. Many of the current LiDAR-based SLAM systems do not integrate a loop-closure detection module, so they will inevitably suffer from cumulative errors. This paper proposes a semantic-based place recognition method called Semantic Scan Context (SSC), which consists of the two-step global ICP and the semantic-based descriptor. Thanks to the use of high-level semantic features, our descriptor can effectively encode scene information. The proposed two-step global ICP can help eliminate the influence of rotation and translation on descriptor matching and provide a good initial value for geometric verification. Further, we built a complete loop-closure detection module based on SSC and combined it with the famous LOAM to form a full LiDAR SLAM system. Exhaustive experiments on the KITTI and KITTI-360 datasets show that our approach is competitive to the state-of-the-art methods, robust to the environment, and has good generalization ability. Our code is available at:https://github.com/lilin-hitcrt/SSC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Angeli, A., Filliat, D., Doncieux, S., & Meyer, J. (2008). Fast and incremental method for loop-closure detection using bags of visual words. IEEE Transactions on Robotics, 24(5), 1027–1037.

    Article  Google Scholar 

  • Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J .(2016). Netvlad: Cnn architecture for weakly supervised place recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5297–5307).

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). SURF: Speeded Up Robust Features. In A. Leonardis, H. Bischof, & A. Pinz (Eds.), Computer Vision - ECCV 2006 (pp. 404–417). Berlin: Springer.

    Chapter  Google Scholar 

  • Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., & Gall, J. (2019). Semantickitti: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEE International Conference on Computer Vision (pp. 9297–9307).

  • Chang, MY., Yeon, S., Ryu, S., & Lee, D. (2020). Spoxelnet: Spherical voxel-based deep place recognition for 3d point clouds of crowded indoor spaces. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 8564–8570).

  • Chen, X., Läbe, T., Milioto, A., Röhling, T., Behley, J., & Stachniss, C .(2021). OverlapNet: a siamese network for computing LiDAR scan similarity with applications to loop closing and localization. Autonomous Robots

  • Chen, X., Läbe, T., Milioto, A., Röhling, T., Vysotska, O., Haag, A., Behley, J., & Stachniss, C .(2020). OverlapNet: Loop Closing for LiDAR-based SLAM. In Proceedings of Robotics: Science and Systems (RSS).

  • Cop, K.P., Borges, P.V.K., & Dubé, R .(2018). Delight: An efficient descriptor for global localisation using lidar intensities. In 2018 IEEE International Conference on Robotics and Automation (ICRA) (pp. 3653–3660).

  • Du, J., Wang, R., & Cremers, D. (2020). Dh3d: Deep hierarchical 3d descriptors for robust large-scale 6dof relocalization. In European Conference on Computer Vision (pp. 744–762).

  • Dubé, R., Cramariuc, A., Dugas, D., Sommer, H., Dymczyk, M., Nieto, J., Siegwart, R., & Cadena, C. (2019). Segmap: Segment-based mapping and localization using data-driven descriptors. The International Journal of Robotics Research p 0278364919863090.

  • Dubé, R., Dugas, D., Stumm, E., Nieto, J., Siegwart, R., & Cadena, C. (2017). Segmatch: Segment based place recognition in 3d point clouds. In 2017 IEEE International Conference on Robotics and Automation (ICRA) (pp. 5266–5272).

  • Fan, Y., He, Y., & Tan, U.X. (2020). Seed: A segmentation-based egocentric 3d point cloud descriptor for loop closure detection. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5158–5163).

  • Ferrarini, B., Waheed, M., Waheed, S., Ehsan, S., Milford, M. J., & McDonald-Maier, K. D. (2020). Exploring performance bounds of visual place recognition using extended precision. IEEE Robotics and Automation Letters, 5(2), 1688–1695.

    Article  Google Scholar 

  • Galvez-López, D., & Tardos, J. D. (2012). Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics, 28(5), 1188–1197.

    Article  Google Scholar 

  • Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013). Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11), 1231–1237.

    Article  Google Scholar 

  • Guo, J., Borges, P. V. K., Park, C., & Gawel, A. (2019). Local descriptor for robust place recognition using lidar intensity. IEEE Robotics and Automation Letters, 4(2), 1470–1477.

    Article  Google Scholar 

  • Han, F., Wang, H., Huang, G., & Zhang, H. (2018). Sequence-based sparse optimization methods for long-term loop closure detection in visual slam. Autonomous Robots, 42(7), 1323–1335.

    Article  Google Scholar 

  • He, L., Wang, X., & Zhang, H. (2016). M2dp: A novel 3d point cloud descriptor and its application in loop closure detection. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 231–237).

  • Ji, X., Zuo, L., Zhang, C., & Liu, Y. (2019).Lloam: Lidar odometry and mapping with loop-closure detection based correction. In 2019 IEEE International Conference on Mechatronics and Automation (ICMA) (pp. 2475–2480).

  • Johnson, A. E., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449.

    Article  Google Scholar 

  • Kim, G., & Kim, A. (2018). Scan context: Egocentric spatial descriptor for place recognition within 3d point cloud map. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4802–4809).

  • Kim, G., Park, B., & Kim, A. (2019). 1-day learning, 1-year localization: Long-term lidar localization using scan context image. IEEE Robotics and Automation Letters, 4(2), 1948–1955.

    Article  Google Scholar 

  • Komorowski, J. (2021). Minkloc3d: Point cloud based large-scale place recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1790–1799).

  • Kong, X., Yang, X., Zhai., G., Zhao, X., Zeng, X., Wang, M., Liu, Y., Li, W., & Wen, F. (2020). Semantic graph based place recognition for 3d point clouds. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 8216–8223).

  • Liao, Y., Xie, J., & Geiger, A. (2021) .KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. arXiv:2109.13410.

  • Liu, Z., Zhou, S., Suo, C., Yin, P., Chen, W., Wang, H. Li, H. & Liu, Y.H. (2019). PD-NET: 3d point cloud learning for large-scale place recognition and environment analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2831–2840).

  • Lu,W., Zhou, Y., Wan, G., Hou, S., & Song, S. (2019). L3-net: Towards learning based lidar localization for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6389–6398).

  • Milioto, A., Vizzo, I., Behley, J., & Stachniss, C. (2019). Rangenet++: Fast and accurate lidar semantic segmentation. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4213–4220).

  • Muhammad, N., Fuentes-Perez, J. F., Tuhtan, J. A., Toming, G., Musall, M., & Kruusmaa, M. (2019). Map-based localization and loop-closure detection from a moving underwater platform using flow features. Autonomous Robots, 43(6), 1419–1434.

    Article  Google Scholar 

  • Mur-Artal, R., & Tardós, J. D. (2017). ORB-SLAM2: An open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Transactions on Robotics, 33(5), 1255–1262.

    Article  Google Scholar 

  • Negre Carrasco, P. L., Bonin-Font, F., & Oliver-Codina, G. (2016). Global image signature for visual loop-closure detection. Autonomous Robots, 40(8), 1403–1417.

    Article  Google Scholar 

  • Qi, C.R., Su, H., Mo, K., & Guibas, L.J. (2017) .Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 652–660).

  • Qin, T., Li, P., & Shen, S. (2018). Vins-mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics, 34(4), 1004–1020.

    Article  Google Scholar 

  • Röhling, T., Mack, J., & Schulz, D. (2015). A fast histogram-based similarity measure for detecting loop closures in 3-d lidar data. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 736–741).

  • Rozenberszki, D., & Majdik, A.L. (2020). Lol: Lidar-only odometry and localization in 3d point cloud maps \(<\)sup\(>\) *\(<\)sup\(>\). In 2020 IEEE International Conference on Robotics and Automation (ICRA) (pp. 4379–4385).

  • Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011). Orb: An efficient alternative to sift or surf. In 2011 International Conference on Computer Vision (pp. 2564–2571).

  • Schaupp, L., Bürki, M., Dubé, R., Siegwart, R., & Cadena, C. (2019). Oreos: Oriented recognition of 3d point clouds in outdoor scenarios. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3255–3261).

  • Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., & Li, H. (2020). PV-RCNN: Point-voxel feature set abstraction for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10529–10538).

  • Tang, H., Liu, Z., Zhao, S., Lin, Y., Lin, J., Wang, H., & Han, S. (2020). Searching efficient 3d architectures with sparse point-voxel convolution. In European Conference on Computer Vision (pp. 685–702).

  • Uy, MA., & Lee, G.H. (2018). Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4470–4479).

  • Wang, Y., Sun, Z., Xu, C.Z., Sarma, S.E., Yang, J., & Kong, H. (2020b). Lidar iris for loop-closure detection. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5769–5775).

  • Wang, H., Wang, C., & Xie, L. (2020a). Intensity scan context: Coding intensity and geometry relations for loop closure detection. In 2020 IEEE International Conference on Robotics and Automation (ICRA) (pp. 2095–2101).

  • Wohlkinger, W., & Vincze, M. (2011). Ensemble of shape functions for 3d object classification. In 2011 IEEE International Conference on Robotics and Biomimetics (pp. 2987–2992).

  • Yang, J., Li, H., Campbell, D., & Jia, Y. (2016). Go-ICP: A globally optimal solution to 3d ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(11), 2241–2254.

    Article  Google Scholar 

  • Yin, H., Tang, L., Ding, X., Wang, Y., & Xiong, R. (2018). Locnet: Global localization in 3d point clouds for mobile vehicles. In 2018 IEEE Intelligent Vehicles Symposium (IV) (pp. 728–733).

  • Yin, P., Wang, F., Egorov, A., Hou, J., Zhang, J., & Choset, H. (2020). Seqspherevlad: Sequence matching enhanced orientation-invariant place recognition. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5024–5029).

  • Zhang, J., & Singh, S. (2017). Low-drift and real-time lidar odometry and mapping. Autonomous Robots, 41(2), 401–416.

    Article  Google Scholar 

  • Zhu, Y., Ma, Y., Chen, L., Liu, C., Ye, M., & Li, L. (2020). Gosmatch: Graph-of-semantics matching for detecting loop closures in 3d lidar data. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5151–5157).

  • Zhu, X., Zhou, H., Wang, T., Hong, F., Ma, Y., Li, W., Li, H., & Lin, D. (2021). Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9939–9948).

Download references

Author information

Authors and Affiliations

  1. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, 310027, People’s Republic of China

    Lin Li, Xin Kong, Xiangrui Zhao, Tianxin Huang & Yong Liu

Authors
  1. Lin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangrui Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianxin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Kong, X., Zhao, X. et al. Semantic scan context: a novel semantic-based loop-closure method for LiDAR SLAM. Auton Robot 46, 535–551 (2022). https://doi.org/10.1007/s10514-022-10037-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-022-10037-w

Keywords

Search

Navigation