Skip to main content
SpringerLink
Log in

KSF-SLAM: A Key Segmentation Frame Based Semantic SLAM in Dynamic Environments

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Simultaneous Localization and Mapping is one of the hotspots in the field of mobile intelligent robot research. Over the past decades, many excellent SLAM systems with good performance have been developed. However, many of the systems make the assumption that the environment is static. In this paper, we propose a key segmentation frame based semantic SLAM (KSF-SLAM) method to deal with autonomous navigation in dynamic environments, which can reduce computational complexity. First, a key segmentation frame selection strategy is designed, so that it is unnecessary to perform segmentation in all the image frames. When a key segmentation frame arrives, the semantic segmentation is performed by SegNet, and the dynamic key points in the frame can be stored at the same time. Moreover, an efficient semantic image generation method is proposed when dealing with non-key segmentation frames. Optical flow tracking of the dynamic key points is performed between key segmentation frame and current frame before the next key segmentation frame arrives to generate semantic images for dynamic key points removal in non-key segmentation frames. By this way, an efficient semantic tracking module is added to the SLAM system to remove dynamic objects in dynamic environments. Experiments on TUM RGB-D datasets, KITTI datasets and in real-world environments are conducted to verify the effect of the proposed method. When compared with ORB-SLAM2 and DS-SLAM, the method proposed in this paper can significantly improve the real-time performance of the SLAM system while the positioning accuracy are equivalent.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Code Availability

Some or all codes generated during the study may only be provided with restrictions upon request.

References

  1. Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2018)

    Article  Google Scholar 

  2. Mur-Artal, R., Tardós, J. D.: ORB-SLAM2: An Open-Source SLAM system for monocular, stereo, and RGB-d cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  3. Qin, T., Li, P., Shen, S.: VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator. IEEE Trans. Robot. 34(4), 1004–1020 (2018)

    Article  Google Scholar 

  4. Cheng, J., Wang, C., Meng, M.Q.-H.: Robust visual localization in dynamic environments based on sparse motion removal. IEEE Trans. Autom. Sci. Eng. 17(2), 658–669 (2020)

    Article  Google Scholar 

  5. Cheng, J., Zhang, H., Meng, M.Q.-H.: Improving visual localization accuracy in dynamic environments based on dynamic region removal. IEEE Trans. Autom. Sci. Eng. 17(3), 1585–1596 (2020)

    Google Scholar 

  6. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with apphcatlons to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  Google Scholar 

  7. Derner, E., Gomez, C., Hernandez, A.C., Barber, R., Babuška, R.: Change Detection Using Weighted Features for Image-based Localization. Robot. Auton. Syst. 135, 103676 (2021)

    Article  Google Scholar 

  8. Bescos, B., Fácil, J.M., Civera, J., Neira, J.: DynaSLAM: Tracking, mapping and inpainting in dynamic scenes. IEEE Robotics and Automation Letters 3(4), 4076–4083 (2018)

    Article  Google Scholar 

  9. Yu, C., Liu, Z., Liu, X., Xie, F., Yang, Y., Wei, Q., Qiao, F.: DS-SLAM: A Semantic Visual SLAM Towords Dynamic Environments. International Conference on Intelligent Robots and Systems (IROS), pp 1168–1174 (2018)

  10. Xiao, L., Wang, J., Qiu, X., Rong, Z., Zou, X.: Dynamic-SLAM: Semantic monocular visual localization and mapping based on deep learning in dynamic environment. Robot. Auton. Syst. 117, 1–16 (2019)

    Article  Google Scholar 

  11. Zhang, J., Henein, M., Mahony, R., Ila, V.: VDO-SLAM:, A Visual Dynamic Object-aware SLAM System, arXiv:2005.11052 (2020)

  12. Bescos, B., Campos, C., Tardós, J.D., Neira, J.: DynaSLAM II:, Tightlycoupled multi-object tracking and SLAM, arXiv:2010.07820 (2020)

  13. Xu, B., Li, W., Tzoumanikas, D., Bloesch, M., Davison, A., Leutenegger, S.: MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM. International Conference on Robotics and Automation (ICRA), pp 5231–5237 (2019)

  14. Runz, M., Buffier, M., Agapito, L.: Maskfusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects. Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (IS-MAR), pp 10–20 (2018)

  15. Hornung, A., Wurm, K.M., Bennewitz, M., Stachniss, C., Burgard, W.: Octomap: an efficient probabilistic 3D mapping framework based on octrees. Auton. Robot. 34(3), 189–206 (2013)

    Article  Google Scholar 

  16. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. Proceedings of ACM Multimedia, pp 675–678 (2014)

  17. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  18. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  19. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-d SLAM systems. International Conference on Intelligent Robots and Systems (IROS), pp 573–580 (2012)

  20. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. Liang Lu, Javier Rodriguez-Vazquez, Miguel Fernández-Cortizas and Yiming Ding for the fruitful discussion, David Perez-Saura and Rafael Perez-Segui for their great help in the experiments.

Funding

This work was partially supported by the National Natural Science Foundation of China (Grant No. 62073163, 61873125), the Introduction plan of high end experts(Grant No. G20200010142), the Natural Science Fund of Jiangsu Province (Grant No. BK20181291), the Fundamental Research Funds for the Central Universities(Grant No. NZ2020004), the Aeronautic Science Foundation of China (Grant No. ASFC-2020Z071052001), Shanghai Aerospace Science and Technology Innovation Fund(Grant No. SAST2019-085), the 111 Project(Grant No. B20007) and China Scholarship Council.

Author information

Authors and Affiliations

  1. College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Yao Zhao, Zhi Xiong, Shuailin Zhou, Zheng Peng & Ling Zhang

  2. Computer Vision and Aerial Robotics Group, Universidad Politécnica de Madrid, Madrid, Spain

    Pascual Campoy

Authors
  1. Yao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuailin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zheng Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pascual Campoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Yao Zhao, Shuailin Zhou, Zheng Peng and Ling Zhang. The first draft of the manuscript was written by Yao Zhao. The paper was reviewed by Zhi Xiong and Pascual Campoy, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Zhi Xiong.

Ethics declarations

Consent for Publication

All authors of this research paper have read and approved the final version submitted.

Competing participate

All authors of this research paper have consented to participate in the research study.

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Xiong, Z., Zhou, S. et al. KSF-SLAM: A Key Segmentation Frame Based Semantic SLAM in Dynamic Environments. J Intell Robot Syst 105, 3 (2022). https://doi.org/10.1007/s10846-022-01613-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01613-4

Keywords

Search

Navigation