Abstract
Simultaneous Localization and Mapping is one of the hotspots in the field of mobile intelligent robot research. Over the past decades, many excellent SLAM systems with good performance have been developed. However, many of the systems make the assumption that the environment is static. In this paper, we propose a key segmentation frame based semantic SLAM (KSF-SLAM) method to deal with autonomous navigation in dynamic environments, which can reduce computational complexity. First, a key segmentation frame selection strategy is designed, so that it is unnecessary to perform segmentation in all the image frames. When a key segmentation frame arrives, the semantic segmentation is performed by SegNet, and the dynamic key points in the frame can be stored at the same time. Moreover, an efficient semantic image generation method is proposed when dealing with non-key segmentation frames. Optical flow tracking of the dynamic key points is performed between key segmentation frame and current frame before the next key segmentation frame arrives to generate semantic images for dynamic key points removal in non-key segmentation frames. By this way, an efficient semantic tracking module is added to the SLAM system to remove dynamic objects in dynamic environments. Experiments on TUM RGB-D datasets, KITTI datasets and in real-world environments are conducted to verify the effect of the proposed method. When compared with ORB-SLAM2 and DS-SLAM, the method proposed in this paper can significantly improve the real-time performance of the SLAM system while the positioning accuracy are equivalent.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Code Availability
Some or all codes generated during the study may only be provided with restrictions upon request.
References
Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2018)
Mur-Artal, R., Tardós, J. D.: ORB-SLAM2: An Open-Source SLAM system for monocular, stereo, and RGB-d cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)
Qin, T., Li, P., Shen, S.: VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator. IEEE Trans. Robot. 34(4), 1004–1020 (2018)
Cheng, J., Wang, C., Meng, M.Q.-H.: Robust visual localization in dynamic environments based on sparse motion removal. IEEE Trans. Autom. Sci. Eng. 17(2), 658–669 (2020)
Cheng, J., Zhang, H., Meng, M.Q.-H.: Improving visual localization accuracy in dynamic environments based on dynamic region removal. IEEE Trans. Autom. Sci. Eng. 17(3), 1585–1596 (2020)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with apphcatlons to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Derner, E., Gomez, C., Hernandez, A.C., Barber, R., Babuška, R.: Change Detection Using Weighted Features for Image-based Localization. Robot. Auton. Syst. 135, 103676 (2021)
Bescos, B., Fácil, J.M., Civera, J., Neira, J.: DynaSLAM: Tracking, mapping and inpainting in dynamic scenes. IEEE Robotics and Automation Letters 3(4), 4076–4083 (2018)
Yu, C., Liu, Z., Liu, X., Xie, F., Yang, Y., Wei, Q., Qiao, F.: DS-SLAM: A Semantic Visual SLAM Towords Dynamic Environments. International Conference on Intelligent Robots and Systems (IROS), pp 1168–1174 (2018)
Xiao, L., Wang, J., Qiu, X., Rong, Z., Zou, X.: Dynamic-SLAM: Semantic monocular visual localization and mapping based on deep learning in dynamic environment. Robot. Auton. Syst. 117, 1–16 (2019)
Zhang, J., Henein, M., Mahony, R., Ila, V.: VDO-SLAM:, A Visual Dynamic Object-aware SLAM System, arXiv:2005.11052 (2020)
Bescos, B., Campos, C., Tardós, J.D., Neira, J.: DynaSLAM II:, Tightlycoupled multi-object tracking and SLAM, arXiv:2010.07820 (2020)
Xu, B., Li, W., Tzoumanikas, D., Bloesch, M., Davison, A., Leutenegger, S.: MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM. International Conference on Robotics and Automation (ICRA), pp 5231–5237 (2019)
Runz, M., Buffier, M., Agapito, L.: Maskfusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects. Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (IS-MAR), pp 10–20 (2018)
Hornung, A., Wurm, K.M., Bennewitz, M., Stachniss, C., Burgard, W.: Octomap: an efficient probabilistic 3D mapping framework based on octrees. Auton. Robot. 34(3), 189–206 (2013)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. Proceedings of ACM Multimedia, pp 675–678 (2014)
Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-d SLAM systems. International Conference on Intelligent Robots and Systems (IROS), pp 573–580 (2012)
Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)
Acknowledgements
The authors would like to thank Dr. Liang Lu, Javier Rodriguez-Vazquez, Miguel Fernández-Cortizas and Yiming Ding for the fruitful discussion, David Perez-Saura and Rafael Perez-Segui for their great help in the experiments.
Funding
This work was partially supported by the National Natural Science Foundation of China (Grant No. 62073163, 61873125), the Introduction plan of high end experts(Grant No. G20200010142), the Natural Science Fund of Jiangsu Province (Grant No. BK20181291), the Fundamental Research Funds for the Central Universities(Grant No. NZ2020004), the Aeronautic Science Foundation of China (Grant No. ASFC-2020Z071052001), Shanghai Aerospace Science and Technology Innovation Fund(Grant No. SAST2019-085), the 111 Project(Grant No. B20007) and China Scholarship Council.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Yao Zhao, Shuailin Zhou, Zheng Peng and Ling Zhang. The first draft of the manuscript was written by Yao Zhao. The paper was reviewed by Zhi Xiong and Pascual Campoy, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Consent for Publication
All authors of this research paper have read and approved the final version submitted.
Competing participate
All authors of this research paper have consented to participate in the research study.
Conflict of Interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhao, Y., Xiong, Z., Zhou, S. et al. KSF-SLAM: A Key Segmentation Frame Based Semantic SLAM in Dynamic Environments. J Intell Robot Syst 105, 3 (2022). https://doi.org/10.1007/s10846-022-01613-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-022-01613-4