Skip to main content
Springer Nature Link
Log in

SIA-SLAM: a robust visual SLAM associated with semantic information in dynamic environments

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Semantic information associated Simultaneous Localization and Mapping (SIA-SLAM), a visual SLAM algorithm using semantic information association, is proposed to solve the problems that dynamic objects lead to the decreased accuracy of the localization and feature matching between two frames due to the lack of object semantic information. Firstly, a Solov2 instance segmentation network is used to obtain instance segmentation images, and the feature points are extracted from the RBG images simultaneously. Secondly, the feature points on dynamic objects are removed, and the semantic information of static objects is associated with the remaining feature points. Then, the static feature points are utilized to estimate the camera poses and update the static map point set. Finally, the camera poses are optimized by using closed-loop detection. When tracking the camera poses and inter-frame feature matching during the closed-loop detection, the semantic information of the feature points is checked first, and then the bag-of-words model is used for feature matching. The proposed SIA-SLAM algorithm is tested on the Technische Universität München (TUM) public dataset. As far as the absolute trajectory errors (ATE) are concerned, the Root Mean Square Errors (RMSE) and Standard Deviation (S.D.) improvement values can reach up to 98.15% and 98.18% in high dynamic scene of TUM dataset, respectively. The proposed SIA-SLAM algorithm is superior to other semantic SLAM algorithms which are tested in the specific datasets. Furthermore, the reliability and robustness of the SIA-SLAM algorithm are verified in a real scenario. The SIA-SLAM algorithm effectively improves the accuracy of the camera trajectory estimation and feature matching.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Yes.

Code availability

No (Not applicable).

References

  1. Cadena C, Carlone L, Carrillo H (2016) Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Trans Robot 32(6):1309–1332

    Article  Google Scholar 

  2. Davison AJ, Reid ID, Molton ND (2007) MonoSLAM: Real-time single camera SLAM[J]. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067

    Article  Google Scholar 

  3. Klein G, Murray D (2007) Parallel tracking and mapping for small AR workspaces. 6th IEEE and ACM International Symposium on Mixed and Augmented Reality 225-234

  4. Engel J, Koltun V, Cremers D (2018) Direct sparse odometry. IEEE Trans Pattern Anal Mach Intell 40(3):611–625

    Article  Google Scholar 

  5. Forster C, Pizzoli M, Scaramuzza D (2014) SVO: Fast semi-direct monocular visual odometry. IEEE international conference on robotics and automation (ICRA) 15-22

  6. Mur-Artal R, Tardós JD (2017) ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans Robot 33(5):1255–1262

    Article  Google Scholar 

  7. Sharma K (2018) Improved visual SLAM: a novel approach to mapping and localization using visual landmarks in consecutive frames. Multimed Tools Appl 77(7): 7955-7976

  8. Brasch N, Bozic A, Lallemand J (2018) Semantic monocular SLAM for highly dynamic environments. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 393-400

  9. Wei H, Zhang T, Zhang L (2021) GMSK-SLAM: a new RGB-D SLAM method with dynamic areas detection towards dynamic environments. Multimed Tools Appl 80(21): 31729-31751

  10. Zou D, Tan P (2012) Coslam: Collaborative visual slam in dynamic environments. IEEE Trans Pattern Anal Mach Intell 35(2): 354-366.

  11. Li S, Lee D (2017) RGB-D SLAM in dynamic environments using static point weighting. IEEE Robot Autom Lett 2(4):2263–2270

    Article  Google Scholar 

  12. Sun Y, Liu M, Meng M Q H (2017) Improving RGB-D SLAM in dynamic environments: A motion removal approach. Robot Autonom Syst 89: 110-122.

  13. Yu C, Liu Z X, Liu X J (2018) DS-SLAM: A semantic visual SLAM towards dynamic environments. IEEE/RSJ International Conference on Intelligent Robots and Systems 1168-1174

  14. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12): 2481-2495

  15. Bescos B, Fácil J M, Civera J (2018) DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes. IEEE Robot Autom Papers 3(4): 4076-4083

  16. He K, Gkioxari G, Dollár P (2017) Mask r-cnn. Proceedings of the IEEE international conference on computer vision 2961-2969

  17. Zhong FW, Wang S, Zhang ZQ (2018) Detect-SLAM: Making object detection and SLAM mutually beneficial. IEEE Winter Conference Appl Comput Vision:1001–1010

  18. Liu W, Anguelov D, Erhan D (2016) SSD: Single shot multibox detector. Lect Notes Comput Sci:21–37

  19. Liu Y, Miura J (2021) RDS-SLAM: real-time dynamic SLAM using semantic segmentation methods. IEEE Access 9:23772–23785

    Article  Google Scholar 

  20. Hempel, Thorsten, Ayoub Al-Hamadi (2022) An online semantic mapping system for extending and enhancing visual SLAM. Eng Appl Artificial Intell 111: 104830.

  21. Bochkovskiy A, Wang C-Y, Liao H (2020) YOLOv4: Optimal speed and accuracy of object detection. ArXiv 10934.

  22. Wang X, Zhang R, Kong T (2020) Solov2: Dynamic and fast instance segmentation. Adv Neural Inf Proces Syst 33:17721–17732

    Google Scholar 

  23. Rublee E, Rabaud V, Konolige K (2011) ORB: An efficient alternative to SIFT or SURF. 2011 International Conference on Computer Vision 2564-2571

  24. Sturm J, Engelhard N, Endres F (2012) A benchmark for the evaluation of RGB-D SLAM systems. IEEE/RSJ international conference on intelligent robots and systems 573-580

  25. Gálvez-López D, Tardos JD (2012) Bags of binary words for fast place recognition in image sequences. IEEE Trans Robot 28(5):1188–1197

    Article  Google Scholar 

  26. Teng R, Liang Y, Zhang J B (2021) RS-SLAM: A Robust Semantic SLAM in Dynamic Environments Based on RGB-D Sensor. IEEE Sensors Journal 20657-20664

  27. Lin T Y, Maire M, Belongie S (2014) Microsoft coco: Common objects in context. European conference on computer vision 740-755

Download references

Acknowledgements

This work was funded in part by the National Natural Science Foundation of China (62263031 and 61863033), and the Natural Science Foundation of Xinjiang Uygur Autonomous Region (2022D01C53).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Xinjiang University, Urumqi, China

    Qiang Liu, Jie Yuan & Benfa Kuang

Authors
  1. Qiang Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jie Yuan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Benfa Kuang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jie Yuan.

Ethics declarations

Conflict of Interest/Competing interests

No.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Yuan, J. & Kuang, B. SIA-SLAM: a robust visual SLAM associated with semantic information in dynamic environments. Multimed Tools Appl 83, 53531–53547 (2024). https://doi.org/10.1007/s11042-023-17650-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17650-6

Keywords