Skip to main content
SpringerLink
Log in

Accurate RGB-D SLAM in dynamic environments based on dynamic visual feature removal

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Visual localization is considered an essential capability in robotics and has attracted increasing interest for the past few years. However, most proposed visual localization systems assume that the surrounding environment is static, which is difficult to maintain in real-world scenarios due to the presence of moving objects. In this paper, we present DFR-SLAM, a real-time and accurate RGB-D SLAM based on ORB-SLAM2 that achieves satisfactory performance in a variety of challenging dynamic scenarios. At the core of our system lies a motion consensus filtering algorithm estimating the initial camera pose and a graph-cut optimization framework combining long-term observations, prior information, and spatial coherence to jointly distinguish dynamic and static visual features. Other systems for dynamic environments detect dynamic components by using the information from short time-span frames, whereas our system uses observations from a long period of keyframes. We evaluate our system using dynamic sequences from the public TUM dataset, and the evaluation demonstrates that the proposed system outperforms the original ORB-SLAM2 system significantly. In addition, our system provides competitive localization accuracy with satisfactory real-time performance compared to closely related SLAM systems designed to adapt to dynamic environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bresson G, Alsayed Z, Yu L, et al. Simultaneous localization and mapping: a survey of current trends in autonomous driving. IEEE Trans Intell Veh, 2017, 2: 194–220

    Article  Google Scholar 

  2. Qin T, Chen T, Chen Y, et al. AVP-SLAM: semantic visual mapping and localization for autonomous vehicles in the parking lot. In: Proceedings of International Conference on Intelligent Robots and Systems, Las Vegas, 2020. 5939–5945

  3. Fang B, Mei G, Yuan X, et al. Visual SLAM for robot navigation in healthcare facility. Pattern Recogn, 2021, 113: 107822

    Article  Google Scholar 

  4. Zhao C H, Fan B, Hu J W, et al. Homography-based camera pose estimation with known gravity direction for UAV navigation. Sci China Inf Sci, 2021, 64: 112204

    Article  Google Scholar 

  5. Yang D F, Sun F C, Wang S C, et al. Simultaneous estimation of ego-motion and vehicle distance by using a monocular camera. Sci China Inf Sci, 2014, 57: 052205

    Article  Google Scholar 

  6. Mur-Artal R, Tardos J D. ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans Robot, 2017, 33: 1255–1262

    Article  Google Scholar 

  7. Lv W J, Kang Y, Qin J H. FVO: floor vision aided odometry. Sci China Inf Sci, 2019, 62: 012202

    Article  Google Scholar 

  8. Yunus R, Li Y, Tombari F. ManhattanSLAM: robust planar tracking and mapping leveraging mixture of manhattan frames. 2021. ArXiv:2103.15068

  9. Fischler M A, Bolles R C. Random sample consensus. Commun ACM, 1981, 24: 381–395

    Article  Google Scholar 

  10. Yu C, Liu Z, Liu X J, et al. DS-SLAM: a semantic visual SLAM towards dynamic environments. In: Proceedings of International Conference on Intelligent Robots and Systems, Madrid, 2018. 1168–1174

  11. Li A, Wang J, Xu M, et al. DP-SLAM: a visual SLAM with moving probability towards dynamic environments. Inf Sci, 2021, 556: 128–142

    Article  Google Scholar 

  12. Cheng J, Zhang H, Meng M Q H. Improving visual localization accuracy in dynamic environments based on dynamic region removal. IEEE Trans Automat Sci Eng, 2020, 17: 1585–1596

    Article  Google Scholar 

  13. Barber C B, Dobkin D P, Huhdanpaa H. The quickhull algorithm for convex hulls. ACM Trans Math Softw, 1996, 22: 469–483

    Article  MathSciNet  MATH  Google Scholar 

  14. Sturm J, Engelhard N, Endres F, et al. A benchmark for the evaluation of RGB-D SLAM systems. In: Proceedings of International Conference on Intelligent Robots and Systems, Vilamoura, 2012. 573–580

  15. Li S, Lee D. RGB-D SLAM in dynamic environments using static point weighting. IEEE Robot Autom Lett, 2017, 2: 2263–2270

    Article  Google Scholar 

  16. Cheng J, Wang C, Meng M Q H. Robust visual localization in dynamic environments based on sparse motion removal. IEEE Trans Automat Sci Eng, 2019, 17: 658–669

    Article  Google Scholar 

  17. Dai W, Zhang Y, Li P, et al. RGB-D SLAM in dynamic environments using point correlations. IEEE Trans Pattern Anal Mach Intell, 2022, 44: 373–389

    Article  Google Scholar 

  18. Kim D H, Han S B, Kim J H. Visual odometry algorithm using an RGB-D sensor and IMU in a highly dynamic environment. In: Robot Intelligence Technology and Applications 3. Cham: Springer, 2015. 11–26

    Chapter  Google Scholar 

  19. Yang D, Bi S, Wang W, et al. DRE-SLAM: dynamic RGB-D encoder SLAM for a differential-drive robot. Remote Sens, 2019, 11: 380

    Article  Google Scholar 

  20. Hyun D, Park C, Yang M C, et al. Target-aware convolutional neural network for target-level sentiment analysis. Inf Sci, 2019, 491: 166–178

    Article  Google Scholar 

  21. Bruno H M S, Colombini E L. LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method. Neurocomputing, 2021, 455: 97–110

    Article  Google Scholar 

  22. Yuan X, Chen S. SaD-SLAM: a visual SLAM based on semantic and depth information. In: Proceedings of International Conference on Intelligent Robots and Systems, Las Vegas, 2020. 4930–4935

  23. Bescos B, Campos C, Tardos J D, et al. DynaSLAM II: tightly-coupled multi-object tracking and SLAM. IEEE Robot Autom Lett, 2021, 6: 5191–5198

    Article  Google Scholar 

  24. He K, Gkioxari G, Dollár P, et al. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, Venice, 2017. 2961–2969

  25. Arthur D, Vassilvitskii S. k-means++: the advantages of careful seeding. In: Proceedings of the 18th Annual ACM-SIAM Symposium on Discrete Algorithms, New Orleans, 2007

  26. Lepetit V, Moreno-Noguer F, Fua P. EPnP: an accurate O(n) solution to the PnP problem. Int J Comput Vis, 2009, 81: 155–166

    Article  Google Scholar 

  27. Moulon P, Monasse P, Perrot R, et al. OpenMVG: open multiple view geometry. In: Proceedings of International Workshop on Reproducible Research in Pattern Recognition, 2016. 60–74

  28. Khoshelham K, Elberink S O. Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors, 2012, 12: 1437–1454

    Article  Google Scholar 

  29. Jaimez M, Kerl C, Gonzalez-Jimenez J, et al. Fast odometry and scene flow from RGB-D cameras based on geometric clustering. In: Proceedings of the IEEE International Conference on Robotics and Automation, Singapore, 2017. 3992–3999

  30. Boykov Y Y, Jolly M P. Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. In: Proceedings of the IEEE international conference on computer vision, Vancouver, 2001. 105–112

  31. Boykov Y, Kolmogorov V. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Machine Intell, 2004, 26: 1124–1137

    Article  MATH  Google Scholar 

  32. Bescos B, Facil J M, Civera J, et al. DynaSLAM: tracking, mapping, and inpainting in dynamic scenes. IEEE Robot Autom Lett, 2018, 3: 4076–4083

    Article  Google Scholar 

  33. Kim D H, Kim J H. Effective background model-based RGB-D dense visual odometry in a dynamic environment. IEEE Trans Robot, 2016, 32: 1565–1573

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by National Natural Science Foundation of China (Grant Nos. 61922076, 61873252).

Author information

Authors and Affiliations

  1. Department of Automation, University of Science and Technology of China, Hefei, 230027, China

    Chenxin Liu, Jiahu Qin, Shuai Wang & Lei Yu

  2. Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, 230088, China

    Jiahu Qin

  3. College of Electrical and Information Engineering, Hunan University, Changsha, 410082, China

    Yaonan Wang

  4. National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, 410082, China

    Yaonan Wang

Authors
  1. Chenxin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiahu Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaonan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiahu Qin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, C., Qin, J., Wang, S. et al. Accurate RGB-D SLAM in dynamic environments based on dynamic visual feature removal. Sci. China Inf. Sci. 65, 202206 (2022). https://doi.org/10.1007/s11432-021-3425-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3425-8

Keywords

Search

Navigation