Skip to main content
SpringerLink
Log in

IQ-VIO: adaptive visual inertial odometry via interference quantization under dynamic environments

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Vision-based localization is susceptible to interference from dynamic objects in the environment, resulting in decreased localization accuracy and even tracking loss. Hence, sensor fusion with IMUs or motor encoders has been widely adopted to improve positioning accuracy and robustness in dynamic environments. Commonly used loose coupling fusion localization methods cannot completely eliminate the error introduced by dynamic objects. In this paper, we propose a novel adaptive visual inertial odometry via interference quantization, namely IQ-VIO. To quantify the confidence of pose estimation through vision frames analysis, we firstly introduce the feature coverage and the dynamic scene interference index based on image information entropy. Then, based on the interference index, we further establish the IQ-VIO multi-sensor fusion model, which can adaptively adjust the measurement error covariance matrix of an extended Kalman filter to suppress and eliminate the impact of dynamic objects on localization. We verify IQ-VIO algorithm on KAIST Urban dataset and actual scenes. Results show that our method achieves favorable performance against other algorithms. Especially under challenging scenes such as low texture, the RPE of our algorithm decreases at least twenty percent. Our approach can effectively eliminate the impact of dynamic objects in the scenes and obtain higher positioning accuracy and robustness than conventional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

The authors make sure that all data and materials as well as software application or custom code support their published claims and comply with field standards.

References

  1. Mur-Artal R, Tardós JD (2017) Orb-slam2: an open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Trans Rob 33(5):1255–1262. https://doi.org/10.1109/TRO.2017.2705103

    Article  Google Scholar 

  2. van Goor P, Mahony R (2021) An equivariant filter for visual inertial odometry. In: 2021 IEEE international conference on robotics and automation (ICRA). IEEE, pp 14432–14438. https://doi.org/10.1109/ICRA48506.2021.9561769

  3. Shao X, Zhang L, Zhang T, Shen Y, Li H, Zhou Y (2020) A tightly-coupled semantic SLAM system with visual, inertial and surround-view sensors for autonomous indoor parking. In: Proceedings of the 28th ACM international conference on multimedia, pp 2691–2699. https://doi.org/10.1145/3394171.3413867

  4. Song S, Lim H, Lee AJ, Myung H (2022) DynaVINS: a visual-inertial SLAM for dynamic environments. IEEE Robot Autom Lett 7(4):11523–11530. https://doi.org/10.1109/LRA.2022.3203231

    Article  Google Scholar 

  5. Bescos B, Fácil JM, Civera J, Neira J (2018) DynaSLAM: tracking, mapping, and inpainting in dynamic scenes. IEEE Robot Autom Lett 3(4):4076–4083. https://doi.org/10.1109/LRA.2018.2860039

    Article  Google Scholar 

  6. Zou D, Tan P (2012) Coslam: collaborative visual slam in dynamic environments. IEEE Trans Pattern Anal Mach Intell 35(2):354–366. https://doi.org/10.1109/TPAMI.2012.104

    Article  Google Scholar 

  7. Su P, Luo S, Huang X (2022) Real-time dynamic SLAM algorithm based on deep learning. IEEE Access 10:87754–87766. https://doi.org/10.1109/ACCESS.2022.3199350

    Article  Google Scholar 

  8. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, vol 30. Cambridge University Press, Cambridge, pp 1333–1341

    MATH  Google Scholar 

  9. Tan W, Liu H, Dong Z, Zhang G, Bao H (2013) Robust monocular SLAM in dynamic environments. In: 2013 IEEE international symposium on mixed and augmented reality (ISMAR), pp 209–218. https://doi.org/10.1109/ISMAR.2013.6671781

  10. Alcantarilla PF, Yebes JJ, Almazán J, Bergasa LM (2012) On combining visual SLAM and dense scene flow to increase the robustness of localization and mapping in dynamic environments. In: 2012 IEEE international conference on robotics and automation, pp 1290–1297. https://doi.org/10.1109/ICRA.2012.6224690

  11. Li S, Lee D (2017) RGB-D SLAM in dynamic environments using static point weighting. IEEE Robot Autom Lett 2(4):2263–2270. https://doi.org/10.1109/LRA.2017.2724759

    Article  Google Scholar 

  12. Atashgah MAA, Malaek SMB (2011) A simulation environment for path and image generation in an aerial single-camera vision system. Proc Inst Mech Eng Part G J Aerosp Eng 225(5):541–558. https://doi.org/10.1177/0954410012450107

    Article  Google Scholar 

  13. Atashgah MAA, Malaek SMB (2013) Prediction of aerial-image motion blurs due to the flying vehicle dynamics and camera characteristics in a virtual environment. Proc Inst Mech Eng Part G J Aerosp Eng 227(7):1055–1067. https://doi.org/10.1243/09544100JAERO813

    Article  Google Scholar 

  14. Derome M, Plyer A, Sanfourche M, Besnerais GLJUS (2015) Moving object detection in real-time using stereo from a mobile platform. Unmanned Syst 3(04):253–266. https://doi.org/10.1142/S2301385015400026

    Article  Google Scholar 

  15. Wang Y, Huang S (2014) Towards dense moving object segmentation based robust dense RGB-D SLAM in dynamic scenarios. In: 13th international conference on control automation robotics and vision (ICARCV), pp 1841–1846. https://doi.org/10.1109/ICARCV.2014.7064596

  16. Kerl C, Sturm J, Cremers D (2013) Dense visual SLAM for RGB-D cameras. In: 2013 IEEE/RSJ international conference on intelligent robots and systems, pp 2100–2106. https://doi.org/10.1109/IROS.2013.6696650

  17. Alcantarilla PF, Yebes JJ, Almazán J, Bergasa LM (2012) On combining visual SLAM and dense scene flow to increase the robustness of localization and mapping in dynamic environments. In: 2012 IEEE international conference on robotics and automation, pp 1290–1297. https://doi.org/10.1109/ICRA.2012.6224690

  18. Yu C et al. (2018) DS-SLAM: a semantic visual SLAM towards dynamic environments. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1168–1174. https://doi.org/10.1109/IROS.2018.8593691

  19. Zhong F, Wang S, Zhang Z, Chen C, Wang Y (2018) Detect-SLAM: making object detection and SLAM mutually beneficial. In: 2018 IEEE winter conference on applications of computer vision (WACV), pp 1001–1010. https://doi.org/10.1109/WACV.2018.00115

  20. Xiao L, Wang J, Qiu X, Rong Z, Zou XJR, Systems A (2019) Dynamic-SLAM: semantic monocular visual localization and mapping based on deep learning in dynamic environment. Robot Auton Syst 117:1–16. https://doi.org/10.1016/j.robot.2019.03.012

    Article  Google Scholar 

  21. Hu Z, Zhao J, Luo Y, Ou J (2022) Semantic SLAM based on improved DeepLabv3\(^+\) in dynamic scenarios. IEEE Access 10:21160–21168. https://doi.org/10.1109/ACCESS.2022.3154086

    Article  Google Scholar 

  22. Ballester I, Fontán A, Civera J, Strobl KH, Triebel R (2021) DOT: dynamic object tracking for visual SLAM. In: 2021 IEEE international conference on robotics and automation (ICRA). IEEE, pp 11705–11711, https://doi.org/10.1109/ICRA48506.2021.9561452

  23. Yin Z, Shi J (2018) GeoNet: unsupervised learning of dense depth optical flow and camera pose. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1983–1992

  24. Wang S, Clark R, Wen H, Trigoni N (2017) DeepVO: towards end-to-end visual odometry with deep recurrent convolutional neural networks. In: 2017 IEEE international conference on robotics and automation (ICRA), pp 2043–2050. https://doi.org/10.1109/ICRA.2017.7989236

  25. Labbe M, Michaud F (2019) RTAB-Map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. J Field Robot 36(2):416–446. https://doi.org/10.1002/rob.21831

    Article  Google Scholar 

  26. Qin T, Li P, Shen S (2018) VINS-mono: a robust and versatile monocular visual-inertial state estimator. IEEE Trans Rob 34(4):1004–1020. https://doi.org/10.1109/TRO.2018.2853729

    Article  Google Scholar 

  27. Babu BPW, Cyganski D, Duckworth J, Kim S (2018) Detection and resolution of motion conflict in visual inertial odometry. In: 2018 IEEE international conference on robotics and automation (ICRA), pp 996–1002. https://doi.org/10.1109/ICRA.2018.8460870

  28. Minoda K, Schilling F, Wüest V, Floreano D, Yairi T (2021) Viode: a simulated dataset to address the challenges of visual-inertial odometry in dynamic environments. IEEE Robot Autom Lett 6(2):1343–1350. https://doi.org/10.1109/LRA.2021.3058073

    Article  Google Scholar 

  29. Wu J, Xiong J, Guo H (2022) Improving robustness of line features for VIO in dynamic scene. Meas Sci Technol 33(6):065204. https://doi.org/10.1088/1361-6501/ac547f

  30. Wei H, Zhang T, Zhang L (2021) GMSK-SLAM: a new RGB-D SLAM method with dynamic areas detection towards dynamic environments. Multimed Tools Appl 80(21):31729–31751. https://doi.org/10.1007/s11042-021-11168-5

    Article  Google Scholar 

  31. Huang C, Lin H, Lin H, Liu H, Gao Z, Huang L (2021) YO-VIO: robust multi-sensor semantic fusion localization in dynamic indoor environments. In: 2021 international conference on indoor positioning and indoor navigation (IPIN). IEEE, pp 1–6. https://doi.org/10.1109/IPIN51156.2021.9662614

  32. Yao J, Zhang P, Wang Y, Luo Z, Ren X (2019) An adaptive uniform distribution ORB based on improved quadtree. IEEE Access 7:143471–143478. https://doi.org/10.1109/ACCESS.2019.2940995

    Article  Google Scholar 

  33. Hero AO, Ma B, Michel OJ, Gorman J (2002) Applications of entropic spanning graphs. IEEE Signal Process Mag 19(5):85–95. https://doi.org/10.1109/MSP.2002.1028355

    Article  Google Scholar 

  34. Hero AO, Ma B, Michel O, Gorman J (2001) Alpha-divergence for classification, indexing and retrieval. Communications and Signal Processing Laboratory (CSPL), Department of EECS, University of Michigan, Ann Arbor, Technical Report 328. http://www.eecs.umich.edu/ hero/detest.html

  35. Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  36. Shan T, Englot B, Meyers D, Wang W, Ratti C, Rus D (2020) Lio-sam: tightly-coupled lidar inertial odometry via smoothing and mapping. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 5135–5142. https://doi.org/10.1109/IROS45743.2020.9341176

  37. Jeong J, Cho Y, Shin YS, Roh H, Kim A (2018) Complex urban lidar data set. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE, pp 6344–6351. https://doi.org/10.1109/ICRA.2018.8460834

Download references

Funding

This work was supported by Department of science and technology of Guangdong Province (No:2021B01420003).

Author information

Author notes

    Authors and Affiliations

    1. School of Mechanical and Automotive Engineering, South China University of Technology, Tianhe, Guangzhou, 510641, Guangdong, China

      Huikun Zhang, Feng Ye, Yizong Lai, Kuo Li & Jinze Xu

    Authors
    1. Huikun Zhang
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Feng Ye
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Yizong Lai
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Kuo Li
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Jinze Xu
      View author publications

      You can also search for this author in PubMed Google Scholar

    Contributions

    All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by HZ, KL and JX. The first draft of the manuscript was written by HZ and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

    Corresponding author

    Correspondence to Feng Ye.

    Ethics declarations

    Conflict of interest

    The authors have no relevant financial or non-financial interests to disclose.

    Ethical statement and informed consent

    Informed consent was obtained from all individual participants included in the study.

    Additional information

    Publisher's Note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Zhang, H., Ye, F., Lai, Y. et al. IQ-VIO: adaptive visual inertial odometry via interference quantization under dynamic environments. Intel Serv Robotics 16, 565–581 (2023). https://doi.org/10.1007/s11370-023-00478-2

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11370-023-00478-2

    Keywords

    Search

    Navigation