Skip to main content
SpringerLink
Log in

Fast stereo visual odometry based on LK optical flow and ORB-SLAM2

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

A stereo visual odometry algorithm based on the fusion of optical flow tracking and feature matching called LK-ORB-SLAM2 was proposed. In LK-ORB-SLAM2, the operation of optical flow tracking is introduced to adjust the intensive and time-consuming operation of feature matching. This requires solving a key issue: how to solve the problem of losing feature points during optical flow tracking. For this reason, an adaptive matching-frame insertion scheme is proposed to stop optical flow tracking in time and inserts matching-frames and detect new feature points at the right time to keep LK-ORB-SLAM2 running. The experiment on the KITTI and EuRoC data set showed that LK-ORB-SLAM2 reduced the average processing time per frame of ORB-SLAM2 by about 70%, with the change of less than 2% in its accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Wu, T., Zhao, J.Y., Zhang, Z.L., Lu, Z.Y., Chang, Z.J.: On current status and development tendency of vehicle visual odometer. Electron Optics Control 24(10), 69–74 (2017)

    Google Scholar 

  2. Lu, W., Xiang, Z., Liu, J.: High-performance visual odometry with two-stage local binocular BA and GPU. In: Presented at the 2013 IEEE Intelligent Vehicles Symposium (IV), 23–26 June 2013

  3. Scaramuzza, D., Fraundorfer, F.: Visual Odometry [Tutorial] Part I: The first 30 Years and Fundamentals. IEEE Robot. Automat. Mag. 18, 80–92 (2011)

    Article  Google Scholar 

  4. Nicola, K., David, D., Sebastian, H., Sven, B.: Feature-based visual odometry prior for real-time semi-dense stereo SLAM. Robot. Auton. Syst. 109, 38–58 (2018)

    Article  Google Scholar 

  5. Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D Cameras. IEEE Trans. Rob. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  6. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: “ORB-SLAM: a Versatile and Accurate Monocular SLAM System. IEEE Trans. Rob. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  7. Gao, X., Zhang, T., Liu, Y., Yan, Q.: 14 Lectures on Visual SLAM: From Theory to Practice. Publishing House of Electronics Industry (2017)

  8. Geiger A.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Presented at the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 16–21 June 2012

  9. Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M.W., Siegwart, R.: The EuRoC micro aerial vehicle datasets. Int. J. Robot. Res. 35(10), 1157–1163 (2016)

    Article  Google Scholar 

  10. Nister, D., Naroditsky, O., Bergen, J.R.: Visual odometry. In: Presented at the 2004 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1 Jan 2004

  11. Davison, A.J.: SLAM with a single camera. In: Presented at the Proceedings of Workshop on Concurrent Mapping an Localization for Autonomous Mobile Robots in Conjunction with ICRA (2002)

  12. Davison, A.J.: Real-time simultaneous localization and mapping with a single camera. In: presented at the Proceedings of the Ninth IEEE International Conference on Computer Vision (2003)

  13. Davison, A.J., Reid, I.D., Molton, N.D.: Mono-SLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  14. Civera, J., Davison, A.J., Montiel, J.M.M.: Inversedepth parametrization for monocular SLAM. IEEE Trans. Rob. 24(5), 932–945 (2008)

    Article  Google Scholar 

  15. Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: IEEE and ACM International Symposium on Mixed and Augmented Reality (2007)

  16. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  17. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: Presented at the International Conference on Computer Vision, 1 Nov. 2012

  18. Zheng, C., Xiang, Z., Liu, J.: Monocular vision odometry based on the fusion of optical flow and feature points matching. J. Zhejiang Univ. (Eng. Sci.) 48(2), 279–284 (2014)

    Google Scholar 

  19. Cheng, C., Hao, X., Zhang, Z., Zhao, M.: Stereo visual odometry based on Kalman fusion of optical flow tracking and trifocal tensor constraint. J. Chin. Inertial Technol. 24(2), 473–479 (2016)

    Google Scholar 

  20. Chen, Y.J., Yang, G.L., Jiang, Y.X., Liu, X.Y.: Monocular visual odometry based on trifocal tensor constraint. J Phys Conf Ser 976, 012002 (2018)

    Article  Google Scholar 

  21. Kitt, B., Geiger, A., Lategahn, H.: Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme. In: Presented at IEEE Intelligent Vehicles Symposium, pp. 486–492 (2010)

  22. Forster, C., Zhang, Z., Gassner, M., Werlberger, M., Scaramuzza, D.: Svo: semidirect visual odometry for monocular and multicamera systems. IEEE Trans. Robot. 33(2), 249–265 (2017)

    Article  Google Scholar 

  23. Wang, R., Schwörer, M., Cremers, D.: Stereo dso: large-scale direct sparse Visual Odometry with stereo cameras. In: Presented at the 2017 IEEE International Conference on Computer Vision (ICCV) (2017)

  24. Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional net-work for real-time 6-DOF camera relocalization. In: IEEE International Conference on Computer Vision (ICCV), pp. 2938–2946 (2015)

  25. Muller, P., Savakis, A.: Flowdometry: an optical flow and deep learning based approach to visual odometry. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 624-631 (2017)

  26. Zhang, J., Su, Q., Liu, P., Xu, C., Chen, Y.: Unsupervised learning of monocular depth and large-ego-motion with multiple loop consistency losses. IEEE Access (2019). https://doi.org/10.1109/ACCESS.2019.2920301

    Article  Google Scholar 

  27. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst 110(3), 346–359 (2008)

    Article  Google Scholar 

  28. Rosten, E., Drummond, T.: Machine Learning for High-Speed Corner Detection. Springer, Berlin (2006)

    Book  Google Scholar 

  29. Calonder, M, Lepetit, V., Strecha, C., Pascal, F.: BRIEF: binary robust independent elementary features. In: Presented at 11th European Conference on Computer Vision, Heraklion, Crete, Greece, 5–11 Sep 2010

  30. Jürgen, S., et al.: A benchmark for the evaluation of RGB-D SLAM systems. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems IEEE (2012)

  31. Strasdat, H., Montiel, J.M.M., Davison, A.J.: Visual SLAM: why filter? ☆. Image Vis. Comput. 30(2), 65–77 (2012)

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grant #61603158, the Senior Talent Fund Project of Jiangsu University under Grant #16JDG067 and the Six Talent Peaks Project in Jiangsu Province under Grant #2016-JXQC-007.

Author information

Authors and Affiliations

  1. Automotive Engineering Research Institute, Jiangsu University, Zhenjiang, 212013, China

    Chuanye Tang, Xinwen Zhao, Jianfeng Chen, Long Chen & Yazhou Zhou

Authors
  1. Chuanye Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinwen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianfeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Long Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yazhou Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinwen Zhao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, C., Zhao, X., Chen, J. et al. Fast stereo visual odometry based on LK optical flow and ORB-SLAM2. Multimedia Systems 28, 2005–2014 (2022). https://doi.org/10.1007/s00530-020-00662-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-020-00662-9

Keywords

Search

Navigation