Skip to main content
SpringerLink
Log in

Replay-Based Online Adaptation for Unsupervised Deep Visual Odometry

  • Conference paper
  • First Online:
Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications (CIARP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14469))

Included in the following conference series:

  • 354 Accesses

Abstract

Online adaptation is a promising paradigm that enables dynamic adaptation to new environments. In recent years, there has been a growing interest in exploring online adaptation for various problems, including visual odometry, a crucial task in robotics, autonomous systems, and driver assistance applications. In this work, we leverage experience replay, a potent technique for enhancing online adaptation, to explore the replay-based online adaptation for unsupervised deep visual odometry. Our experiments reveal a remarkable performance boost compared to the non-adapted model. Furthermore, we conduct a comparative analysis against established methods, demonstrating competitive results that showcase the potential of online adaptation in advancing visual odometry.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bian, J., et al.: Unsupervised scale-consistent depth and ego-motion learning from monocular video. In: Advances in Neural Information Processing Systems 32 (2019)

    Google Scholar 

  2. Casser, V., Pirk, S., Mahjourian, R., Angelova, A.: Depth prediction without the sensors: leveraging structure for unsupervised learning from monocular videos. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8001–8008 (2019)

    Google Scholar 

  3. Chen, Y., Tan, Y.: Self-supervised depth completion with adaptive online adaptation. In: Proceedings of the 2023 7th International Conference on Machine Learning and Soft Computing, pp. 168–174 (2023)

    Google Scholar 

  4. Chen, Y., Schmid, C., Sminchisescu, C.: Self-supervised learning with geometric constraints in monocular video: connecting flow, depth, and camera. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7063–7072 (2019)

    Google Scholar 

  5. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)

    Google Scholar 

  6. Godard, C., Mac Aodha, O., Firman, M., Brostow, G.J.: Digging into self-supervised monocular depth estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3828–3838 (2019)

    Google Scholar 

  7. Gordon, A., Li, H., Jonschkowski, R., Angelova, A.: Depth from videos in the wild: unsupervised monocular depth learning from unknown cameras. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8977–8986 (2019)

    Google Scholar 

  8. Hariat, M., Manzanera, A., Filliat, D.: Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1267–1276 (2023)

    Google Scholar 

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  10. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  11. Kuznietsov, Y., Proesmans, M., Van Gool, L.: Comoda: continuous monocular depth adaptation using past experiences. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2907–2917 (2021)

    Google Scholar 

  12. Kuznietsov, Y., Proesmans, M., Van Gool, L.: Towards unsupervised online domain adaptation for semantic segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 261–271 (2022)

    Google Scholar 

  13. Li, R., Wang, S., Long, Z., Gu, D.: Undeepvo: monocular visual odometry through unsupervised deep learning. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 7286–7291. IEEE (2018)

    Google Scholar 

  14. Li, S., Wang, X., Cao, Y., Xue, F., Yan, Z., Zha, H.: Self-supervised deep visual odometry with online adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6339–6348 (2020)

    Google Scholar 

  15. Li, S., Wu, X., Cao, Y., Zha, H.: Generalizing to the open world: Deep visual odometry with online adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13184–13193 (2021)

    Google Scholar 

  16. Li, S., Xue, F., Wang, X., Yan, Z., Zha, H.: Sequential adversarial learning for self-supervised deep visual odometry. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2851–2860 (2019)

    Google Scholar 

  17. Loo, S.Y., Shakeri, M., Tang, S.H., Mashohor, S., Zhang, H.: Online mutual adaptation of deep depth prediction and visual slam. arXiv preprint arXiv:2111.04096 (2021)

  18. Ranjan, A., et al.: Competitive collaboration: Joint unsupervised learning of depth, camera motion, optical flow and motion segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12240–12249 (2019)

    Google Scholar 

  19. Shen, T., et al.: Beyond photometric loss for self-supervised ego-motion estimation. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 6359–6365. IEEE (2019)

    Google Scholar 

  20. Tonioni, A., Rahnama, O., Joy, T., Stefano, L.D., Ajanthan, T., Torr, P.H.: Learning to adapt for stereo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9661–9670 (2019)

    Google Scholar 

  21. Tonioni, A., Tosi, F., Poggi, M., Mattoccia, S., Stefano, L.D.: Real-time self-adaptive deep stereo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 195–204 (2019)

    Google Scholar 

  22. Umeyama, S.: Least-squares estimation of transformation parameters between two point patterns. IEEE Trans. Pattern Analy. Mach. Intell. 13(04), 376–380 (1991)

    Article  Google Scholar 

  23. Vödisch, N., Petek, K., Burgard, W., Valada, A.: Codeps: online continual learning for depth estimation and panoptic segmentation. arXiv preprint arXiv:2303.10147 (2023)

  24. Wang, D., Shelhamer, E., Liu, S., Olshausen, B., Darrell, T.: Tent: fully test-time adaptation by entropy minimization. In: International Conference on Learning Representations (2020)

    Google Scholar 

  25. Wang, F., Cheng, J., Liu, P.: Cbwloss: constrained bidirectional weighted loss for self-supervised learning of depth and pose. IEEE Trans. Intell. Trans. Syst. (2023)

    Google Scholar 

  26. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  27. Yin, Z., Shi, J.: Geonet: unsupervised learning of dense depth, optical flow and camera pose. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1983–1992 (2018)

    Google Scholar 

  28. Zhan, H., Garg, R., Weerasekera, C.S., Li, K., Agarwal, H., Reid, I.: Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 340–349 (2018)

    Google Scholar 

  29. Zhang, J., Sui, W., Wang, X., Meng, W., Zhu, H., Zhang, Q.: Deep online correction for monocular visual odometry. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 14396–14402. IEEE (2021)

    Google Scholar 

  30. Zhang, Z., Lathuiliere, S., Ricci, E., Sebe, N., Yan, Y., Yang, J.: Online depth learning against forgetting in monocular videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4494–4503 (2020)

    Google Scholar 

  31. Zhong, Y., Li, H., Dai, Y.: Open-world stereo video matching with deep RNN. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11206, pp. 104–119. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01216-8_7

    Chapter  Google Scholar 

  32. Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1851–1858 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. KU Leuven, Leuven, Belgium

    Yevhen Kuznietsov, Marc Proesmans & Luc Van Gool

  2. ETH Zurich, Zurich, Switzerland

    Luc Van Gool

  3. INSAIT Sofia, Sofia, Bulgaria

    Luc Van Gool

Authors
  1. Yevhen Kuznietsov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Proesmans
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luc Van Gool
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yevhen Kuznietsov .

Editor information

Editors and Affiliations

  1. Polytechnic Institute of Coimbra, Coimbra Institute of Engineering, Coimbra, Portugal

    Verónica Vasconcelos

  2. Polytechnic Institute of Coimbra, Coimbra Institute of Engineering, Coimbra, Portugal

    Inês Domingues

  3. Polytechnic Institute of Coimbra, Coimbra Institute of Engineering, Coimbra, Portugal

    Simão Paredes

Rights and permissions

Reprints and permissions

Copyright information

© 2024 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kuznietsov, Y., Proesmans, M., Van Gool, L. (2024). Replay-Based Online Adaptation for Unsupervised Deep Visual Odometry. In: Vasconcelos, V., Domingues, I., Paredes, S. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2023. Lecture Notes in Computer Science, vol 14469. Springer, Cham. https://doi.org/10.1007/978-3-031-49018-7_48

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49018-7_48

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49017-0

  • Online ISBN: 978-3-031-49018-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation