Skip to main content
SpringerLink
Log in

Continual Learning for Multi-camera Relocalisation

  • Conference paper
  • First Online:
Advances in Computational Intelligence (MICAI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13067))

Included in the following conference series:

Abstract

Visual relocalisation is a well-known problem in the robotics community, where chromatic images are used to recognise a place that is being re-visited or re-observed again. Due to the success of deep neural networks in several computer vision tasks, convolutional neural networks have been proposed to address the visual relocalisation problem as well. However, these solutions follow the conventional off-line training in order to generate a model that can be used to regress a camera’s pose w.r.t to an input image. In this work, we present a methodology based on continual learning to address the visual relocalisation problem aiming at performing on-line model training, seeking to generate a model that is updated continuously to learn new acquired images associated with GPS coordinates. Moreover, we apply this methodology to the multi-camera case, where 8 images are acquired from a multi-rig camera, seeking to improve the localisation accuracy, this is, by using a multi-camera, we obtain a set of images observing different viewpoints of the scene for a given GPS position. Therefore, by using a voting scheme, our on-line learned model is capable of performing visual relocalisation with an accuracy of 0.78, performing at 50 fps.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Assa, A., Janabi-Sharifi, F.: Virtual visual servoing for multicamera pose estimation. IEEE/ASME Trans. Mechatron. 20(2), 789–798 (2014)

    Article  Google Scholar 

  2. Blanton, H., Greenwell, C., Workman, S., Jacobs, N.: Extending absolute pose regression to multiple scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 38–39 (2020)

    Google Scholar 

  3. Bresson, G., Yu, L., Joly, C., Moutarde, F.: Urban localization with street views using a convolutional neural network for end-to-end camera pose regression. In: 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 1199–1204. IEEE (2019)

    Google Scholar 

  4. Cabrera-Ponce, A.A., Martinez-Carranza, J.: Aerial geo-localisation for MAVs using PoseNet. In: 2019 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED UAS), pp. 192–198. IEEE (2019)

    Google Scholar 

  5. Cai, Z., Yu, C., Zhang, J., Ren, J., Zhao, H.: Leveraging localization for multi-camera association. arXiv preprint arXiv:2008.02992 (2020)

  6. Charco, J.L., Sappa, A.D., Vintimilla, B.X., Velesaca, H.O.: Camera pose estimation in multi-view environments: from virtual scenarios to the real world. Image Vis. Comput. 110, 104182 (2021)

    Article  Google Scholar 

  7. Charco, J.L., Vintimilla, B.X., Sappa, A.D.: Deep learning based camera pose estimation in multi-view environment. In: 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 224–228. IEEE (2018)

    Google Scholar 

  8. Clark, R., Wang, S., Markham, A., Trigoni, N., Wen, H.: VidLoc: a deep spatio-temporal model for 6-DoF video-clip relocalization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6856–6864 (2017)

    Google Scholar 

  9. Frahm, J.-M., Köser, K., Koch, R.: Pose estimation for multi-camera systems. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 286–293. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28649-3_35

    Chapter  Google Scholar 

  10. Geppert, M., Liu, P., Cui, Z., Pollefeys, M., Sattler, T.: Efficient 2d-3d matching for multi-camera visual localization. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 5972–5978. IEEE (2019)

    Google Scholar 

  11. Grenzdörffer, T., Günther, M., Hertzberg, J.: YCB-M: a multi-camera RGB-D dataset for object recognition and 6dof pose estimation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3650–3656. IEEE (2020)

    Google Scholar 

  12. Harmat, A., Trentini, M., Sharf, I.: Multi-camera tracking and mapping for unmanned aerial vehicles in unstructured environments. J. Intell. Robot. Syst. 78(2), 291–317 (2015)

    Article  Google Scholar 

  13. Hee Lee, G., Pollefeys, M., Fraundorfer, F.: Relative pose estimation for a multi-camera system with known vertical direction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 540–547 (2014)

    Google Scholar 

  14. Kasyanov, A., Engelmann, F., Stückler, J., Leibe, B.: Keyframe-based visual-inertial online slam with relocalization. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6662–6669. IEEE (2017)

    Google Scholar 

  15. Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)

    Google Scholar 

  16. Le, T., Gjevestad, J.G.O., From, P.J.: Online 3D mapping and localization system for agricultural robots. IFAC-PapersOnLine 52(30), 167–172 (2019)

    Article  Google Scholar 

  17. Lee, G.H., Li, B., Pollefeys, M., Fraundorfer, F.: Minimal solutions for the multi-camera pose estimation problem. Int. J. Robot. Res. 34(7), 837–848 (2015)

    Article  Google Scholar 

  18. Muñoz-Salinas, R., Yeguas-Bolivar, E., Saffiotti, A., Medina-Carnicer, R.: Multi-camera head pose estimation. Mach. Vis. Appl. 23(3), 479–490 (2012)

    Article  Google Scholar 

  19. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  20. Ott, F., Feigl, T., Loffler, C., Mutschler, C.: ViPR: visual-odometry-aided pose regression for 6DoF camera localization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 42–43 (2020)

    Google Scholar 

  21. Pellegrini, L., Graffieti, G., Lomonaco, V., Maltoni, D.: Latent replay for real-time continual learning. arXiv preprint arXiv:1912.01100 (2019)

  22. Remelli, E., Han, S., Honari, S., Fua, P., Wang, R.: Lightweight multi-view 3D pose estimation through camera-disentangled representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6040–6049 (2020)

    Google Scholar 

  23. Sewtz, M., Luo, X., Landgraf, J., Bodenmüller, T., Triebel, R.: Robust approaches for localization on multi-camera systems in dynamic environments. In: 2021 7th International Conference on Automation, Robotics and Applications (ICARA), pp. 211–215. IEEE (2021)

    Google Scholar 

  24. Valada, A., Radwan, N., Burgard, W.: Deep auxiliary learning for visual localization and odometry. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6939–6946. IEEE (2018)

    Google Scholar 

  25. Xu, Y., Roy, V., Kitani, K.: Estimating 3D camera pose from 2D pedestrian trajectories. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2568–2577. IEEE (2020)

    Google Scholar 

  26. Yang, A.J., Cui, C., Bârsan, I.A., Urtasun, R., Wang, S.: Asynchronous multi-view slam. arXiv preprint arXiv:2101.06562 (2021)

  27. Yang, S., Yi, X., Wang, Z., Wang, Y., Yang, X.: Visual slam using multiple RGB-D cameras. In: 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1389–1395. IEEE (2015)

    Google Scholar 

  28. Yang, Y., Tang, D., Wang, D., Song, W., Wang, J., Fu, M.: Multi-camera visual slam for off-road navigation. Robot. Auton. Syst. 128, 103505 (2020)

    Article  Google Scholar 

  29. Yogamani, S., et al.: WoodScape: a multi-task, multi-camera fisheye dataset for autonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9308–9318 (2019)

    Google Scholar 

  30. Zhang, R., Luo, Z., Dhanjal, S., Schmotzer, C., Hasija, S.: PoseNet++: a CNN framework for online pose regression and robot re-localization (2018)

    Google Scholar 

Download references

Acknowledgments

The first author is thankful for her scholarship funded by Consejo Nacional de Ciencia y Tecnología (CONACYT) under the grant 727018.

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Benemérita Universidad Autónoma de Puebla (BUAP), 72592, Puebla, Mexico

    Aldrich A. Cabrera-Ponce & Manuel Martin-Ortiz

  2. Department of Computational Science, Instituto Nacional de Astrofísica, Óptica y Electrónica (INAOE), 72840, Puebla, Mexico

    J. Martinez-Carranza

  3. Department of Computer Science, University of Bristol, Bristol, BS8 1UB, UK

    J. Martinez-Carranza

Authors
  1. Aldrich A. Cabrera-Ponce
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manuel Martin-Ortiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Martinez-Carranza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aldrich A. Cabrera-Ponce .

Editor information

Editors and Affiliations

  1. Instituto Politécnico Nacional, Centro de Investigación en Computación, Mexico City, Mexico

    Ildar Batyrshin

  2. Instituto Politécnico Nacional, Centro de Investigación en Computación, Mexico City, Mexico

    Alexander Gelbukh

  3. Instituto Politécnico Nacional, Centro de Investigación en Computación, Mexico City, Mexico

    Grigori Sidorov

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cabrera-Ponce, A.A., Martin-Ortiz, M., Martinez-Carranza, J. (2021). Continual Learning for Multi-camera Relocalisation. In: Batyrshin, I., Gelbukh, A., Sidorov, G. (eds) Advances in Computational Intelligence. MICAI 2021. Lecture Notes in Computer Science(), vol 13067. Springer, Cham. https://doi.org/10.1007/978-3-030-89817-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89817-5_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89816-8

  • Online ISBN: 978-3-030-89817-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation