Skip to main content
SpringerLink
Log in

Real-time SLAM relocalization with online learning of binary feature indexing

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

A visual simultaneous localization and mapping (SLAM) system usually contains a relocalization module to recover the camera pose after tracking failure. The core of this module is to establish correspondences between map points and key points in the image, which is typically achieved by local image feature matching. Since recently emerged binary features have orders of magnitudes higher extraction speed than traditional features such as scale invariant feature transform, they can be applied to develop a real-time relocalization module once an efficient method of binary feature matching is provided. In this paper, we propose such a method by indexing binary features with hashing. Being different from the popular locality sensitive hashing, the proposed method constructs the hash keys by an online learning process instead of pure randomness. Specifically, the hash keys are trained with the aim of attaining uniform hash buckets and high collision rates of matched feature pairs, which makes the method more efficient on approximate nearest neighbor search. By distributing the online learning into the simultaneous localization and mapping process, we successfully apply the method to SLAM relocalization. Experiments show that camera poses can be recovered in real time even when there are tens of thousands of landmarks in the map.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: Brief: binary robust independent elementary features. In: Proceedings of European Conference Computer Vision, pp. 778–792 (2010)

  2. Davison, A., Reid, I., Molton, N., Stasse, O.: Monoslam: real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  3. Engel, J., Schöps, T., Cremers, D.: Lsd-slam: Large-scale direct monocular slam. In: Proceedings of European Conference on Computer Vision, pp. 834–849. Springer (2014)

  4. Feng, Y., Fan, L., Wu, Y.: Online learning of binary feature indexing for real-time slam relocalization. In: Proceedings of Asian Conference on Computer Vision Workshops, pp. 206–217 (2014)

  5. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and auto cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  Google Scholar 

  6. Galvez-Lpez, D., Tardos, J.D.: Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot. 28(5), 1188–1197 (2012)

    Article  Google Scholar 

  7. Gionis, A., Indyk, P., Motwani, R.: Similarity search in high dimensions via hashing. In: Proceedings of 25th International Conference on Very Large Data Bases, pp. 518–529 (1999)

  8. Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: Proceedings of IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 225–234 (2007)

  9. Lepetit, V., Moreno-Noguer, F., Fua, P.: Epnp: an accurate o(n) solution to the pnp problem. Int. J. Comput. Vis. 81(2), 155–166 (2009)

    Article  Google Scholar 

  10. Leutenegger, S., Chli, M., Siegwart, R.: Brisk: binary robust invariant scalable keypoints. In: Proceedings of IEEE International Conference on Computer Vision, pp. 2548–2555 (2011)

  11. Lim, H., Sinha, S., Cohen, M., Uyttendaele, M.: Real-time image-based 6-dof localization in large-scale environments. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1043–1050 (2012)

  12. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  13. Muja, M., Lowe, D.G.: Fast matching of binary features. In: Conference on Computer and Robot Vision, pp. 404–410 (2012)

  14. Mur-Artal, R., Montiel, J.M.M., Tards, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  15. Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2161–2168 (2006)

  16. Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 448–461 (2010)

    Article  Google Scholar 

  17. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Proceedings of European Conference on Computer Vision, pp. 430–443 (2006)

  18. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to sift or surf. In: Proceedings of IEEE International Conference on Computer Vision, pp. 2564–2571 (2011)

  19. Sattler, T., Leibe, B., Kobbelt, L.: Fast image-based localization using direct 2d-to-3d matching. In: Proceedings of IEEE International Conference on Computer Vision, pp. 667–674 (2011)

  20. Silpa-Anan, C., Hartley, R.: Optimised kd-trees for fast image descriptor matching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  21. Straub, J., Hilsenbeck, S., Schroth, G., Huitl, R., Moller, A., Steinbach, E.: Fast relocalization for visual odometry using binary features. In: Proceedings of IEEE International Conference on Image Processing, pp. 2548–2552 (2013)

  22. Tola, E., Lepetit, V., Fua, P.: Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 815–830 (2010)

    Article  Google Scholar 

  23. Trzcinski, T., Lepetit, V., Fua, P.: Thick boundaries in binary space and their influence on nearest-neighbor search. Pattern Recogn. Lett. 33(16), 2173–2180 (2012)

    Article  Google Scholar 

  24. Williams, B., Klein, G., Reid, I.: Automatic relocalization and loop closing for real-time monocular slam. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1699–1712 (2011)

    Article  Google Scholar 

  25. Yianilos, P.: Data structures and algorithms for nearest neighbor search in general metric spaces. In: Proceedings of the fourth annual ACM-SIAM symposium on Discrete algorithms, pp. 311–321 (1993)

  26. Yunpeng, L., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3d point clouds. In: Proceedings of European Conference on Computer Vision, pp. 15–29 (2012)

Download references

Acknowledgements

This work was supported by the National High Technology Research and Development Program of China (2015AA020504), the National Natural Science Foundation of China under Grant No. 61572499, 61421004 and Nokia Research Grant No. LF14011659182.

Author information

Authors and Affiliations

  1. Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing, China

    Youji Feng

  2. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Youji Feng & Yihong Wu

  3. Nokia Technologies, Tampere, Finland

    Lixin Fan

Authors
  1. Youji Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yihong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lixin Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yihong Wu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, Y., Wu, Y. & Fan, L. Real-time SLAM relocalization with online learning of binary feature indexing. Machine Vision and Applications 28, 953–963 (2017). https://doi.org/10.1007/s00138-017-0873-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-017-0873-z

Keywords

Search

Navigation