Skip to main content
Log in

Multi-resolution map building and loop closure with omnidirectional images

Autonomous Robots Aims and scope Submit manuscript

Abstract

A topological mapping approach for omnidirectional images capable of answering loop closure queries at multiple resolutions is presented. The environment is mapped hierarchically using two layers. The first layer consists of individual images and the second layer represents regions of the environment composed of groups of images from the first layer. A hierarchical algorithm is formulated that exploits this map structure for an efficient and accurate loop closure without the need of geometric verification. The vital parameters of loop closure are automatically learned from training data. Performance of our loop closure algorithm is experimentally evaluated on various publicly available datasets and compared to two state of the art techniques. The results show that agreeable performance is achieved even on low quality datasets without the need for geometric verification of loop closures common among many contemporary approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. http://hemanthk.me/wordpress/datasets/.

  2. http://ipds.univ-bpclermont.fr/.

  3. http://www.robots.ox.ac.uk/NewCollegeData/

  4. http://www.rawseeds.org/rs/datasets/view//6.

  5. http://www.robots.ox.ac.uk/~mjc/Software.htm.

  6. https://github.com/dorian3d/DLoopDetector.

References

  • Aggarwal, C. C., & Chandan, K. R. (2013). Data clustering: Algorithms and applications. Chapman & Hall/CRC, Data mining and knowledge discovery series (Book 31).

  • Aihara, N., Iwasa, H., Yokoya, N., & Takemura, H. (1998). Memory-based self-localization using omnidirectional images. In Proceedings of international conference on pattern recognition (ICPR) (Vol. 2, pp. 1799–1803).

  • Anati, R. & Daniilidis, K. (2009). Constructing topological maps using markov random fields and loop-closure detection. In Advances in Neural Information Processing Systems, 37–45.

  • Angeli, A., Doncieux, S., Meyer, J.-A., & Filliat, D. (2009). Visual topological slam and global localization. In Proceedings of IEEE international conference on robotics and automation (ICRA) (pp. 4300–4305).

  • Angeli, A., Filliat, D., Doncieux, S., & Meyer, J.-A. (2008). Fast and incremental method for loop-closure detection using bags of visual words. IEEE Transactions on Robotics, 24, 1027–1037.

    Article  Google Scholar 

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2008). SURF: Speeded up robust features. Computer Vision and Image Understanding (CVIU), 110(3), 346–359.

    Article  Google Scholar 

  • Bhattacharyya, A. (1943). On a measure of divergence between two statistical populations defined by their probability distributions. Bulletin of the Calcutta Mathematical Society, 35, 99–109.

    MathSciNet  MATH  Google Scholar 

  • Cadena, C., Galvez-Lopez, D., Ramos, F., Tardos, J.D., & Neira, J. (2010). Robust place recognition with stereo cameras. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 5182–5189)

  • Calonder, M., Lepetit, V., Strecha, C., & Fua, P. (2010). BRIEF: Binary robust independent elementary features. In 11th European conference on computer vision (ECCV).

  • Calonder, M., Lepetit, V., Ozuysal, M., Trzcinski, T., Strecha, Christoph, & Fua, Pascal. (2012). Brief: Computing a local binary descriptor very fast. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7), 1281–1298.

    Article  Google Scholar 

  • Cummins, M. & Newman, P. (2009). Highly scalable appearance-only SLAM - FAB-MAP 2.0. In Proceedings of robotics: Science and systems (RSS). USA.

  • Cummins, M. & Newman, P. (2010). Appearance-only SLAM at large scale with FAB-MAP 2.0, The International Journal of Robotics Research.

  • Cummins, M., & Newman, P. (2008). FAB-MAP: Probabilistic localization and mapping in the space of appearance. The International Journal of Robotics Research (IJRR), 24, 647–665.

    Article  Google Scholar 

  • Douze, M., Jégou, H., Sandhawalia, H., Amsaleg, L., & Schmid, C. (2009). Evaluation of gist descriptors for web-scale image search. In Proceedings of the ACM international conference on image and video retrieval (CIVR). New York.

  • Dudek, G., & Jugessur, D. (2000). Robust place recognition using local appearance based methods. In Proceedings of IEEE international conference on robotics and automation (ICRA) (Vol.2, pp. 1030–1035).

  • Ferreira, F., Santos, V., & Dias, J. (2006). Integration of multiple sensors using binary features in a bernoulli mixture model. In Proceedings of IEEE international conference on multi-sensor fusion and integration for intelligent systems. (pp. 104–109)

  • Filliat, D. (2007). A visual bag of words method for interactive qualitative localization and mapping. In Proceedings of IEEE international conference on robotics and automation (ICRA) (pp. 3921–3926).

  • Fraundorfer, F., Engels, C., & Nistér, D. (2007). Topological mapping,localization and navigation using image collections. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3872–3877).

  • Galvez-Lopez, D., & Tardos, J.D. (2011). Real-time loop detection with bags of binary words. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 51–58).

  • Hartigan, J. A., & Wong, M. A. (1979). A K-means clustering algorithm. Applied Statistics, 28, 100–108.

    Article  MATH  Google Scholar 

  • Ho, K., & Newman, P. (2005). Multiple map intersection detection using visual appearance. In International conference on computational intelligence, robotics and autonomous systems.

  • Ho, K., & Newman, P. (2005). SLAM-loop closing with visually salient features. In Proceedings of IEEE international conference on robotics and automation (ICRA).

  • Ho, K. L., & Newman, P. (2007). Detecting loop closure with scene sequences. International Journal of Computer Vision (IJCV), 74(3), 261–286.

    Article  Google Scholar 

  • Jeffreys, H. (1948). Theory of probability. Oxford: Oxford University Press.

    MATH  Google Scholar 

  • Jegou, H., Douze, M., Schmid, C., & Perez, P. (2010). Aggregating local descriptors into a compact image representation. In IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3301–3311).

  • Jgou, H., Perronnin, F., Douze, M., Sanchez, J., Prez, P., & Schmid, C. (2012). Aggregating local images descriptors into compact codes. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 34, 1704–1716.

    Article  Google Scholar 

  • Jogan, M., & Leonardis, A. (1999). Panoramic eigen images for spatial localisation. In Proceedings of the 8th international conference on computer analysis of images and patterns (CAIP) (pp. 558-567). London, UK.

  • Koller, D., & Friedman, N. (2009). Probabilistic graphical models. Cambridge: The MIT Press.

    MATH  Google Scholar 

  • Korrapati, H., Courbon, J., Mezouar, Y., & Martinet, P. (2012). Image Sequence Partitioning for Outdoor Mapping. In International conference on robotics and automation (ICRA).

  • Korrapati, H., Uzer, F., & Mezouar, Y. (2013). Hierarchical visual mapping with omnidirectional images. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3684–3690).

  • Korrapati, P., & Mezouar, Y. (2014). Vision-based sparsetopological mapping. Robotics and Autonomous Systems (RAS), 62, 1259–1270.

    Article  Google Scholar 

  • Koseck, J., Li, F., & Yang, X. (2005). Global localization and relative positioning based on scale-invariant keypoints. Robotics and Autonomous Systems, 52(1), 27–38.

    Article  Google Scholar 

  • Kröse, B. J. A., Vlassis, N., Bunschoten, R., & Motomura, Y. (2001). A probabilistic model for appearance-based robot localization. Image and Vision Computing, 19(6), 381–391.

    Article  Google Scholar 

  • Li, F. & Kosecká, J. (2006). Probabilistic location recognition using reduced feature set. In Proceedings of IEEE international conference on robotics and automation (ICRA). (pp. 3405–3410)

  • Liu, M. & Siegwart, R. (2012). DP-FACT: Towards topological mapping and scene recognition with color for omnidirectional camera. In IEEE international conference on robotics and automation (ICRA).

  • Liu, Y. & Zhang, H. (2012). Visual loop closure detection with a compact image descriptor. In IEEE/RSJ international conference on in intelligent robots and systems (IROS) (pp. 1051–1056)

  • Liu, M., Scaramuzza, D., Pradalier, C., Siegwart, R., Chen, Q. (2009). Scene recognition with omnidirectional vision for topological map using lightweight adaptive descriptors. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 116–121).

  • Lowe, D. G. (2004). Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision (IJCV), 60, 91–110.

    Article  Google Scholar 

  • Mahalanobis, P. C. (1936). On the generalised distance in statistics. Proceedings of the National Institute of Sciences of India, 2(1), 49–55.

    MathSciNet  MATH  Google Scholar 

  • Majdik, A., Galvez-Lopez, D., Lazea, G., & Castellanos, J.A. (2011). Adaptive appearance based loop-closing in heterogeneous environments. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1256–1263)

  • Menegatti, E., Maeda, T., & Ishiguro, H. (2004). Image-based memory for robot navigation using properties of omnidirectional images. Robotics and Autonomous Systems (RAS), 47(4), 251–267.

    Article  Google Scholar 

  • Murillo, C., Campos, P., Kosecka, J., & Guerrero, J. (2010). Gist vocabularies in omnidirectional images for appearance based mapping and localization. In Proceedings of the 10th IEEE workshop on omnidirectional vision,camera networks and non-classical cameras (OMNIVIS), held with robotics: Science and systems.

  • Murillo, A. C., Sagus, C., Guerrero, J. J., Goedem, T., Tuyte-laars, T., & Van Gool, L. (2007). From omnidirectional images to hierarchical localization. Robotics and Autonomous Systems, 55, 372–382.

    Article  Google Scholar 

  • Murillo, A. C., Singh, G., Kosecká, J., & Guerrero, J. J. (2013). Localization in urban environments using a panoramic gist descriptor. IEEE Transactions on Robotics, 29(1), 2013.

    Article  Google Scholar 

  • Newman, P., Cole, D., & Ho, K. (2006). Outdoor slam using visual appearance and laser ranging. In Proceedings of IEEE international conference on robotics and automation (ICRA) (pp. 1180-1187).

  • Nistér, D. & Stewenius, H. (2006). Scalable recognition with a vocabulary tree. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 2161–2168). USA.

  • Nourani-Vatani, N., & Pradalier, C. (2010). Scene change detection for vision-based topological mapping and localization. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3792–3797.

  • Oliva, A., & Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision (IJCV), 42(3), 145–175.

    Article  MATH  Google Scholar 

  • Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2008). Lost in quantization: Improving particular object retrieval in large scale image databases. In IEEE conference on computer vision and pattern recognition (CVPR).

  • Ranganathan, A. (2010). Pliss: Detecting and labeling places using online change-point detection. Robotics: Science and Systems (RSS).

  • Rituerto, A., Murillo, A. C., & Guerrero, J. J. (2014). Semantic labeling for indoor topological mapping using a wearable catadioptric system. Robotics and Autonomous Systems, 62(5), 2014.

    Article  Google Scholar 

  • Rublee, E., Rabaud, V., Konolige, K., & Gary R. (2011). Bradski: ORB: An efficient alternative to SIFT or SURF. In International conference on computer vision (ICCV).

  • Schindler, G., Brown, M., & Szeliski, R. (2007). City-scale location recognition. In IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1–7)

  • Sim, R., & Dudek, G. (1998). Mobile robot localization from learned landmarks. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) (Vol. 2).

  • Sivic, J., & Zisserman, A., Video Google: A Text Retrieval Approach to Object Matching in Videos. In Proceedings of the international conference on computer vision (ICCV).

  • Sivic, J., & Zisserman, A. (2003). Video google: Efficient visual search of videos. Toward category-level object recognition, Lecture Notes in Computer science (pp. 127–144). Berlin: Springer.

    Google Scholar 

  • Sunderhauf, N., & Protzel, P. (2011). Brief-gist - closing the loop by simple means. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1234–1241)

  • Tapus, A. & Siegwart, R. (2005). Incremental robot mapping with fingerprints of places. In IEEE/RSJ international conference on in intelligent robots and systems (IROS) (pp. 2429–2434)

  • Ulrich, I., & Nourbakhsh, I. (2000). Appearance-based place recognition for topological localization. In Proceedings of IEEE international conference on robotics and automation (ICRA), (Vol. 2, pp. 1023–1029).

  • Valgren, C., Duckett, T., & Lilienthal, A. J. (2007). Incremental spectral clustering and its application to topological mapping. In IEEE international conference on robotics and automation (ICRA) (pp. 4283–4288).

  • Wang, Junqiu, Cipolla, R., & Zha, Hongbin (2005). Vision-based global localization using a visual vocabulary. In Proceedings of IEEE international conference on robotics and automation (ICRA) (pp. 4230–4235).

  • Wolf, J., Burgard, W., & Burkhardt, H. (2005). Robust vision-based localization by combining an image retrieval system with monte-carlo localization. IEEE Transactions on Robotics, 21(2), 208–216.

    Article  Google Scholar 

  • Zivkovic, Z., Bakker, B., & Kröse, B.J.A. (2005). Hierarchical map building using visual landmarks and geometric constraints. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2480–2485).

Download references

Acknowledgments

This work was partly funded by the Auvergne Region and the French government research program Investissements d’avenir through the RobotEx Equipment of Excellence (ANR-10-EQPX-44) and the LABEX IMOBS3 (ANR-7107-LABX-716701).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hemanth Korrapati.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Korrapati, H., Mezouar, Y. Multi-resolution map building and loop closure with omnidirectional images. Auton Robot 41, 967–987 (2017). https://doi.org/10.1007/s10514-016-9560-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-016-9560-6

Keywords

Navigation