Skip to main content
SpringerLink
Log in

Detecting Loop Closure with Scene Sequences

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper is concerned with “loop closing” for mobile robots. Loop closing is the problem of correctly asserting that a robot has returned to a previously visited area. It is a particularly hard but important component of the Simultaneous Localization and Mapping (SLAM) problem. Here a mobile robot explores an a-priori unknown environment performing on-the-fly mapping while the map is used to localize the vehicle. Many SLAM implementations look to internal map and vehicle estimates (p.d.fs) to make decisions about whether a vehicle is revisiting a previously mapped area or is exploring a new region of workspace. We suggest that one of the reasons loop closing is hard in SLAM is precisely because these internal estimates can, despite best efforts, be in gross error. The “loop closer” we propose, analyze and demonstrate makes no recourse to the metric estimates of the SLAM system it supports and aids---it is entirely independent. At regular intervals the vehicle captures the appearance of the local scene (with camera and laser). We encode the similarity between all possible pairings of scenes in a “similarity matrix”. We then pose the loop closing problem as the task of extracting statistically significant sequences of similar scenes from this matrix. We show how suitable analysis (introspection) and decomposition (remediation) of the similarity matrix allows for the reliable detection of loops despite the presence of repetitive and visually ambiguous scenes. We demonstrate the technique supporting a SLAM system driven by scan-matching laser data in a variety of settings. Some of the outdoor settings are beyond the capability of the SLAM system itself in which case GPS was used to provide a ground truth. We further show how the techniques can equally be applied to detect loop closure using spatial images taken with a scanning laser. We conclude with an extension of the loop closing technique to a multi-robot mapping problem in which the outputs of several, uncoordinated and SLAM-enabled robots are fused without requiring inter-vehicle observations or a-priori frame alignment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alter, O., Brown, P., and Botstein, D. 2000. Singular value decomposition for genome-wide expression data processing and modelling. In Proceedings of National Academy of Science, 97(18).

  • Altschul, S. and Erickson, B. 1985. Significance of nucleotide sequence alignments: A method for random sequence permutation that preserves Dinuclotide and Codon usage. Molecular Biology and Evolution, 2:526–538.

    Google Scholar 

  • Bar-Shalom, Y. 1987. Tracking and Data Association. Academic Press Professional, Inc. San Diego, CA, USA.

    Google Scholar 

  • Bosse, M., Newman, P., Leonard, J.J., and Teller, S. 2004. SLAM in large-scale cyclic environments using the atlas framework. International Journal of Robotics Research, 23:1113–1139.

    Article  Google Scholar 

  • Cole, D. and Newman, P. 2006. Using laser range data for 3D SLAM in outdoor environments. In Proceedings of International Conference on Robotics and Automation, Florida.

  • Dedeoglu, G. and Sukhatme, G. 2000. Landmark-based matching algorithm for cooperative mapping mapping by autonomous robots. In Proceedings of the Fifth International Symposium on Distribution Autonomous Robotics Systems.

  • Estrada, C., Neira, J., and Tardos, J. D. 2005. Heirarchical SLAM: Real-time accurate mapping or large environments. IEEE Transactions on Robotics Research, 8(4):588–597.

    Article  Google Scholar 

  • Davison, A.J. and Murray, D.W. 2002. Simultaneous localization and map-building using active vision. IEEE Transactions of Pattern Analysis and Machine Intelligence, 24(7):865–880.

    Article  Google Scholar 

  • Davison, A.J. 2003. Real-time simultaneous localisation and mapping with a single camera. In Proceedings of International Conference on Computer Vision.

  • Duda, R., Hart, P., and Stork, D. 2001. Pattern Classification. Wiley, New York: Chichester.

    MATH  Google Scholar 

  • Eustice, R., Pizarro, O., and Singh, H. 2004. Visually augmented navigation in an unstructured environment using a delayed state history. In Proceedings of International Conference on Robotics and Automation.

  • Eustice, R., Singh, H., and Leonard, J. 2005. Exactly sparse delayed-state filters. In Proceedings of International Conference on Robotics and Automation.

  • Fenwick, J., Newman, P., and Leonard, J. 2002. Cooperative concurrent mapping and localization. In Proceedings of the 2002 IEEE International Conference on Robotics and Automation, pp. 1810–1817.

  • Fox, D., Burgard, W., Kruppa, H., and Thrun, S. 2000. A probabilistic approach to collaborative multi-robot localization. Autonomous Robots, 8(3).

  • Gumbel, E.J. 1958. Statistics of Extremes. Columbia Univeristy Press: New York, NY.

    MATH  Google Scholar 

  • Fitzgibbon, A. 2001. Robust registration of 2D and 3D point sets. In Proceedings of the British Machine Vision Conference.

  • Gutmann, J. and Konolige, K. 1999. Incremental mapping of large cyclic environment. In Proceedings of the Conference on Intelligent Robots and Applications (CIRA), Monterey, CA.

  • Hajjdiab, H. and Laganiere, R. 2004. Vision-based multi-robot simultaneous localization and mapping. In Canadian Conference on Computer and Robot Vision, pp. 155–162.

  • Hartley, R. and Zisserman, A. 2000. Multiple View Geometry in Computer Vision. Cambridge University Press: Cambridge.

    MATH  Google Scholar 

  • Ho, K. and Newman, P. 2005. Multiple map intersection detection using visual appearance. In International Conference on Computational Intelligence, Robotics and Autonomous Systems.

  • Ho, K. and Newman, P. 2005. Combining visual and spatial appearance for loop closure detection. In Proceedings of European Conference on Mobile Robotics.

  • Kosecká, J. and Yang, X. 2004. Global localization and relative pose estimation based on scale-invariant features. In Proceedings of International Conference on Pattern Recognition.

  • Kosecka, J., Li, F., and Yang, X. 2005. Global localization and relative positioning based on scale-invariant keypoints. Robotics and Autonomous Systems, 52(1).

  • Konolige, K., Fox, D., Limketkai, B., Ko, J., and Stewart, B. 2003. Map Merging for Distributed Robot Navigation Proceedings of International Conference on Intelligent Robots and Systems.

  • Konolige, K. 2004. Large-scale map-making. In Proceedings of the National Conference on AI (AAAI), San Jose, CA.

  • Leonard, J.J. and Newman, P. 2003. Consistent, convergent, and constant-time SLAM. In Proceedings of International Joint Conference on Artificial Intelligence.

  • Levin, A. and Szeliski, R. 2004. Visual odometry and map correlation. IEEE Conference on Computer Vision and Pattern Recognition.

  • Lowe, D.G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110.

    Article  Google Scholar 

  • Lu, F. and Milios, E. 1997. Robot pose estimation in unknown environments by matching 2D range scans. Journal of Intelligent and Robotic Systems, 18:249–275.

    Article  Google Scholar 

  • Matas, J., Chum, O., Urban, M., and Pajdla, T. 2002. Robust wide baseline stereo from maximally stable extremal regions. In Proceedings of British Machine Vision Conference.

  • Mikolajczyk, C. and Schmid, C. 2004. Scale and affine invariant interest point detectors. International Journal of Computer Vision, 60(1):63–86.

    Article  Google Scholar 

  • Neira, J. and Tardõs, J. D. 2001. Data association in stochastic mapping using the joint compatibility test. IEEE Transactions of Robotics and Automation, 17(6):890–897.

    Article  Google Scholar 

  • Newman, P. and Ho, K. 2005. SLAM—Loop closing with visually salient features. In Proceedings of International Conference on Robotics and Automation.

  • Newman, P., Cole, D. and Ho, K. 2006. Outdoor SLAM using visual appearance and laser ranging. In Proceedings of International Conference on Robotics and Automation, Florida.

  • Nister, D. 2004. An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(6):756–770.

    Article  Google Scholar 

  • Ranganathan, A., Menegatti, E., and Dellaert, F. 2006. IEEE Transactions on Robotics, 22(1):92–107.

    Article  Google Scholar 

  • Royer, E., Lhuiller, M., Dhome, M., and Chateau, T. 2004. Towards an alternative GPS sensor in dense urban environment from visual memory. In Proceedings of British Machine Vision Conference.

  • Se, S., Lowe, D.G., and Little, J. 2002. Mobile robot localization and mapping with uncertainty using scale-invariant visual landmarks. International Journal of Robotics Research, 21(8):735–758.

    Article  Google Scholar 

  • Se, S., Lowe, D.G., and Little, J. 2005. Vision based global localisation and mapping for mobile robots. IEEE Transactions on Robotics, 21(3):364–375.

    Article  Google Scholar 

  • Silpa-Anan, C. and Hartley, R. 2005. Visual localization and loop-back detection with a high resolution omnidirectional camera. Workshop on Omnidirectional Vision.

  • Sparck Jones, K. 1972. Exhaustivity and specificity. Journal of Documentation, 28(1):11–21.

    Google Scholar 

  • Smith, R., Self, M., and Cheeseman, P. 1987. A stochastic map for uncertain spatial relationships. In 4th International Symposium on Robotics Research.

  • Smith, T.F. and Waterman, M.S. 1981. Identification of common molecular subsequences. Journal of Molecular Biology, 147:195–197.

    Article  Google Scholar 

  • Sivic, J. and Zisserman, A. 2003. Visual Google: A text retrieval approach to object matching in videos. In Proceedings of the International Conference on Computer Vision.

  • Thrun, S. 2001. A probabilistic online mapping algorithm for teams of mobile robots. International Journal of Robotics Research, 20(5):335–363.

    Article  Google Scholar 

  • Thrun, S. and Liu, Y. 2003. Multi-robot SLAM with sparse extended information filers. In Proceedings of the 11th International Symposium of Robotics Research.

  • Torralba, A., Murphy, K., Freeman, W., and Rubin, M. 2003. Context-based vision system for place and object recognition. In Proceedings of International Conference on Computer Vision.

  • Wang, J., Cipolla, R., and Zha, H. 2005. Vision-based global localization using a visual vocabulary. In Proceedings of International Conference on Robotics and Automation.

  • Wolf, J., Burgard, W., and Burkhardt, H. 2005. Robust vision-based localization by combining an image-retrieval system with monte carlo localization. IEEE Transactions on Robotics, 21(2):208–216.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Oxford Robotics Research Group, University of Oxford, Parks Road, Oxford, OX1 3PJ, United Kingdom

    Kin Leong Ho & Paul Newman

Authors
  1. Kin Leong Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Newman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kin Leong Ho.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ho, K.L., Newman, P. Detecting Loop Closure with Scene Sequences. Int J Comput Vision 74, 261–286 (2007). https://doi.org/10.1007/s11263-006-0020-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-006-0020-1

Keywords

Search

Navigation