Skip to main content
SpringerLink
Log in

Combining Multiple Image Descriptions for Loop Closure Detection

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The success of visual loop closure detection depends on the discrimination ability of the image descriptions. Different sources of image descriptions may carry complementary information as well as redundant information. Though integrating them properly can be beneficial, a main obstacle is the lack of analytical quality indicators to weigh different descriptions jointly. Inspired by the linear discriminant analysis, we propose an efficacy index to evaluate the weighted linear combinations of multiple image descriptions for loop closure detection. When a collection of image descriptions is given, the optimal weights maximizing the efficacy index are deduced analytically. As negative weights may negatively affect the performance of detection, a gradient descent algorithm is further proposed to jointly optimize the nonnegative weights. We use the proposed weighting strategies to combine the image descriptions extracted from multiple local image patches by multiple descriptor extractors. It is experimentally demonstrated that our weighted combinations of image descriptions can greatly improve the performance of loop closure detection by emphasizing informative components and de-emphasizing redundant components.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bay, H, Ess, A, Tuytelaars, T, Van Gool, L: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  2. Lowe, D G: Distinctive image features from scale-invariant keypoints. Int. J Comput. Vis. 60(2), 91–110 (2004)

    Article  MathSciNet  Google Scholar 

  3. Sivic, J, Zisserman, A, et al.: Video google: A text retrieval approach to object matching in videos. Iccv 2, 1470–1477 (2003)

    Google Scholar 

  4. Datar, M, Immorlica, N, Indyk, P, Mirrokni, VS: Locality-sensitive hashing scheme based on p-stable distributions. In: Proceedings of the Twentieth Annual Symposium on Computational Geometry, pp. 253–262. ACM (2004)

  5. Angeli, A, Filliat, D, Doncieux, S, Meyer, J.-A.: Fast and incremental method for loop-closure detection using bags of visual words. IEEE Trans. Robot. 24(5), 1027–1037 (2008)

    Article  Google Scholar 

  6. Shahbazi, H., Zhang, H: Application of locality sensitive hashing to realtime loop closure detection. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1228–1233. IEEE (2011)

  7. Cummins, M, Newman, P : Fab-map: Probabilistic localization and mapping in the space of appearance. Int. J. Robot. Res. 27(6), 647–665 (2008)

    Article  Google Scholar 

  8. Milford, M.J, Wyeth, G.F: Seqslam: Visual route-based navigation for sunny summer days and stormy winter nights. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1643–1649. IEEE (2012)

  9. Engel, J, Schöps, T, Cremers, D: Lsd-slam: Large-scale direct monocular slam. In: European Conference on Computer Vision, pp. 834–849. Springer (2014)

  10. Mur-Artal, R, Montiel, J M M, Tardos, J D: Orb-slam: A versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  11. Valgren, C, Lilienthal, A J: Sift, surf and seasons: Long-term outdoor localization using local features. In: European Conference on Mobile Robots (ECMR), pp. 253–258 (2007)

  12. Oliva, A, Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)

    Article  Google Scholar 

  13. Calonder, M, Lepetit, V, Ozuysal, M, Trzcinski, T, Strecha, C, Fua, P: Brief: Computing a local binary descriptor very fast. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1281–1298 (2012)

    Article  Google Scholar 

  14. Ojala, T, Pietikainen, M, Maenpaa, T: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  15. Rublee, E, Rabaud, V, Konolige, K, Bradski, G: Orb: An efficient alternative to sift or surf. In: 2011 IEEE International conference on computer vision (ICCV), pp. 2564–2571. IEEE (2011)

  16. Liu, Y, Zhang, H: Visual loop closure detection with a compact image descriptor. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1051–1056. IEEE (2012)

  17. Sünderhauf, N, Protzel, P: Brief-gist: Closing the loop by simple means. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1234–1241. IEEE (2011)

  18. Campos, F.M., Correia, L, Calado, J.M.F: Loop closure detection with a holistic image feature. In: Portuguese Conference on Artificial Intelligence, pp. 247–258. Springer (2013)

  19. Arroyo, R, Alcantarilla, P F, Bergasa, L M, Javier Yebes, J, Gámez, S: Bidirectional loop closure detection on panoramas for visual navigation. In: 2014 Intelligent Vehicles Symposium Proceedings IEEE, pp. 1378–1383. IEEE (2014)

  20. Perronnin, F, Dance, C R: Fisher kernels on visual vocabularies for image categorization. Comput Vis. Pattern Recogn. 1–8 (2007)

  21. Jegou, H, Douze, M, Schmid, C, Perez, P: Aggregating local descriptors into a compact image representation. Comput. Vis. Pattern Recog.,3304–3311 (2010)

  22. Arandjelovic, R, Zisserman, A: All about vlad. Comput. Vis. Pattern Recog., 1578–1585 (2013)

  23. Yi, H, Zhang, H, Zhou, S: Convolutional neural network-based image representation for visual loop closure detection. In: 2015 IEEE International Conference on Information and Automation, pp. 2238–2245. IEEE (2015)

  24. Sünderhauf, N, Shirazi, S, Dayoub, F, Upcroft, B, Milford, M: On the performance of convnet features for place recognition. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4297–4304. IEEE (2015)

  25. Kittler, J, Hatef, M, Duin, R PW, Matas, J: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)

    Article  Google Scholar 

  26. Campos, F M, Correia, L, Calado, J.M.F: Robot visual localization through local feature fusion: An evaluation of multiple classifiers combination approaches. J. Intell. Robot. Syst. 77(2), 377–390 (2015)

    Article  Google Scholar 

  27. Peng, H, Long, F, Ding, C: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)

    Article  Google Scholar 

  28. Levner, I, Zhang, H: Classification-driven watershed segmentation. IEEE Trans. Image Process. 16(5), 1437–1445 (2007)

    Article  MathSciNet  Google Scholar 

  29. Li, W, Mao, K, Zhang, H, Chai, T: Selection of gabor filters for improved texture feature extraction. In: 2010 17th IEEE International conference on Image Processing (ICIP), pp. 361–364. IEEE (2010)

  30. Guyon, I, Elisseeff, A: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  31. Quanquan, G, Li, Z, Han, J: Generalized fisher score for feature selection. arXiv:1202.3725 (2012)

  32. Fisher, R A: The use of multiple measurements in taxonomic problems. Ann. Eugenics 7(2), 179–188 (1936)

    Article  Google Scholar 

  33. Duda, R O, Hart, P E, Stork, D G: Pattern Classification. Wiley (2012)

  34. Li, Q, Ke, L, You, X, Shuhui, B, Liu, Z: Place recognition based on deep feature and adaptive weighting of similarity matrix. Neurocomputing 199, 114–127 (2016)

    Article  Google Scholar 

  35. Tola, E, Lepetit, V, Fua, P: Daisy: An efficient dense descriptor applied to wide-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 815–830 (2010)

    Article  Google Scholar 

  36. Lazebnik, S, Schmid, C, Ponce, J: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178. IEEE (2006)

  37. The Norwegian Broadcasting Corporation: The Nordlandsbanen Dataset. http://nrkbeta.no/2013/01/15/nordlandsbanen-minute-by-minute-season-by-season/ (2013)

  38. Glover, AJ, Maddern, WP, Milford, MJ, Wyeth, GF: Fab-map+ ratslam: Appearance-based slam for multiple times of day. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 3507–3512. IEEE (2010)

  39. Davis, J, Goadrich, M: The relationship between precision-recall and roc curves. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 233–240. ACM (2006)

  40. Wang, X, Zhang, H, Peng, G: A chordiogram image descriptor using local edgels. J. Vis. Commun. Represent. 49, 129–140 (2017)

    Article  Google Scholar 

  41. Toshev, A, Taskar, B, Daniilidis, K: Shape-based object detection via boundary structure segmentation. Int. J. Comput. Vis. 99(2), 123–146 (2012)

    Article  MathSciNet  Google Scholar 

  42. Ce, L, Yuen, J, Torralba, A: Sift flow: Dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)

    Article  Google Scholar 

  43. Boyd, S, Vandenberghe, L: Convex Optimization. Cambridge University Press (2004)

Download references

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, School of Natural and Applied Sciences, Northwestern Polytechnical University, Xi’an, 710072, People’s Republic of China

    Xiaolong Wang & Guohua Peng

  2. Department of Computing Science, University of Alberta, Edmonton, AB, Canada

    Hong Zhang

Authors
  1. Xiaolong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guohua Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaolong Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Peng, G. & Zhang, H. Combining Multiple Image Descriptions for Loop Closure Detection. J Intell Robot Syst 92, 565–585 (2018). https://doi.org/10.1007/s10846-017-0755-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-017-0755-7

Keywords

Search

Navigation