Skip to main content
SpringerLink
Log in

Coarse-to-Fine Visual Place Recognition

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13111))

Included in the following conference series:

  • 2037 Accesses

Abstract

Visual Place Recognition (VPR) aims to locate one or more images depicting the same place in the geotagged database with a given query and is typically conducted as an image retrieval task. Currently, global-based and local-based descriptors are two mainstream representations to solve VPR. However, they still struggle against viewpoint change, confusion from similar patterns in different places, or high computation complexity. In this paper, we propose a progressive Coarse-To-Fine (CTF-VPR) framework, which has a strong ability on handling irrelevant matches and controlling time consumption. It employs global descriptors to discover visually similar references and local descriptors to filter those with similar but irrelative patterns. Besides, a region-specific representing format called regional descriptor is introduced with region augmentation and increases the possibilities of positive references with partially relevant areas via region refinement. Furthermore, during the spatial verification, we provide the Spatial Deviation Index (SDI) considering coordinate deviation to evaluate the consistency of matches. It discards exhaustive and iterative search and reduces the time consumption hundreds of times. The proposed CTF-VPR outperforms existing approaches by 2%–3% recalls on Pitts250k and Tokyo24/7 benchmarks.

This work is supported in part by the National Natural Science Foundation of China Under Grants No. U20B2066, the Open Research Projects of Zhejiang Lab (Grant No. 2021KB0AB01), and the National Key R&D Program of China (Grant No. 2020AAA0109304).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: Netvlad: CNN architecture for weakly supervised place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5297–5307 (2016)

    Google Scholar 

  2. Arandjelovic, R., Zisserman, A.: All about VLAD. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1578–1585 (2013)

    Google Scholar 

  3. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  4. Cao, B., Araujo, A., Sim, J.: Unifying deep local and global features for image search. In: European Conference on Computer Vision (2020)

    Google Scholar 

  5. Castle, R., Klein, G., Murray, D.W.: Video-rate localization in multiple maps for wearable augmented reality. In: 2008 12th IEEE International Symposium on Wearable Computers, pp. 15–22. IEEE (2008)

    Google Scholar 

  6. Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, pp. 1–22, Prague (2004)

    Google Scholar 

  7. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 337–33712 (2018)

    Google Scholar 

  8. Doan, A.D., Latif, Y., Chin, T.J., Liu, Y., Do, T.T., Reid, I.: Scalable place recognition under appearance change for autonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

    Google Scholar 

  9. Dusmanu, M., et al.: D2-net: A trainable CNN for joint description and detection of local features. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  10. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  11. Ge, Y., Wang, H., Zhu, F., Zhao, R., Li, H.: Self-supervising fine-grained region similarities for large-scale image localization. In: European Conference on Computer Vision (2020)

    Google Scholar 

  12. Häne, C., Heng, L., Lee, G.H., Fraundorfer, F., Furgale, P., Sattler, T., Pollefeys, M.: 3d visual perception for self-driving cars using a multi-camera system: calibration, mapping, localization, and obstacle detection. Image Vis. Comput. 68, 14–27 (2017)

    Article  Google Scholar 

  13. Hausler, S., Garg, S., Xu, M., Milford, M., Fischer, T.: Patch-NetVLAD: multi-scale fusion of locally-global descriptors for place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  14. Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2599–2606. IEEE (2009)

    Google Scholar 

  15. Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3304–3311 (2010)

    Google Scholar 

  16. Kim, H.J., Dunn, E., Frahm, J.M.: Learned contextual feature reweighting for image geo-localization. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3251–3260 (2017)

    Google Scholar 

  17. Liu, L., Li, H., Dai, Y.: Stochastic attraction-repulsion embedding for large scale image localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2570–2579 (2019)

    Google Scholar 

  18. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  19. Lowry, S., Sünderhauf, N., Newman, P., Leonard, J.J., Cox, D., Corke, P., Milford, M.J.: Visual place recognition: a survey. IEEE Trans. Rob. 32(1), 1–19 (2016)

    Article  Google Scholar 

  20. Middelberg, S., Sattler, T., Untzelmann, O., Kobbelt, L.: Scalable 6-DOF localization on mobile devices. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 268–283. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10605-2_18

    Chapter  Google Scholar 

  21. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: a versatile and accurate monocular slam system. IEEE Trans. Rob. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  22. Noh, H., Araujo, A., Sim, J., Weyand, T., Han, B.: Large-scale image retrieval with attentive deep local features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3456–3465 (2017)

    Google Scholar 

  23. Perronnin, F., Liu, Y., Sánchez, J., Poirier, H.: Large-scale image retrieval with compressed fisher vectors. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3384–3391. IEEE (2010)

    Google Scholar 

  24. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

    Google Scholar 

  25. Radenović, F., Tolias, G., Chum, O.: Fine-tuning CNN image retrieval with no human annotation. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1655–1668 (2018)

    Article  Google Scholar 

  26. Revaud, J., Almazán, J., Rezende, R.S., Souza, C.R.: Learning with average precision: training image retrieval with a listwise loss. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5107–5116 (2019)

    Google Scholar 

  27. Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperGlue: learning feature matching with graph neural networks. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  28. Sattler, T., Leibe, B., Kobbelt, L.: Fast image-based localization using direct 2D-to-3D matching. In: 2011 International Conference on Computer Vision, pp. 667–674. IEEE (2011)

    Google Scholar 

  29. Sattler, T., Weyand, T., Leibe, B., Kobbelt, L.: Image retrieval for image-based localization revisited. In: 2012 British Machine Vision Conference, p. 4 (2012)

    Google Scholar 

  30. Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104–4113 (2016)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  32. Torii, A., Arandjelović, R., Sivic, J., Okutomi, M., Pajdla, T.: 24/7 place recognition by view synthesis. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1808–1817 (2015)

    Google Scholar 

  33. Torii, A., Sivic, J., Okutomi, M., Pajdla, T.: Visual place recognition with repetitive structures. IEEE Trans. Pattern Anal. Mach. Intell. 37(11), 2346–2359 (2015)

    Article  Google Scholar 

  34. Yi, K.M., Trulls, E., Lepetit, V., Fua, P.: LIFT: learned invariant feature transform. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 467–483. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_28

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

    Junkun Qi, Rui Wang, Chuan Wang & Xiaochun Cao

  2. Zhejiang Lab, Hangzhou, China

    Rui Wang

  3. School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

    Junkun Qi, Rui Wang, Chuan Wang & Xiaochun Cao

Authors
  1. Junkun Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaochun Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rui Wang .

Editor information

Editors and Affiliations

  1. Sampoerna University, Jakarta, Indonesia

    Teddy Mantoro

  2. Kyungpook National University, Daegu, Korea (Republic of)

    Minho Lee

  3. Sampoerna University, Jakarta, Indonesia

    Media Anugerah Ayu

  4. Murdoch University, Murdoch, WA, Australia

    Kok Wai Wong

  5. Universitas Indonesia, Depok, Indonesia

    Achmad Nizar Hidayanto

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Qi, J., Wang, R., Wang, C., Cao, X. (2021). Coarse-to-Fine Visual Place Recognition. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13111. Springer, Cham. https://doi.org/10.1007/978-3-030-92273-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-92273-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-92272-6

  • Online ISBN: 978-3-030-92273-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation