Skip to main content
SpringerLink
Log in

Place Inference via Graph-Based Decisions on Deep Embeddings and Blur Detections

  • Conference paper
  • First Online:
Book cover Computational Science – ICCS 2021 (ICCS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12746))

Included in the following conference series:

Abstract

Current approaches to visual place recognition for loop closure do not provide information about confidence of decisions. In this work we present an algorithm for place recognition on the basis of graph-based decisions on deep embeddings and blur detections. The graph constructed in advance permits together with information about the room category an inference on usefulness of place recognition, and in particular, it enables the evaluation the confidence of final decision. We demonstrate experimentally that thanks to proposed blur detection the accuracy of scene recognition is much higher. We evaluate performance of place recognition on the basis of manually selected places for recognition with corresponding sets of relevant and irrelevant images. The algorithm has been evaluated on large dataset for visual place recognition that contains both images with severe (unknown) blurs and sharp images. Images with 6-DOF viewpoint variations were recorded using a humanoid robot.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., Leonard, J.: Past, present, and future of simultaneous localization and mapping: towards the robust-perception age. IEEE Trans. Robot. 32(6), 1309–1332 (2016)

    Article  Google Scholar 

  2. Cebollada, S., Paya, L., Flores, M., Peidro, A., Reinoso, O.:A state-of-the-art review on mobile robotics tasks using artificial intelligence and visual data. Expert Syst. Appl. 167, 114–195 (2020)

    Google Scholar 

  3. Lowry, S., Sünderhauf, N., Newman, P., Leonard, J., Cox, D., Corke, P., Milford, M.J.: Visual place recognition: a survey. IEEE Trans. Robot 32, 1–19 (2016)

    Article  Google Scholar 

  4. Odo, A., McKenna, S., Flynn, D., Vorstius, J.: Towards the automatic visual monitoring of electricity pylons from aerial images. In: 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP) (2020)

    Google Scholar 

  5. Zhao, J., et al. J.:Place recognition with deep superpixel features for brain-inspire dnavigation. Rev. Sci. Instrum. 91(12), 125110 (2020)

    Google Scholar 

  6. Tolias, G., Avrithis, Y., Jégou, H.: Image search with selective match kernels: aggregation across single and multiple images. Int. J. Comput. Vision 116(3), 247–261 (2015)

    Article  MathSciNet  Google Scholar 

  7. Ovalle-Magallanes, E., Aldana-Murillo, N.G., Avina-Cervantes, J.G., Ruiz-Pinales, J., Cepeda-Negrete, J., Ledesma, S.: Transfer learning for humanoid robot appearance-based localization in a visual map. IEEE Access 9, 6868–6877 (2021)

    Article  Google Scholar 

  8. Pretto, A., Menegatti, E., Bennewitz, M., Burgard, W., Pagello, E.: A visual odometry framework robust to motion blur. In: IEEE International Conference on Robotics and Automation, pp. 2250–2257(2009)

    Google Scholar 

  9. Torii, A., Arandjelović, R., Sivic, J., Okutomi, M., Pajdla, T.: 24/7 place recognition by view synthesis. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1808–1817 (2015)

    Google Scholar 

  10. Maffra, F., Chen, Z., Chli, M.: Viewpoint-tolerant place recognition combining 2D and 3D information for UAV navigation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2542–2549(2018)

    Google Scholar 

  11. Garg, S., Milford, M.: Straightening sequence-search for appearance-invariant place recognition using robust motion estimation. In: Proceedings of Australasian Conference on Robotics and Automation (ACRA), pp. 203–212 (2017)

    Google Scholar 

  12. Chen, Z., Lam, O., Adam, J., Milford, M.: Convolutional neural network-based place recognition. In: Proceedings of Australasian Conference on Robotics and Automation, pp. 1–8 (2014)

    Google Scholar 

  13. Suenderhauf, N., Shirazi, S., Dayoub, F., Upcroft, B., Milford, M.: On the performance of ConvNet features for place recognition. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4297–4304 (2015)

    Google Scholar 

  14. Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1437–1451 (2018)

    Google Scholar 

  15. Arandjelovic, R., Zisserman, A.: All About VLAD. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1578–1585. IEEE Computer Society (2013)

    Google Scholar 

  16. Zaffar, M., Khaliq, A., Ehsan, S., Milford, M., McDonald-Maier, K.: Levelling the playing field: a comprehensive comparison of visual place recognition approaches under changing conditions. CoRR abs/1207.0016 (2019)

    Google Scholar 

  17. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: A 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018)

    Google Scholar 

  18. López-Cifuentes, A., Escudero-Vin̄olo, M., Bescós, J., Álvaro García-Martín: Semantic-aware scene recognition. Pattern Recogn. 102, 107256 (2020)

    Google Scholar 

  19. Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J. Comput. Vision 42(3), 145–175 (2001)

    Article  Google Scholar 

  20. Yandex, A.B., Lempitsky, V.: Aggregating local deep features for image retrieval. In: IEEE International Conference on Computer Vision (ICCV), pp. 1269–1277 (2015)

    Google Scholar 

  21. Ma, J., Jiang, X., Fan, A., Jiang, J., Yan, J.: Image matching from handcrafted to deep features: a survey. Int. J. Comput. Vision 129(1), 23–79 (2020)

    Article  MathSciNet  Google Scholar 

  22. Kwolek, B.: Visual odometry based on gabor filters and sparse bundle adjustment. In: Proceedings IEEE International Conference on Robotics and Automation, pp. 3573–3578 (2007)

    Google Scholar 

  23. Arth, C., Pirchheim, C., Ventura, J., Schmalstieg, D., Lepetit, V.: Instant outdoor localization and SLAM initialization from 2.5d maps. IEEE Trans. Visual. Comput. Graph. 21(11), 1309–1318 (2015)

    Google Scholar 

  24. Chen, Z., Maffra, F., Sa, I., Chli, M.: Only look once, mining distinctive landmarks from ConvNet for visual place recognition. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 9–16 (2017)

    Google Scholar 

  25. Hou, Y., Zhang, H., Zhou, S.: Evaluation of object proposals and ConvNet features for landmark-based visual place recognition. J. Intell. Robot. Syst. 92(3–4), 505–520 (2017)

    Google Scholar 

  26. Philbin, J., Isard, M., Sivic, J., Zisserman, A.: Descriptor learning for efficient retrieval. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 677–691. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15558-1_49

    Chapter  Google Scholar 

  27. Tolias, G., Avrithis, Y., Jégou, H.: To aggregate or not to aggregate: selective match kernels for image search. In: IEEE International Conference on Computer Vision, pp. 1401–1408 (2013)

    Google Scholar 

  28. Mao, J., Hu, X., He, X., Zhang, L., Wu, L., Milford, M.J.: Learning to fuse multiscale features for visual place recognition. IEEE Access 7, 5723–5735 (2019)

    Article  Google Scholar 

  29. Camara, L.G., Pr̆euc̆il, L.: Visual place recognition by spatial matching of high-level CNN features. Robot. Auton. Syst. 133, 103625 (2020)

    Google Scholar 

  30. Wozniak, P., Afrisal, H., Esparza, R.G., Kwolek, B.: Scene recognition for indoor localization of mobile robots using deep CNN. In: Chmielewski, L.J., Kozera, R., Orłowski, A., Wojciechowski, K., Bruckstein, A.M., Petkov, N. (eds.) ICCVG 2018. LNCS, vol. 11114, pp. 137–147. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00692-1_13

    Chapter  Google Scholar 

  31. Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 413–420 (2009)

    Google Scholar 

  32. Tolias, G., Sicre, R., Jégou, H.: Particular object retrieval with integral max-pooling of CNN activations. In: International Conference Learning Representations (ICLR 2016) (2016)

    Google Scholar 

  33. Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: IEEE International Conference on Computer Vision, pp. 1470–1477(2003)

    Google Scholar 

  34. Zhong, C., Malinen, M., Miao, D., Fränti, P.: A fast minimum spanning tree algorithm based on k-means. Inf. Sci. 295(C), 1–17 (2015)

    Google Scholar 

  35. Tax, D.M.: Data description toolbox - dd tools, ver. 2.1.3. https://github.com/DMJTax/dd_tools (2021)

  36. Narvekar, N., Karam, L.: A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Trans. Image Process. 20(9), 2678–2683 (2011)

    Article  MathSciNet  Google Scholar 

  37. Pech-Pacheco, J.L., Cristobal, G., Chamorro-Martinez, J., Fernandez-Valdivia, J.: Diatom autofocusing in brightfield microscopy: a comparative study. In: Proceedings of the 15th International coneference on Pattern Recognition, vol. 3, pp. 314–317 (2000)

    Google Scholar 

  38. Sun, J., Wenfei Cao, Zongben Xu, Ponce, J.: Learning a convolutional neural network for non-uniform motion blur removal. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 769–777 (2015)

    Google Scholar 

  39. Cun, X., Pun, C.M.: Defocus blur detection via depth distillation. In: European Conference on Computer Vision (ECCV), pp. 747–763. Springer (2020)

    Google Scholar 

  40. Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8174–8182 (2018)

    Google Scholar 

Download references

Acknowledgment

This work was supported by Polish National Science Center (NCN) under a research grant 2017/27/B/ST6/01743.

Author information

Authors and Affiliations

  1. AGH University of Science and Technology, 30 Mickiewicza, 30-059, Kraków, Poland

    Bogdan Kwolek

  2. Rzeszów University of Technology, Al. Powstańców Warszawy 12, 35-959, Rzeszów, Poland

    Piotr Wozniak

Authors
  1. Piotr Wozniak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bogdan Kwolek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bogdan Kwolek .

Editor information

Editors and Affiliations

  1. AGH University of Science and Technology, Krakow, Poland

    Maciej Paszynski

  2. Ludwig-Maximilians-Universität München, Munich, Germany

    Dieter Kranzlmüller

  3. University of Amsterdam, Amsterdam, The Netherlands

    Valeria V. Krzhizhanovskaya

  4. University of Tennessee at Knoxville, Knoxville, TN, USA

    Jack J. Dongarra

  5. University of Amsterdam, Amsterdam, The Netherlands

    Peter M.A. Sloot

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wozniak, P., Kwolek, B. (2021). Place Inference via Graph-Based Decisions on Deep Embeddings and Blur Detections. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12746. Springer, Cham. https://doi.org/10.1007/978-3-030-77977-1_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77977-1_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77976-4

  • Online ISBN: 978-3-030-77977-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation