Skip to main content
SpringerLink
Log in

Hough Pyramid Matching: Speeded-Up Geometry Re-ranking for Large Scale Image Retrieval

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Exploiting local feature shape has made geometry indexing possible, but at a high cost of index space, while a sequential spatial verification and re-ranking stage is still indispensable for large scale image retrieval. In this work we investigate an accelerated approach for the latter problem. We develop a simple spatial matching model inspired by Hough voting in the transformation space, where votes arise from single feature correspondences. Using a histogram pyramid, we effectively compute pair-wise affinities of correspondences without ever enumerating all pairs. Our Hough pyramid matching algorithm is linear in the number of correspondences and allows for multiple matching surfaces or non-rigid objects under one-to-one mapping. We achieve re-ranking one order of magnitude more images at the same query time with superior performance compared to state of the art methods, while requiring the same index space. We show that soft assignment is compatible with this matching scheme, preserving one-to-one mapping and further increasing performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. http://image.ntua.gr/iva/datasets/world_cities

  2. http://image.ntua.gr/iva/research/relaxed_spatial_matching/

  3. http://image.ntua.gr/iva/research/relaxed_spatial_matching

References

  • Avrithis, Y., Kalantidis, Y., Tolias, G., & Spyrou, E. (2010). Retrieving landmark and non-landmark images from community photo collections. Firenze, Italy: ACM Multimedia.

    Google Scholar 

  • Avrithis, Y., Tolias, G., & Kalantidis, Y. (2010). Feature map hashing: Sub-linear indexing of appearance and global geometry. Firenze, Italy: ACM Multimedia.

    Google Scholar 

  • Ballard, D. (1981). Generalizing the hough transform to detect arbitrary shapes. Pattern Recognition, 13(2), 111–122.

    Article  MATH  Google Scholar 

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). SURF: Speeded up robust features. In ECCV.

  • Belongie, S., Malik, J., & Puzicha, J. (2000). Shape context: A new descriptor for shape matching and object recognition. NIPS, 12, 827–831.

    Google Scholar 

  • Berg, A., Berg, T., & Malik, J. (2005). Shape matching and object recognition using low distortion correspondences. In CVPR.

  • Cao, Y., Wang, C., Li, Z., Zhang, L., & Zhang, L. (2010). Spatial-bag-of-features. In CVPR (pp. 3352–3359).

  • Carneiro, G., & Jepson, A. (2007). Flexible spatial configuration of local image features. PAMI, 29(12), 2089–2104.

    Article  Google Scholar 

  • Cheng, Y. (1995). Mean shift, mode seeking, and clustering. PAMI, 17(8), 790–799.

    Article  Google Scholar 

  • Chum, O., Philbin, J., Sivic, J., Isard, M., & Zisserman, A. (2007). Total recall: Automatic query expansion with a generative feature model for object retrieval. In ICCV.

  • Enqvist, O., Josephson, K., & Kahl, F. (2009). Optimal correspondences from pairwise constraints. In ICCV.

  • Fischler, M., & Bolles, R. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Grauman, K., & Darrell, T. (2007). The pyramid match kernel: Efficient learning with sets of features. Journal of Machine Learning Research, 8, 725–760.

    MATH  Google Scholar 

  • Indyk, P., & Thaper, N. (2003). Fast image retrieval via embeddings. In Workshop on Statistical and Computational Theories of Vision.

  • Jégou, H., Douze, M., & Schmid, C. (2008). Hamming embedding and weak geometric consistency for large scale image search. In ECCV.

  • Jégou, H., Douze, M., & Schmid, C. (2009). On the burstiness of visual elements. In CVPR.

  • Jégou, H., Douze, M., & Schmid, C. (2010). Improving bag-of-features for large scale image search. IJCV, 87(3), 316–336.

    Google Scholar 

  • Jiang, H., & Yu, S. X. (2009). Linear solution to scale and rotation invariant object matching. In CVPR.

  • Lazebnik, S., Schmid, C., & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In CVPR (Vol. 2, p. 1).

  • Leibe, B., Leonardis, A., & Schiele, B. (2008). Robust object detection with interleaved categorization and segmentation. IJCV, 77(1), 259–289.

    Google Scholar 

  • Leordeanu, M., & Hebert, M. (2005). A spectral technique for correspondence problems using pairwise constraints. In: ICCV, (Vol. 2, pp. 1482–1489).

  • Lin, Z., & Brandt, J. (2010). A local bag-of-features model for large-scale object retrieval. In ECCV (pp. 294–308).

  • Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. IJCV, 60(2), 91–110.

    Google Scholar 

  • Mikulik, A., Perdoch, M., Chum, O., & Matas, J. (2010). Learning a fine vocabulary. In ECCV.

  • Olsson, C., Eriksson, A., & Kahl, F. (2007). Solving large scale binary quadratic problems: Spectral methods vs. semidefinite programming. In CVPR.

  • Perdoch, M., Chum, O., & Matas, J. (2009). Efficient representation of local geometry for large scale object retrieval. In CVPR.

  • Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2007). Object retrieval with large vocabularies and fast spatial matching. In CVPR.

  • Philbin, J., Chum, O., Sivic, J., Isard, M., & Zisserman, A. (2008). Lost in quantization: Improving particular object retrieval in large scale image databases. In CVPR.

  • Raguram, R., & Frahm, J. M. (2011). Recon: Scale-adaptive robust estimation via residual consensus. In ICCV.

  • Sahbi, H., Audibert, J. Y., Rabarisoa, J., & Keriven, R. (2008). Context-dependent kernel design for object matching and recognition. In CVPR.

  • Scott, G., & Longuet-Higgins, H. (1991). An algorithm for associating the features of two images. Proceedings of the Royal Society of London, 244(1309), 21.

    Article  Google Scholar 

  • Shen, X., Lin, Z., Brandt, J., Avidan, S., & Wu, Y. (2012). Object retrieval and localization with spatially-constrained similarity measure and k-nn re-ranking. In CVPR. IEEE.

  • Sivic, J., & Zisserman, A. (2003) Video Google: A text retrieval approach to object matching in videos. In: ICCV (pp. 1470–1477).

  • Tolias, G., & Avrithis, Y. (2011). Speeded-up, relaxed spatial matching. In ICCV.

  • Vedaldi, A., & Soatto, S. (2008). Quick shift and kernel methods for mode seeking. In ECCV.

  • Vedaldi, A., & Soatto, S. (2008). Relaxed matching kernels for robust image comparison. In CVPR.

  • Wu, Z., Ke, Q., Isard, M., & Sun, J. (2009). Bundling features for large scale partial-duplicate web image search. In CVPR.

  • Zhang, Y., Jia, Z., & Chen, T. (2011). Image retrieval with geometry-preserving visual phrases. In CVPR. IEEE (pp. 809–816).

  • Zhou, W., Lu, Y., Li, H., Song, Y., & Tian, Q. (2010). Spatial coding for large scale partial-duplicate web image search. Firenze, Italy: ACM Multimedia.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Technical University of Athens, Iroon Polytexneiou 9, Zografou, Greece

    Yannis Avrithis & Giorgos Tolias

Authors
  1. Yannis Avrithis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giorgos Tolias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giorgos Tolias.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Avrithis, Y., Tolias, G. Hough Pyramid Matching: Speeded-Up Geometry Re-ranking for Large Scale Image Retrieval. Int J Comput Vis 107, 1–19 (2014). https://doi.org/10.1007/s11263-013-0659-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0659-3

Keywords

Search

Navigation