Skip to main content
SpringerLink
Log in

Object-based visual query suggestion

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

State-of-the-art visual search systems allow to retrieve efficiently small rigid objects in very large datasets. They are usually based on the query-by-window paradigm: a user selects any image region containing an object of interest and the system returns a ranked list of images that are likely to contain other instances of the query object. User’s perception of these tools is however affected by the fact that many submitted queries actually return nothing or only junk results (complex non-rigid objects, higher-level visual concepts, etc.). In this paper, we address the problem of suggesting only the object’s queries that actually contain relevant matches in the dataset. This requires to first discover accurate object’s clusters in the dataset (as an offline process); and then to select the most relevant objects according to user’s intent (as an on-line process). We therefore introduce a new object’s instances clustering framework based on a major contribution: a bipartite shared-neighbours clustering algorithm that is used to gather object’s seeds discovered by matching adaptive and weighted sampling. Shared nearest neighbours methods were not studied beforehand in the case of bipartite graphs and never used in the context of object discovery. Experiments show that this new method outperforms state-of-the-art object mining and retrieval results on the Oxford Building dataset. We finally describe two object-based visual query suggestion scenarios using the proposed framework and show examples of suggested object queries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://www.robots.ox.ac.uk/~vgg/data/oxbuildings/

  2. http://www-rocq.inria.fr/imedia/belga-logo.html

References

  1. Anjulan A, Canagarajah N (2009) A unified framework for object retrieval and mining. IEEE Trans Circuits Syst Video Technol 19(1):63–76

    Article  Google Scholar 

  2. Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  3. Broder A (1997) On the resemblance and containment of documents. In: Proceedings of the compression and complexity of sequences 1997. IEEE Computer Society, Washington, DC, USA, pp 21–29

    Google Scholar 

  4. Chum O, Matas J (2010) Large-scale discovery of spatially related images. IEEE Trans Pattern Anal Mach Intell 32:371–377

    Article  Google Scholar 

  5. Chum O, Perdoch M, Matas J (2009) Geometric min-hashing: finding a (thick) needle in a haystack. In: IEEE computer society conference on computer vision and pattern recognition. Miami, Florida, pp 17–24

  6. Chum O, Philbin J, Sivic J, Isard M, Zisserman A (2007) Total recall: automatic query expansion with a generative feature model for object retrieval. In: Proceedings of the 11th international conference on computer vision. Rio de Janeiro, Brazil, pp 1–8

  7. Chum O, Philbin J, Zisserman A (2008) Near duplicate image detection: min-hash and tf-idf weighting. In: Proceedings of the British machine vision conference. Leeds, UK, pp 493–502

  8. Devroye L (1986) Non-uniform random variate generation. Springer

  9. Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42:177–196

    Article  MATH  Google Scholar 

  10. Grauman K, Darrell T (2006) Unsupervised learning of categories from sets of partially matching image features. In: IEEE computer society conference on computer vision and pattern recognition, vol 1. New York, NY, pp 19–25

  11. Jégou H, Douze M, Schmid C (2010) Improving bag-of-features for large scale image search. Int J Comput Vis 87:316–336

    Article  Google Scholar 

  12. Joly A, Buisson O (2008) A Posteriori multi-probe locality sensitive hashing. In: ACM international conference on multimedia (MM’08). Vancouver, British Columbia, Canada, pp 209–218

  13. Joly A, Buisson O (2009) Logo retrieval with a contrario visual query expansion. In: Proceedings of the seventeen ACM international conference on multimedia, MM ’09. ACM, Beijing, China, pp 581–584

    Chapter  Google Scholar 

  14. Hamzaoui A, Joly A, Boujemaa N (2011) Multi-source shared nearest neighbours for multi-modal image clustering. Multimed Tools Appl 51:479–503

    Article  Google Scholar 

  15. Houle ME (2008) The relevant-set correlation model for data clustering. Stat Anal Data Min 1:157–176

    Article  MathSciNet  Google Scholar 

  16. Kuo Y-H, Chen K-T, Chiang C-H, Hsu WH (2009) Query expansion for hash-based image object retrieval. In: Proceedings of the 17th ACM international conference on multimedia, MM ’09. Beijing, China, pp 65–74

  17. Letessier Letessier P, Buisson O, Joly A (2011) Consistent visual words mining with adaptive sampling. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR ’11. ACM, Trento, Italy, pp 49:1–49:8

    Google Scholar 

  18. Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the seventh IEEE international conference on computer visio, IEEE Computer Society, vol 2. Kerkyra, Greece, pp 1150–1157

  19. Olken F (1993) Random sampling from databases. Ph.D. thesis, U.C. Berkeley

  20. Philbin J (2010) Scalable object retrieval in very large image collections. Ph.D. thesis, University of Oxford

  21. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2008) Lost in quantization: improving particular object retrieval in large scale image databases. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Anchorage, Alaska

  22. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  23. Philbin J, Sivic J, Zisserman A (2008) Geometric LDA: a generative model for particular object discovery. In: Proceedings of the British machine vision conference. Leeds, UK

  24. Philbin J, Zisserman A (2008) Object mining using a matching graph on very large image collections. In: Sixth Indian conference on Computer Vision, Graphics Image Processing, ICVGIP ’08. Bhubaneswar, India, pp 738–745

  25. Rajeev SG, Rastogi R, Shim K (1999) Rock: a robust clustering algorithm for categorical attributes. In: Information systems, pp 512–521

  26. Sivic J, Russell BC, Zisserman A, Freeman WT, Efros AA (2008) Unsupervised discovery of visual object class hierarchies. In: IEEE conference on computer vision and pattern recognition, CVPR 2008. Anchorage, Alaska, pp 1–8

  27. Tang J, Lewis P (2008) Non-negative matrix factorisation for object class discovery and image auto-annotation. In: ACM international conference on image and video retrieval. Niagara Falls, Canada, pp 105–112

  28. Thompson SK (1995) Adaptive sampling. In: The survey statistician, pp 13–15

  29. Tuytelaars T, Lampert CH, Blaschko MB, Buntine W (2010) Unsupervised object discovery: a comparison. Int J Comput Vis 88:284–302

    Article  Google Scholar 

  30. Wang X, Grimson E (2007) Spatial latent dirichlet allocation. In: Platt JC, Koller D, Singer Y, Roweis S (eds) Advances in neural information processing systems, vol 20. MIT Press, Cambridge, MA, pp 1577–1584

    Google Scholar 

  31. Xu G, Zong Y, Dolog P, Zhang Y (2010) Co-clustering analysis of weblogs using bipartite spectral projection approach. In: Proceedings of the 14th international conference on knowledge-based and intelligent information and engineering systems: Part III, KES’10. Cardiff, Wales, UK, pp 398–407

  32. Zha H, He X, Ding C, Simon H, Gu M (2001) Bipartite graph partitioning and data clustering. In: Proceedings of the tenth international conference on information and knowledge management, CIKM ’01. ACM, Atlanta, Georgia, pp 25–32

    Chapter  Google Scholar 

  33. Zha Z-J, Yang L, Mei T, Wang M, Wang Z (2009) Visual query suggestion. In: Proceedings of the 17th ACM international conference on Multimedia. Beijing, China, pp 15–24

Download references

Acknowledgment

A part of this work has been supported by the EU FP7 project I-SEARCH.

Author information

Authors and Affiliations

  1. INRIA-Rocquencourt, team-project:IMEDIA, BP 105, 78153, Le Chesnay Cedex, France

    Amel Hamzaoui & Alexis Joly

  2. Institut National de l’Audiovisuel (INA), 4 avenue de l’Europe, 94366, Bry-sur-marne Cedex, France

    Pierre Letessier & Olivier Buisson

  3. Inria Saclay, 4 rue Jacques Monod, 91893, Orsay Cedex, France

    Nozha Boujemaa

Authors
  1. Amel Hamzaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pierre Letessier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexis Joly
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olivier Buisson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nozha Boujemaa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amel Hamzaoui.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hamzaoui, A., Letessier, P., Joly, A. et al. Object-based visual query suggestion. Multimed Tools Appl 68, 429–454 (2014). https://doi.org/10.1007/s11042-012-1340-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-012-1340-5

Keywords

Search

Navigation