Skip to main content
SpringerLink
Log in

Improving image retrieval by using spatial relations

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper we proposed the use of spatial relations as a way of improving annotation-based image retrieval. We analyzed different types of spatial relations and selected the most adequate ones for image retrieval. We developed an image comparison and retrieval method based on conceptual graphs, which incorporates spatial relations. Additionally, we proposed an alternative term-weighting scheme and explored the use of more than one sample image for retrieval using several late fusion techniques. Our methods were evaluated with a rich and complex image dataset, based on the 39 topics developed for the ImageCLEF 2008 photo retrieval task. Results show that: (i) incorporating spatial relations produces a significant increase in performance, (ii) the label weighting scheme we proposed obtains better results than other traditional schemes, and (iii) the combination of several sample images using late fusion produces an additional improvement in retrieval according to several metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Arni T, Sanderson M, Clough P, Grubinger M (2008) Overview of the imageclef 2008 photographic retrieval task. Working Notes of the CLEF

  2. Aslam JA, Montague M (2001) Models for metasearch. In: Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval, pp 276–284

  3. Bekhatir M, Chiaramella Y, Mulhem P (2005) A signal/semantic framework for image retrieval. In: JCDL 005: proceedings of the 5th ACM/IEEE-CS joint conference on digital libraries. ACM, pp 368–368

  4. Berretti S, Bimbo AD, Vicario E (2003) Weighted walkthroughs between extended entities for retrieval by spatial arrangement. IEEE Trans Multimedia 5(1):52–70

    Article  Google Scholar 

  5. Berretti S, Del Bimbo A, Vicario E (2001) Efficient matching and indexing of graph models in content-based retrieval. IEEE Trans Pattern Anal Mach Intell 23(10):1089–1105. doi:10.1109/34.954600

    Article  Google Scholar 

  6. Carneiro G, Chan AB, Moreno PJ, Vasconcelos N (2007) Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans Pattern Anal Mach Intell 29:2007

    Google Scholar 

  7. Chang SK, Shi QY, Yan CW (1987) Iconic indexing by 2-d strings. IEEE Trans Pattern Anal Mach Intell 9(3):413–428

    Article  Google Scholar 

  8. Chen J, Li Z, Li C, Gold CM (1998) Describing topological relations with voronoi-based 9-intersection model. Int Arch Photogramm Remote Sens 32(4):99–104

    Google Scholar 

  9. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 39:1–60

    Article  Google Scholar 

  10. Duygulu P, Barnard K, de Freitas JFG, Forsyth DA (2002) Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: Proceedings of the seventh European conference on computer vision, vol 4, pp 97–112. citeseer.ist.psu.edu/duygulu02object.html

  11. Egenhofer MJ (1989) A formal definition of binary topological relationships. In: Proceedings of the 3rd international conference, FODO 1989 on foundations of data organization and algorithms, pp 457–472

  12. Egenhofer MJ (1993) Definitions of line-line relations for geographic databases. IEEE Data Eng Bull 16(3):40–45

    Google Scholar 

  13. Egenhofer MJ, Franzosa RD (1991) Point-set topological spatial relations. Int J Geogr Inf Syst 5(2):161–174

    Article  Google Scholar 

  14. Escalante H, Hernández-Gracidas C, González J, López A, Montes M, Morales E, Sucar L, Nor LV, Grubinger M (2010) The segmented and annotated iapr-tc12 benchmark. Comput Vis Image Underst 114(4):419–428

    Google Scholar 

  15. Frank AU (1992) Qualitative spatial reasoning about distances and directions in geographic space. J Visual Lang Comput 3(4):343–371

    Article  Google Scholar 

  16. Goodrum A (2000) Image information retrieval: an overview of current research. Inf Sci 3:2000

    Google Scholar 

  17. Goyal RK, Egenhofer MJ (2000) Cardinal directions between extended spatial objects. IEEE Trans Knowl Data Eng

  18. Grubinger M (2007) Analysis and evaluation of visual information systems performance. PhD thesis, School of Computer Science and Mathematics, Faculty of Health, Engineering and Science, Victoria University, Melbourne, Australia

  19. Hernández-Gracidas C, Sucar LE (2007) Markov random fields and spatial information to improve automatic image annotation. In: Proceedings of the 2nd Pacific Rim conference on advances in image and video technology, PSIVT’07. Springer, Berlin, Heidelberg, pp 879–892

    Chapter  Google Scholar 

  20. Hernández-Gracidas C, Sucar LE, Montes-y Gómez M (2009) Modeling spatial relations for image retrieval by conceptual graphs. In: Proceedings of the first Chilean workshop on pattern recognition

  21. Hernández-Gracidas C, Juárez A, Sucar LE, Montes-y Gómez M, Villaseñor L (2010) Data fusion and label weighting for image retrieval based on spatio-conceptual information. In: Proceedings of the 9th international conference on adaptivity, personalization and fusion of heterogeneous information (RIAO)

  22. Jones KS (1972) A statistical interpretation of term specificity and its application in retrieval. J Doc 28:11–21

    Article  Google Scholar 

  23. Kanji GK (1993) 100 statistical tests / Gopal K. Kanji. Sage Publications, London, Newbury Park, California

    Google Scholar 

  24. Lee JH (1997) Analyses of multiple evidence combination. In: SIGIR ’97: proceedings of the 20th annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 267–276

    Chapter  Google Scholar 

  25. Mechkour M, Berrut C, Chiaramella Y (1995) Using a conceptual graph framework for image retrieval. In: Proc. of the MMM95 (multimedia modeling) conference, pp 127–142

  26. Millard DE, Gibbins NM, Michaelides DT, Weal MJ (2005) Mind the semantic gap. In: HYPERTEXT ’05: proceedings of the sixteenth ACM conference on hypertext and hypermedia. ACM, New York, pp 54–62

    Chapter  Google Scholar 

  27. Montes M, Gelbukh A, López-López A, Baeza-Yates R (2000) Comparison of conceptual graphs. In: Proceedings of the Mexican international conference on articial intelligence, MICAI 2000

  28. Paul K (1995) Decision level data fusion for routing of documents in the trec3 context: a best case analysis of worst case results. In: Proceedings of the third text retrieval conference (TREC-3)

  29. Picard RW, Minka TP (1995) Vision texture for annotation. In: Multimedia systems

  30. Rathi V, Majumdar AK (2002) Content based image search over the world wide web. In: Proceedings of the Indian conference on computer vision, graphics and image processing

  31. Ren W, Singh M, Singh S (2002) Image retrieval using spatial context. In: Proceedings of the 9th international workshop on systems, signals and image processing, pp 44–49

  32. Rui Y, Huang TS, Chang S (1999) Image retrieval: current techniques, promising directions and open issues. J Vis Commun Image Represent 10(1):39–62

    Article  Google Scholar 

  33. Shaw JA, Fox EA, Shaw JA, Fox EA (1994) Combination of multiple searches. In: Proceedings of the second text retrieval conference (TREC-2), pp 243–252

  34. Sistla AP, Yu C, Haddack R (1994) Reasoning about spatial relationships in picture retrieval systems. In: Proceedings of the 20th international conference on very large data bases, pp 570–581

  35. Smeulders AWM, Member S, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22:1349–1380

    Article  Google Scholar 

  36. Sowa JF (1984) Conceptual structures: information processing in mind and machine. Addison-Wesley Longman Publishing Co., Inc., Boston

    MATH  Google Scholar 

  37. Vogt CC, Cottrell GW (1999) Fusion via a linear combination of scores. Inf Retr 1(3):151–173

    Article  Google Scholar 

  38. Wu S, McClean S (2006) Performance prediction of data fusion for information retrieval. Inf Process Manag 42(4):899–91

    Article  Google Scholar 

  39. Yuan J, Li J, Zhang B (2007) Exploiting spatial context constraints for automatic image region annotation. In: MULTIMEDIA ’07: proceedings of the 15th international conference on multimedia. ACM, New York, pp 595–604

    Chapter  Google Scholar 

  40. Zhang Q, Yau SS (2004) On intractability of spatial relationships in content-based image database systems. Commun Inf Syst 4(2):181–190

    MathSciNet  MATH  Google Scholar 

  41. Zhou X, Chen J, Li Z, Zhao R, Zhu J (2005) A model for topological relationships based on euler number. In: Proceedings of the ISPRS Hangzhou workshop

Download references

Acknowledgements

This research was partially supported by CONACyT under project grant CB-2008-01-103878 and postdoctoral fellowship 10216. Thanks to Dr. Antonio Juárez who kindly processed our lists with his fusion methods to obtain results for fuzzy Borda count and combMNZ.

Author information

Authors and Affiliations

  1. National Institute of Astrophysics, Optics and Electronics, Luis Enrique Erro No. 1, Tonantzintla, Puebla, Mexico

    Carlos Arturo Hernández-Gracidas, Luis Enrique Sucar & Manuel Montes-y-Gómez

Authors
  1. Carlos Arturo Hernández-Gracidas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis Enrique Sucar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel Montes-y-Gómez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Arturo Hernández-Gracidas.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hernández-Gracidas, C.A., Sucar, L.E. & Montes-y-Gómez, M. Improving image retrieval by using spatial relations. Multimed Tools Appl 62, 479–505 (2013). https://doi.org/10.1007/s11042-011-0911-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0911-1

Keywords

Search

Navigation