Skip to main content
SpringerLink
Log in

An operational model based on knowledge representation for querying the image content with concepts and relations

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to overcome the semantic gap (i.e. the gap between low-level extracted features and semantic description in state-of-the-art content-based image retrieval systems, a class of frameworks proposed within the framework of the European Fermi project, consisted of modeling the semantic content of images following a sharp process of human-assisted indexing. These approaches, based on expressive knowledge-based representation models provide satisfactory results in terms of retrieval quality but are not easily usable on large collections of images because of the necessary human intervention required for indexing. We propose in this paper to integrate the content-based and semantic-based solutions through a model featuring semantic and relational characterizations of the multimedia (image) content for automatic symbolic indexing and retrieval. Its instantiation as an image retrieval framework relies on a representation formalism handling high-level image descriptions and allowing to query with conceptual descriptors. Our approach complements content-based solutions through the mapping of low-level extracted features to semantic concepts and the manipulation of graph-based symbolic index and query structures; and extends the semantic-based solutions by considering automatically-extracted semantic and relational information. At the experimental level, we evaluate the retrieval performance of our system on queries coupling both semantic and relational characterizations through recall and precision indicators on a test collection of 2,500 color photographs and the TRECVID keyframe corpus.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Belkhatir M, Mulhem P, & Chiaramella Y (2004) Integrating perceptual signal features within a multi-facetted conceptual model for automatic image retrieval. ECIR 267–282

  2. Belkhatir M (2005) A full-text framework for the image retrieval signal/semantic integration. DEXA 113–123

  3. Bosch P et al. (2001) Exact matching in image databases. IEEE ICME

  4. Bradshaw B (2000) Semantic based image retrieval: A probabilistic approach. ACM MM 167–176

  5. Carson C et al. (1999) Blobworld: a system for region-based image indexing and retrieval. VISUAL 509–516

  6. Cohn A et al (1997) Qualitative spatial representation and reasoning with the region connection calculus. Geoinformatica 1:1–44

    Article  Google Scholar 

  7. Cox I (2000) The bayesian image retrieval system, pichunter: theory, implementation and psychophysical experiments. IEEE Trans. Image Processing 9(1):20–37

    Article  Google Scholar 

  8. Di Sciascio E, Donini MF, Mongiello M (2002) Spatial layout representation for query-by-sketch content-based image retrieval. Pattern Recognition Letters 23(13):1599–1612

    Article  MATH  Google Scholar 

  9. Egenhofer M (1991) Reasoning about binary topological relations. SSD 143–160

  10. Hollink L et al (2004) Classification of user image descriptions. Int. Journal of Human Computer Studies 61(5):601–626

    Article  Google Scholar 

  11. Ianeva T et al. (2004) Probabilistic approaches to video retrieval. TREC Video retrieval Evaluation Online Proceedings, TRECVID URL: www-nlpir.nist.gov/projects/tvpubs/tvpapers04/cwi-twente.pdf

  12. Jeon J et al. (2003) Automatic image annotation and retrieval using cross-media relevance models. SIGIR 119–126

  13. La Cascia et al. (1998) Combining textual and visual cues for content-based image retrieval on the World Wide Web. IEEE Workshop on Content-Based Access of Image and Video Libraries 24–28

  14. Lim JH (2000) Explicit query formulation with visual keywords. ACM MM 407–412

  15. Lu Y et al. (2000) A unified framework for semantics and feature based relevance feedback in image retrieval systems. ACM MM 31–37

  16. Ma W, Manjunath B (1997) NeTra: A toolbox for navigating large image databases. ICIC 568–571

  17. Mechkour M (1995) EMIR2: An extended model for image representation and retrieval. DEXA 395–404

  18. Mittal A, Cheong LF (2003) Framework for synthesizing semantic-level indexes. Multimedia Tools and Applications 20(2):135–158

    Article  Google Scholar 

  19. Moghaddam B, Biermann H, Margaritis D (1999) Defining image content with multiple regions-of-interest. MERL Tech Report TR99–10

  20. Mojsilovic A, Rogowitz B (2001) Capturing image semantics with low-level descriptors. ICIP 18–21

  21. Mulhem P, Lim JH (2002) Symbolic photograph content-based retrieval. CIKM 94–101

  22. Naphade MR, Kozintsev I, Huang TS (2002) Factor graph framework for semantic video indexing. IEEE Trans. Circuits Syst. Video Techn. 12(1):40–52

    Article  Google Scholar 

  23. Niblack W et al. (1993) The QBIC project: Querying images by content using color, texture and shape. SPIE, Storage and Retrieval for Image and Video Databases 40–48

  24. Nie J (1988) An outline of a general model for information retrieval systems. SIGIR 495–506

  25. Ounis I, Pasca M (1998) RELIEF: Combining expressiveness and rapidity into a single system. SIGIR 266–274

  26. Pentland R et al. (1994) Photobook: Tools for content-based manipulation of image databases. SPIE 34–47

  27. Rui Y et al. (1997) Content-based image retrieval with relevance feedback in MARS. IEEE ICIP 815–818

  28. Smeulders AWM (2000) Content-based image retrieval at the end of the early years. IEEE PAMI, 22(12):1349–1380

    Google Scholar 

  29. Sowa JF (1984) “Conceptual structures: Information processing in mind and machine”. Addison-Wesley

  30. Town CP, Sinclair D (2000) CBIR using semantic visual categories”. TR2000-14, AT&T Labs Cambridge

  31. Westerveld T, De Vries AP (2003) Experimental evaluation of a generative probabilistic image retrieval model on ‘Easy’ data. Proceedings of SIGIR Multimedia Information Retrieval Workshop

  32. Zhou XM, Ang CH, Ling TW (2001) Image retrieval based on object’s orientation spatial relationship. Pattern Recognition Letters 22(5):469–477

    Article  MATH  Google Scholar 

  33. Zhou XS, Huang TS (2002) Unifying keywords and visual contents in image retrieval. IEEE Multimedia 9(2):23–33

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Monash University, Sunway, 46150, Malaysia

    Mohammed Belkhatir

Authors
  1. Mohammed Belkhatir
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed Belkhatir.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Belkhatir, M. An operational model based on knowledge representation for querying the image content with concepts and relations. Multimed Tools Appl 43, 1–23 (2009). https://doi.org/10.1007/s11042-008-0254-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-008-0254-8

Keywords

Search

Navigation