Marin Ferecatu
ABSTRACT
Many of the available image databases have keyword annotations associated with the images. In spite of the availability of good quality low-level visual features that reflect well the physical content, image retrieval based on visual features alone is subject to semantic gap. Text annotations are related to image context or semantic interpretation of the visual content and are not necessarely directly linked to the visual appearance of the images. Keywords and visual features thus provide complementary information. Using both sources of information is an advantage in many applications and recent work in this area reflects this interest. In this paper, we address the challenge of semantic gap reduction using a hybrid visual and conceptual representation of the content within an active relevance feedback context. We introduce a new feature vector, based on the keyword annotations available for the images, which makes use of conceptual information extracted from an external lexical database, information represented by a set of "core concepts". Our experiments show that the use of the proposed hybrid conceptual and visual feature vector dramatically improves the quality of the relevance feedback results.
- W. H. Adams, G. Iyengar, C.-Y. Lin, M. R. Naphade, C. Neti, H. J. Nock, and J. R. Smith. Semantic indexing of multimedia content using visual, audio and text cues. EURASIP Journal on Applied Signal Processing, 3(2):170--185, 2003.Google Scholar
- K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107--1135, March 2003. Google ScholarDigital Library
- R. Beckwith, C. Fellbaum, D. Gross, and G. Miller. WordNet: A lexical database organized on psycholinguistic principles. In U. Zernik, editor, Lexical Acquisition: Exploiting On-Line Resources to Build a Lexicon, pages 211--232. Erlbaum, 1991.Google Scholar
- N. Boujemaa, J. Fauqueur, M. Ferecatu, F. Fleuret, V. Gouet, B. L. Saux, and H. Sahbi. Ikona: Interactive generic and specific image retrieval. In Proceedings of the International workshop on Multimedia Content-Based Indexing and Retrieval (MMCBIR'2001), 2001.Google Scholar
- A. Budanitsky and G. Hirst. Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. In Proceedings of the Workshop on WordNet and Other Lexical Resources NAACL 2001, 2001.Google Scholar
- I. J. Cox, M. L. Miller, S. M. Omohundro, and P. N. Yianilos. An optimized interaction strategy for Bayesian relevance feedback. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 553--558. IEEE Computer Society, 1998. Google ScholarDigital Library
- A. del Bimbo. Visual Information Retrieval. Morgan Kaufmann, 1999. Google ScholarDigital Library
- P. Duygulu, K. Barnard, J. F. G. de Freitas, and D. A. Forsyth. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In Proceedings of the 7th European Conference on Computer Vision-Part IV, pages 97--112. Springer-Verlag, 2002. Google ScholarDigital Library
- C. Fellbaum and G. Miller, editors. WordNet: An Electronic Lexical Database. The MIT Press, 1998.Google Scholar
- M. Ferecatu, M. Crucianu, and N. Boujemaa. Retrieval of difficult image classes using SVM-based relevance feedback. In Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, pages 23--30, October 2004. Google ScholarDigital Library
- F. Fleuret and H. Sahbi. Scale-invariance of support vector machines based on the triangular kernel. In 3rd International Workshop on Statistical and Computational Theories of Vision, October 2003.Google Scholar
- T. Gevers and A. W. M. Smeulders. Content-based image retrieval: An overview. In G. Medioni and S. B. Kang, editors, Emerging Topics in Computer Vision. Prentice Hall, 2004.Google Scholar
- M. La Cascia, S. Sethi, and S. Sclaroff. Combining textual and visual cues for content-based image retrieval on the world wide web. In IEEE Workshop on Content-Based Access of Image and Video Libraries, pages 24--28, 1998. Google ScholarDigital Library
- C. Leacock, M. Chodorow, and G. A. Miller. Using corpus statistics and WordNet relations for sense identification. Computational Linguistics, 24(1):147--165, 1998. Google ScholarDigital Library
- M. S. Lew. Principles of Visual Information Retrieval. Springer-Verlag, 2001. Google ScholarDigital Library
- D. Lin. An information-theoretic definition of similarity. In Proc. 15th International Conf. on Machine Learning, pages 296--304, 1998. Google ScholarDigital Library
- V. Mezaris, I. Kompatsiaris, and M. G. Strintzis. Region-based image retrieval using an object ontology and relevance feedback. EURASIP Journal on Applied Signal Processing, 2004(6):886--901, June 2004. Google ScholarDigital Library
- P. Mitra and S. K. Pal. A probabilistic active support vector learning algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(3):413--418, 2004. Google ScholarDigital Library
- P. Resnik. Using information content to evaluate semantic similarity in a taxonomy. In C. S. Mellish, editor, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 448--453, San Mateo, Aug. 20-25 1995. Morgan Kaufmann. Google ScholarDigital Library
- B. Schölkopf. The kernel trick for distances. In Advances in Neural Information Processing Systems, volume 12, pages 301--307. MIT Press, 2000.Google Scholar
- F. Seydoux and J.-C. Chappelier. Indexation sémantique au moyen de coupes de redondance minimale dans une ontologie. In Proceedings of Traitement Automatique du Langage Naturel (TALN'05), pages 33--42, June 2005.Google Scholar
- A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Analysis and Machine Intelligence, 22(12):1349--1380, 2000. Google ScholarDigital Library
- J. R. Smith, S. Basu, C.-Y. Lin, M. R. Naphade, and B. Tseng. Integrating features, models and semantics for content-based retrieval. In Proceedings of the international workshop on MultiMedia Content-Based Indexing and Retrieval (MMCBIR'01), pages 95--98, September 2001.Google Scholar
- S. Tong and E. Chang. Support vector machine active learning for image retrieval. In Proceedings of the 9th ACM international conference on Multimedia, pages 107--118. ACM Press, 2001. Google ScholarDigital Library
- Z. Wu and M. Palmer. Verb semantics and lexical selection. In 32nd. Annual Meeting of the Association for Computational Linguistics, pages 133--138, New Mexico State University, Las Cruces, New Mexico, 1994. Google ScholarDigital Library
- H.-J. Zhang and Z. Su. Improving CBIR by semantic propagation and cross-mode query expansion. In Proceedings of the international workshop on MultiMedia Content-Based Indexing and Retrieval (MMCBIR'01), pages 83--86, September 2001.Google Scholar
- X. S. Zhou and T. S. Huang. Unifying keywords and visual contents in image retrieval. IEEE Multimedia, 9(2):23--33, 2002. Google ScholarDigital Library
- X. S. Zhou and T. S. Huang. Relevance feedback for image retrieval: a comprehensive review. Multimedia Systems, 8(6):536--544, 2003.Google ScholarCross Ref
Index Terms
- Hybrid visual and conceptual image representation within active relevance feedback context
Recommendations
Image retrieval model based on weighted visual features determined by relevance feedback
An accurate and rapid method is required to retrieve the overwhelming majority of digital images. To date, image retrieval methods include content-based retrieval and keyword-based retrieval, the former utilizing visual features such as color and ...
The effect of low-level image features on pseudo relevance feedback
Relevance feedback (RF) is a technique popularly used to improve the effectiveness of traditional content-based image retrieval systems. However, users must provide relevant and/or irrelevant images as feedback for their queries, which is a tedious ...
Techniques for Improving the Performance of Image Retrieval using Relevance Feedback
ICVGIP '14: Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image ProcessingRelevance feedback is important in bridging the semantic gap between the low-level visual features and the high-level semantic concepts during the image retrieval. In this work, the image retrieval using relevance feedback involves four phases. In the ...
Comments