A Conceptual Image Retrieval Architecture Combining Keyword-Based Querying with Transparent and Penetrable Query-by-Example

Abstract

Performance of state-of-the-art image retrieval systems is strongly limited due to the difficulty of accurately relating semantics conveyed by images to low-level extracted features. Moreover, dealing with the problem of combining modalities for querying is of huge importance in forthcoming retrieval methodologies and is the only solution for achieving significant retrieval performance on image documents. This paper presents an architecture addressing both of these issues which is based on an expressive formalism handling high-level image descriptions. First, it features a multi-facetted conceptual framework which integrates semantics and signal characterizations and operates on image objects (abstractions of visual entities within a physical image) in an attempt to perform indexing and querying operations beyond trivial low-level processes and region-based frameworks. Then, it features a query-by-example framework based on high-level image descriptions instead of their extracted low-level features and operate both on semantics and signal features. The flexibility of this module and the rich query language it offers, consisting of both boolean and quantification operators, lead to optimized user interaction and increased retrieval performance. Experimental results on a test collection of 2500 images show that our approach gives better results in terms of recall and precision measures than state-of-the-art frameworks which couple loosely keyword-based query modules and relevance feedback processes operating on low-level features.