Information Retrieval Methods for Multimedia Objects

Abstract

We describe five major concepts that are essential for multimedia retrieval: uncertain inference addresses vagueness of queries and imprecision of content representations. Predicate logic allows for dealing with spatial and temporal relationships. The document structure has to be considered in order to retrieve the most relevant part of a document in response to a query. Whereas fact retrieval employs an open world assumption, content-based retrieval should be based on an open world assumption. In order to perform inferences based on the content of multimedia objects, inconsistencies have to be dealt with. Based on these concepts, we present DOLORES, a logic-based multimedia retrieval system with a multilayered architecture. Below the top-level presentation layer, the semantic layer uses a conceptual model for structured documents which is transformed into a probabilistic object-oriented logic (POOL) supporting aggregated objects, different kinds of propositions (terms, classifications and attributes) and even rules as being contained in objects. This four-valued logic is translated into probabilistic Datalog which is interpreted by the HySpirit inference engine. Multimedia objects are stored either in a relational database management system or an information retrieval engine.