Elsevier

Image and Vision Computing

Volume 19, Issue 13, 1 November 2001, Pages 979-986
Image and Vision Computing

Color-based image retrieval using spatial-chromatic histograms

https://doi.org/10.1016/S0262-8856(01)00060-9Get rights and content

Abstract

The paper describes a new indexing methodology for image databases integrating color and spatial information for content-based image retrieval. This methodology, called Spatial-Chromatic Histogram (SCH), synthesizing in few values information about the location of pixels having the same color and their arrangement within the image, can be more satisfactory than standard techniques when the user would like to retrieve from the database the images that actually resemble the query image selected in their color distribution characteristics. Experimental trials on a database of about 3000 images are reported and compared with more standard techniques, like Color Coherence Vectors, on the basis of human perceptual judgments.

Introduction

The dramatic improvements in hardware technology have made it possible in the last few years to process, store and retrieve huge amounts of data in multimedia format. First attempts to manage pictorial databases relied on textual description provided by a human operator. This time-consuming approach rarely captures the richness of visual content of the images cannot be easily expressed (either interactively or automatically) through words. For this reason research has been focused on the automatic extraction of the visual content of image to enable indexing and retrieval. A broad range of techniques [1], [2] are now available. In particular, techniques exploiting low-level visual features have become a promising research issue [3], [4], [6], [11], [12], [13], [14]. General-purpose systems of this type are also now available e.g. [15], [17].

These systems usually make it possible to extract image representations in terms of color, texture, shape and layout features from the images and define the relative search/matching functions that can be used to retrieve those of interest. However, not withstanding the substantial progress made, the integrated management of the different features remains complex and application dependent [16], [19].

We limit ourselves here to work with color and introduce an efficient and effective method to characterize the amount and arrangement of the color pixels in an image. Such a feature may coexist with other features (shape, texture, size, distance, relative position, etc.) expressing different aspects of image content in the framework of a Visual Information Retrieval System, such as [18].

The problem here addressed has been defined by Mehtre et al. [16] as: “Assume that there are a large number of color images in the database. Given a query image, we would like to obtain a list of images from the database which are “most” similar in color to the query image”. More formally, given a query image Q and a database of images, find the images having chromatic content similar to Q. The chromatic similarity is to be computed using a function which evaluates the closeness of features extracted from images.

The image retrieval process can be divided in two steps: (1) indexing, for each image in a database a set or a vector of features summarizing its content properties is computed and stored; (2) retrieval, given a query image its features are extracted and compared to the others in the database. Database images are then ordered following a similarity criterion, as stated above.

In order to allow a software system to automatically perform image indexing, based on image content, we developed a simple technique to compute both color and spatial features. We believe that simple color information, such as those provided by a histogram, are not sufficient in many situations.

In this paper, we describe a novel, effective technique, called Spatial-Chromatic Histogram (SCH), to code spatial-chromatic features of the indexed image. SCH has been defined to answer the following questions:

  • 1.

    How many meaningful colors are there in the image?

  • 2.

    Where are the pixels with similar color approximately located?

  • 3.

    How are these pixels spatially arranged?

The paper is organized as follows. We review the state-of-the art in Section 2, while SCH are formally defined in Section 3. A suitable metrics for evaluating SCH similarity is defined and discussed in Section 4. Section 5 summarizes the experimental results, while we give our conclusions in Section 6.

Section snippets

Previous work

Color has been used for content-based image and video retrieval, probably because color features are quite easy to compute. In particular the color histograms of quantized images has been widely used to describe the image color content. The similarity between two color histograms is commonly measured with techniques based upon the Minkowski metric. Example of the use of histograms can be found in the work of Swain and Ballard [5], oriented towards pattern recognition for robot vision, and in

Color quantization

The effective and efficient computation of the indices requires a drastic reduction in the number of colors used to represent the color contents of our 24-bit images.

Our quantization method [14] exploits the partition of the gamut of feasible colors in equivalence classes corresponding to standardized linguistic tags. This method partitions the CIELAB color space into 256 subspaces (categories), in each of which the color remains perceptually the same, is labeled with an unique linguistic tag,

Similarity function

Feature vectors extracted from images are usually compared using L1 and L2 distance functions.1

There are, of course, examples of more sophisticated metrics like that in [6], where a quadratic form function takes accounts for

Experimental results

In order to test the performances of SCH-based indexing methodology, we compared it with the CCV-based one, performing different experiments.

Conclusions and future work

The research presented here focused on one pictorial feature, color, which, together with shape, texture, size, distance and relative position, characterizes scenes and pictures. We have introduced an image retrieval algorithm based on SCH that makes it possible to take into account spatial information in a flexible way without drastically increasing the computation cost.

The method proposed in this paper has several advantages if compared with other existing ones. Its main features are: (a) it

References (33)

  • Y. Rubner et al.

    A metric for distributions with applications to image databases

    Proc. of the IEEE Int. Conf. on Computor Vision pp. 59–66

    (1998)
  • S.-C. Pie et al.

    extracting color features and dynamic matching for image data-base retrieval

    IEEE Tran. Circuits Syst. Video Technol.

    (1999)
  • X. Wan et al.

    A new approach to image retrieval with hierarchical color clustering

    IEEE Tran. Circuits Syst. Video Technol.

    (1998)
  • A. Del Bimbo et al.

    Visual image retrieval by elastic matching of user sketches

    Tran. Pattern Anal. Machine Intelligence

    (1997)
  • B. Barolo et al.

    An effective strategy for querying image databases by color distribution

    Comput. History Art J.

    (1997)
  • A. Della Ventura et al.

    Indexing color-texture image patterns

    Adv. Comput. Vision Machine Intelligence

    (1998)
  • Cited by (0)

    View full text