An efficient indexing method for content-based image retrieval

Abstract

In this paper, we propose an efficient indexing method for content-based image retrieval. The proposed method introduces the ordered quantization to increase the distinction among the quantized feature descriptors. Thus, the feature point correspondences can be determined by the quantized feature descriptors, and they are used to measure the similarity between query image and database image. To implement the above scheme efficiently, a multi-dimensional inverted index is proposed to compute the number of feature point correspondences, and then approximate RANSAC is investigated to estimate the spatial correspondences of feature points between query image and candidate images returned from the multi-dimensional inverted index. The experimental results demonstrate that our indexing method improves the retrieval efficiency while ensuring the retrieval accuracy in the content-based image retrieval.

Introduction

Content-based image retrieval has attracted increasing interests in recent years. Given a query image, the image retrieval system obtains the images of the same object or scene from an image database. Due to large collections of images in the database, efficiency is an important factor for content-based image retrieval. Therefore, developing an efficient indexing method for content-based image retrieval is of great significance.

Recently, bag-of-visual-words model [25] is widely used in the indexing methods. In the bag-of-visual-words model, feature points are detected in the images and feature descriptors are computed to describe the feature points. Then a sample group of feature descriptors are clustered into a vocabulary of visual words. The cluster center is defined as a visual word, and each feature descriptor is assigned to the nearest visual word. After that, an image is viewed as a bag of visual words and represented by a frequency histogram of visual words. The similarity between query image and database image is computed by the normalized visual word histograms, which are defined as image vectors. To speed up the on-line query process, the database image vectors are generated in the off-line process.

Based on bag-of-visual-words model, many state-of-the-art indexing methods have been proposed to improve retrieval performance. In Ref. [23], Philbin et al. proposed an indexing method that employs Visual Word quantization and Traditional Inverted index (VWTI). In the visual word quantization, approximate k-means is used to cluster the feature descriptors of database images, and each feature descriptor is assigned to the nearest visual word. TF-IDF weighting [32] is then employed to compute the visual word frequencies and generate the image vectors. According to the sparsity of image vectors, the traditional inverted index is constructed to index the database images. When the query image vector is searched in the inverted index, the cosine similarity between query image vector and database image vector is computed, and the database images with the greatest similarity are returned as the retrieved results. In Ref. [24], Philbin et al. further proposed an indexing method that employs Soft quantization and Traditional Inverted index (STI). In the soft quantization, each feature descriptor is mapped to a weighted combination of visual words, and the weight value is computed by the Gaussian distance between feature descriptor and visual word. TF-IDF weighting is also employed to generate the image vectors, but the visual word frequency is replaced by the multiple normalized weight values for each visual word. To retrieve the most similar database images, the traditional inverted index is constructed by the database image vectors, and the cosine similarity between query image vector and database image vector is computed in the inverted index.

Although the above two indexing methods are simple and effective, they have some limitations for content-based image retrieval. In the VWTI method, the quantized feature descriptors which are represented by the same visual word cannot distinguish from each other, thus the distinction among the quantized feature descriptors is decreased. Moreover, the computation of visual word frequency adds the computational cost to the retrieval process. Because the cosine similarity between the image vectors is computed in the inverted index, it also increases the computational burden and reduces the retrieval efficiency. In the STI method, the quantized feature descriptors which are represented by a weighted combination of visual words decrease the sparsity of image vectors, and the computation of Gaussian distance adds the computational cost to the retrieval process. Because the computational efficiency of inverted index depends on the sparsity of image vectors, the decreased sparsity of image vectors further increases the computational cost in the inverted index and reduces the retrieval efficiency.

In this paper, we propose an indexing method that employs feature point correspondences to improve retrieval efficiency. Once the feature point correspondences are determined, there is no need to generate the image vectors and compute the cosine similarity between the image vectors. For this purpose, we introduce the ordered quantization which assigns the feature descriptors to the ordered visual words. The ordered visual words distinguish the quantized feature descriptors that originally belong to the same visual word, thereby increasing the distinction among the quantized feature descriptors. Thus, if the quantized feature descriptors are represented by the same ordered visual words, the feature point correspondences can be generated to compute the similarity between query image and database image. To compute the image similarity efficiently, a multi-dimensional inverted index is constructed to organize the database images and count the number of feature point correspondences. Afterwards, approximate RANSAC is investigated to estimate the spatial correspondences of feature points between query image and candidate images returned from the multi-dimensional inverted index. The final retrieved images are the candidate images with the greatest spatial similarity.

The remaining of this paper is organized as follows. Section 2 reviews the related work in the indexing methods. Section 3 details our indexing method. Section 4 presents experiments performed to evaluate the indexing method. Section 5 summaries this paper.