Integrating wavelets with clustering and indexing for effective content-based image retrieval
Introduction
Traditionally, people adapted various techniques to store and handle printed images. However, the need to deal with printed images is diminishing. The development in technology has influenced all aspects of our daily life from personal to business to security to medication, etc. Most people have hand held devices capable of storing large number of digital images and videos. Digital images are available everywhere and in large volumes which make it extremely difficult if at all possible to handle them manually. Medical centers are equipped with sophisticated devices for scanning and capturing images for various parts of the body. Surveillance cameras are affordable even for personal use. They have been installed everywhere to help in maintaining better homeland security. However, the rapid increase in the number of images digitally stored has made it very difficult to browse and search digital libraries of images. Even manual annotation of images becomes a tedious process when we consider the large number of new images daily added to existing collections. Therefore, there is a need for advanced techniques to help in sorting out and easily searching the captured and stored images, including personal collections. Content-based classification, storage and retrieval is the only affordable effective technique that could meet the expectations of naive and professional users. Fortunately, the research community realized this need several decades ago and there exist a number of software packages available to serve various application domains. Each of them has its own advantages and disadvantages which we will discuss later in Section 2. The target can be articulated as follows. For a given image locate all or most of the similar images in a given collection of previously captured and stored images. For instance, a medical lab may keep a database where images of various critical cases are classified and a new image from a current patient may be classified to predict the case for better diagnosis by the medical doctor. A surveillance camera may capture faces of humans entering the metro station and every face could be used to search the database for identifying suspects, if any. Motivated by existing and emerging needs, we concentrate on image databases and hence tackle the problem of developing effective and efficient ways for image retrieval from large databases.
In general, images could be classified into two classes, texture and non-texture. Texture images form an important class, where one pattern is repeated periodically throughout the image. Some medical images such as X-rays, and some topographic images fall under this category. Non-texture images tend to have objects of interest clustered in one or more regions of an image. Most real world images that people are familiar with fall under this second category. In this study, we focus on non-texture images; they are more challenging to handle. However, the same technique can be used to analyze texture images; we have left as future work to report test results on texture images.
To get more insight into the image retrieval systems, lets start by giving a brief definition of this term. An image retrieval system is a comprehensive system for browsing, searching and retrieving images from a large database of digital images. The system may be manual but it is mostly automated. Actually, manual techniques are ad hoc and reflect personal capabilities and preferences of the involved parties. Further, as the number of images increases manual techniques become impractical and the development of automated methods to deal with digital images becomes a need. First efforts to automate the process involve the development of naive approaches that map a given image into a set of attributes which are thought to best describe the image. Common attributes include captions, keywords, description of various parts within the image. The extraction of such attributes is highly subjective and could lead to a messy state in case persons involved in the process have different opinions and perspectives. Regardless of the accompanying difficulties, the latter process leads to storing images and their attributes in a database and using traditional queries to retrieve images by specifying some of their known attributes. All images with attributes that satisfy a given query will be retrieved as the answer set. Such a process limits the image only to its descriptive attributes and may not be acceptable in a professional environment where achieving high accuracy is a critical issue.
An image is a rich source of information and every pixel in the image may count in the process of matching and retrieval. Therefore, the research community shifted the interest from attribute-driven to content-based classification, storage and retrieval. As a result, a number of content-based image retrieval (CBIR) systems have been developed where the search engine can analyze the actual content of the image. The word ‘content’ in this context might refer to color, shape, texture, or any other information that can be derived from the image itself. Within this new trend every image is characterized by a feature vector rather than a set of attributes. A feature vector summarizes the content of the image rather than its characteristics. The target is to extract from each image a compact yet informative feature vector where the redundancy is minimized by considering pixels in groups rather than individually. Feature vectors can be further reduced by applying a feature reduction technique. There are variety of techniques out there for effective feature reduction. Actually part of the framework described in this paper may be used for feature reduction as well. The same automated feature vector extraction process is applied on the stored images and on the query image. Thus, the matching of images in the querying phase is based on comparing feature vectors and hence will achieve higher accuracy. However, linear search of all existing feature vectors to find the one(s) matching the query image becomes unaffordable and time consuming as the number of images increases which is the case for real applications. Therefore, there is a need to applying some preprocessing techniques to categorize images into groups such that the search will be narrowed down to a manageable set of images without sacrificing the accuracy. In this paper, we are proposing a method in which we are integrating data mining, wavelet transform [24] and indexing techniques into the automated search process leading to effective and efficient content-based image retrieval system. Wavelets are employed for feature extraction and hence to help construct the feature vector per image.
In the proposed approach, we combined a clustering algorithm [14] with an indexing technique. Clustering based image retrieval was also used in WAVEQ [5] which is another approached developed earlier by our research group; WAVEQ employs OPTICS [6] distance-based clustering to group images and then performs the search within each cluster. However, none of the clustering algorithms is perfect and there is always probability that most matches to the query image are found to be in cluster i while there are some images in cluster j which are closer to the query image that some of the images retrieved from cluster i. Further, though OPTICS claims to not requiring the number of clusters as input, it works based on two major values that implicitly encapsulate the number of clusters. In other words, OPTICS would require the distance threshold for determining objects in the neighborhood of a given object and an object is considered core if it has a minimum predefined number of objects in its neighborhood. These two values indirectly lead to the number of clusters, i.e., different clustering; and number of clusters will be achieved for different combinations of these two values. Thus, we would argue that going back to simplicity might be a better choice and hence we are using a simpler clustering algorithm, namely k-means [17], [26] to group similar images together. Using a clustering algorithm, we can reduce the search space when a query image is entered to the system. Taking a closer look at the results from the approach that employs only clustering, we realized that filtering all images in different clusters is not a very accurate approach. We can overcome the problem of limiting the search to a single cluster by searching the images in other clusters as well, based on a distance range. For this purpose, we have to store the distance between the cluster center and images in the cluster. We are using B+-tree [25], [33] in this phase in order to take advantage of indexing capabilities of B+-tree and hence reduce the total I/O cost. Searching clusters based on a distance range also has the other advantage when the cluster size is very large. In such a case, we do not have to search the whole cluster.
To summarize, the main target of the research described in this paper is to develop a general purpose CBIR technique that can effectively and efficiently handle large image databases and can be smoothly embedded into different image retrieval systems. The following list outlines the main contributions of the work described in this paper.
- 1.
The proposed approach is decomposable and the various ideas described in this paper can be embedded into other projects. For example, one can adapt another feature extraction algorithm in the preprocessing step and then use our models for narrowing down the search space or for reducing the dimensionality of the feature vectors. Another effort to benefit from our approach is to embed the searching of clusters/classes in a range of B+-tree into any image retrieval project which is using classification or clustering. One could also use a different approach for extracting features and apply image segmentation on top of this. In such case, clustering the image segments and searching clusters in a range of B+-tree would provide many benefits as the search space will increase considerably after segmentation.
- 2.
The proposed approach is scalable. When we developed our system efficiency and scalability were among our main priorities and we reduced the retrieval cost by minimizing the I/O cost, by narrowing the search space, and by reducing the dimensionality of the feature vector.
- 3.
A well-known clustering algorithm is combined with B+-tree implementation. Using parameters cG and cS to search the clusters in a distance range enabled us to get more accurate results in small image clusters, whereas in large image clusters the search space was considerably narrowed down. Best values for these parameters can be found iteratively by using the gain measure proposed in this paper.
- 4.
We have demonstrated the applicability, effectiveness and efficiency of the proposed integrated framework on widely used benchmark datasets.
The rest of the paper is organized as follows. Section 2 provides the background information necessary to understand the proposed approach. We mainly cover the algorithms and data structures necessary to understand the different aspects of the CBIR system we have developed. This section also includes an overview of related work with emphasis on CBIR systems. The CBIR system we have developed by integrating B+-tree and k-means clustering algorithm is described in Section 3. Section 4 reports results of the experiments that we conducted for testing the proposed approach. In Section 5,we discuss the advantages and limitations of the proposed approach and we highlight future research.
Section snippets
Related work
The development of automated CBIR systems has been an attractive research area due to its wide range of applications in critical fields like online monitoring, homeland security, bioinformatics and medical imaging, space images, etc. There are many CBIR approaches described in the literature, e.g., [10], [11], [18], [22], [27], [32], [34], [35], [37], [45], [46], [49]. The two papers [39], [47] include good surveys of CBIR systems. In this section, we will briefly overview some of the already
The proposed approach
In this section, we present the proposed approach for CBIR system in which we use k-means as the clustering algorithm and B+-tree for speeding up the retrieval process. For representing the images, we extract their feature vectors using Daubechies’ wavelets. Then we construct a model using k-means clustering and B+-tree. We use this model during the query phase to find similar images in short time.
The proposed approach consists of three main steps: (1) feature extraction; (2) model construction
Experimental results
In this section, we represent the experimental results of the proposed approach that we discussed in Section 3. We will compare our results with some existing image retrieval systems and explain the advantages of our approach compared to traditional models.
We have tested our system with three different non-texture image databases. The first database contains 1000 general purpose images. The second database contains 224 non-texture images from five different classes; it has been taken from
Conclusions and future research directions
The research scope for this paper focused on the development of CBIR system for non-texture image databases. In the proposed approach, we combined a well-known clustering algorithm k-means with B+-tree data structure for efficient image retrieval. Compared to WaveQ which uses a distance based clustering algorithm, our approach overcomes two main issues. We recursively apply k-means combined with validity analysis for a range of values until the appropriate number of clusters is identified. We
References (49)
Supporting image retrieval framework with rule base system
Knowledge-Based Systems
(2011)A novel image retrieval model based on the most relevant features
Knowledge-Based Systems
(2011)- et al.
A multi-instance ensemble learning model based on concept lattice
Knowledge-Based Systems
(2011) - et al.
Using multiple uncertain examples and adaptive fuzzy reasoning to optimize image characterization
Knowledge-Based Systems
(2007) - et al.
Iconic pictorial retrieval using multiple attributes and spatial relationships
Knowledge-Based Systems
(2006) - et al.
Content-based facial image retrieval using constrained independent component analysis
Information Sciences
(2011) Satisfying general proximity/similarity queries with metric trees
Information Processing Letters
(1991)- et al.
Shape-based image retrieval using support vector machines, Fourier descriptors and self-organizing maps
Information Sciences
(2007) - et al.
Efficient content-based image retrieval using multiple support vector machines ensemble
Expert Systems with Applications
(2012) - et al.
Relevance feedback in content-based image retrieval: some recent advances
Information Sciences
(2002)
Some new indexes of cluster validity
IEEE Transactions on Systems Man, and Cybernetics – Part B
Texture analysis and classification with tree-structured wavelet transform
IEEE Transactions on Image Processing
Image retrieval: ideas, influences, and trends of the new age
ACM Computing Surveys
Randomized clustering forests for image classification
IEEE Transactions on Pattern Analysis and Machine Intelligence
Cited by (54)
A decisive content based image retrieval approach for feature fusion in visual and textual images
2019, Knowledge-Based SystemsAn efficient image retrieval system with structured query based feature selection and filtering initial level relevant images using range query
2018, OptikCitation Excerpt :None of the clustering algorithms is universally used to solve all problems and so clustering algorithm can be selected based on the data set application with different proximity measure and a criterion function [22]. It uses k-means clustering with B+ tree indexing; relevant images are retrieved by considering the closest cluster and other nearby clusters and it has two parameters CG and CS to determine the distance range [24]. Even though the search space and average distance are reduced, the computation complexity increases to find the gain measure for CG and CS.
Content-based image retrieval using color and a novel texture descriptor: orientational-based local binary pattern
2022, Concurrency and Computation: Practice and ExperienceMultimedia Indexing and Retrieval: Optimized Combination of Low-level and High-level Features
2022, International Conference on Enterprise Information Systems, ICEIS - ProceedingsAn image retrieval approach based on feature extraction and self-supervised learning
2022, 2022 2nd International Conference on Distributed Computing and High Performance Computing, DCHPC 2022