Color and grey level object retrieval using a 3D representation of force histogram

doi:10.1016/S0262-8856(03)00016-7

Image and Vision Computing

Volume 21, Issue 6, 1 June 2003, Pages 483-495

https://doi.org/10.1016/S0262-8856(03)00016-7 Get rights and content

Abstract

A new method for both grey level and color object retrieval is presented in this paper. Our feature is based on previous works on force histogram notion which is extended here to handle with photometric information. This kind of feature has low computing time and allows keeping fundamental geometric transformations as scale, translation, symmetry and rotation. More precisely objects processed by defining a tridimensional signature which takes into account their photometric variations and their shapes. Experimental results show the promising aspect of our approach.

Introduction

A new approach for the content-based retrieval problem is presented in this paper. Our feature is based on force histogram introduced by Matsakis and Wendling [18], [19] which allows keeping fundamental geometric transformations as scale factor, translation, rotation and symmetry. It has been shown in previous works that such a feature is a powerful tool to define spatial relations [18], [19] between objects and also to describe robust signatures for binary objects [19], [35]. In this paper we show that such a notion can be extended to retrieve both grey level and color objects by including photometric variations in the force histogram. First, a brief overview of content-based retrieval approaches is given in Section 2. Then, the definition of a force histogram and its main properties are recalled in Section 3. The calculation of $F$ _3D-signatures on grey level and color objects is presented in Section 4. The measure of similarity between two $F$ _3D-signatures is given in Section 5. Finally, some experimental studies using image databases and a discussion about the advantages and the limits of our approach are provided in Section 6.

Section snippets

Related woks

More and more content-based image retrieval systems have been developed in the early years [29]. Full overviews concerning the indexing problem can be found in the literature [1], [11], [29]. Some systems allow querying by keywords, assuming that each image has been manually indexed when inserted into the database. Others systems often use one or more features to describe the content of images in order to provide an automatic content-based retrieval process. We give here a succinct description

Description of the method

In this section, the scheme for computing the $F$ -signature of a binary graphical object, which is a particular histogram of forces [18], [19], [35] is recalled. A histogram of forces can be assumed to be the calculation of all the forces exerted between the pixels of a same object. Let ϕ_r be the map from $R$ into $R$ ₊, null on $R$ ₋ and continuous on $R$ ₊^∗, such that: $∀d∈ R_{+}^{∗}, ϕ_{r} (d)=1/d^{r}$ Let a₁ and a₂ be two points of $R$ ², and d be the distance between a₁ and a₂. The attraction force is given by ϕ(d). The

Usefulness

It is possible to integrate the photometric variations during the definition of a $F$ _2D-signature. In this case the grey level difference between two consecutive cuts should be processed. Actually two kinds of schemes may be used to handle the level cuts α_i using force histogram [18]. The first directly derives from the generic double sum scheme defined by Dubois and Jaulent [8] and has a high processing time. We have then used the simple sum scheme, proposed by Khrisnapuram et al. [13] which

Butterfly database

We have used a database provided by Derrode et al. [7]. It consists of around a hundred of images of butterflies describing several clusters following different scales and orientations. Fig. 6 presents four types of butterflies with their associated $F$ _3D-signatures.

Fig. 7 gives the nine nearest images of butterfly pap28_1 and the scores reached. We can see that for each image the results provided are right and both the scale and the rotation factors are taken into account by our approach.

The

Conclusion

We have shown in this paper that an $F$ _3D-signature allows to reach a fast and robust retrieval using both color and grey level objects. Preliminary tests were performed on color photos. Currently, we search to extend our approach to the main regions of an image. The study of fast methods of ‘rough’ segmentation of the image is under consideration. The main connected regions obtained will be used like masks and the $F$ _3D-signatures computed on them will be matched with the database. Such an

References (35)

S. Antani et al.
A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video
Pattern Recognition
(2002)
Y.Q. Chen et al.
Statistical geometrical features for texture classification
Pattern Recognition
(1995)
D. Dubois et al.
A general approach to parameter evaluation in fuzzy digital pictures
Pattern Recognition Letters
(1987)
T. Gevers et al.
Context-based image retrieval by viewpoint-invariant color indexing
Image and Vision Computing
(1999)
L. Wendling et al.
Fast and robust recognition of orbit and sinus drawings using histograms of forces
Pattern Recognition Letters
(2002)
S.K. Bhattacharjee, A computational approach to image retrieval, PhD Thesis number 2002, EPFL, Lausanne, Switzerland,...
J. Bigun et al.
Orientation radiograms for image retrieval: an alternative to segmentation
IEEE International Conference on Pattern Recognition, Vienna, Austria
(1996)
S. Bres et al.
Detection of interest point for image indexation
Third International Conference on Visual Information Systems, Amsterdam
(1999)
J.E. Bresenham
Algorithm for computer control of a digital plotter
IBM Systems Journal
(1965)
S. Derrode et al.
Invariant content-based image retrieval using a complete set of Fourier-Mellin descriptors
IEEE International Conference on Multimedia Computing and Systems
(1999)

T. Gevers, Color image invariant segmentation and retrieval, PhD thesis, University of Amsterdam, The Netherlands,...

A.K. Jain et al.

Statistical pattern recognition: a review

IEEE Transactions on PAMI

(2000)

T. Kato

Image Storage and Retrieval Systems

(1992)

R. Krishnapuram et al.

Quantitative analysis of properties and spatial relations of fuzzy image regions

IEEE Transactions on Fuzzy Systems

(1993)

C.P. Lam et al.

STAR—a system for trademark archival and retrieval

Proceedings of the Second Asian Conference on Computer Vision, Singapore

(1995)

T. Lonnestad

A new set of texture features based on the Haar transform

Proceedings of the 11th IAPR International Conference on Pattern Recognition, The Hague, Netherlands

(1992)

B. Manjunath et al.

Texture features for browsing and retrieval of image data

IEEE Transactions on PAMI

(1996)

Cited by (13)

Fuzzy directional enlacement landscapes for the evaluation of complex spatial relations
2020, Pattern Recognition
Citation Excerpt :
They have a known behavior with regards to affine transformations [13], and efficient algorithms have been designed for their computation [14]. These descriptors are applied in different works, such as the linguistic description of spatial relations [15], scene matching [16] or image retrieval [17,18]. To summarize, fuzzy landscapes consist in determining the region of space matching a specific spatial relation, and relative position descriptors consist in characterizing the position of an object with regards to another, by combining different spatial features into a whole descriptor.
Structural spatial relations between image components are fundamental in the human perception of image similarity, and constitute a challenging topic in the domain of image analysis. By definition, some specific relations are ambiguous and difficult to formalize precisely by humans. In this work, we deal with the issue of evaluating complex spatial configurations, where objects can surround each other, potentially with multiple levels of depth. Based on a recently introduced spatial relation called enlacement, which generalizes the idea of surrounding for arbitrary objects, we propose a fuzzy landscape model that allows both to visualize and evaluate this relation directly in the image space, following different directions. Experiments on several characteristic examples highlight the interest and the behavior of this approach, allowing for rich interpretations of these complex spatial configurations.
Learning spatial relations and shapes for structural object description and scene recognition
2018, Pattern Recognition
Citation Excerpt :
Notably, Force Histograms are isotropic and less sensible to discretization issues, while also allowing to explicitly take into account the distance between objects, depending on the application needs. Force Histograms are involved in several application domains such as linguistic descriptions [10], scene matching [11] or content-based image retrieval [12–14]. Note that many other approaches were also proposed for modeling more specific spatial relations such as the “surrounded by” relation [15], the “between” relation [16,17], the “enlaced by” relation [18].
Being able to describe the content of an image, adapted to a particular application, is essential in various domains related to image analysis and pattern recognition. In this context, taking into account the spatial organization of objects is fundamental to increase both the understanding and the accuracy of the perceived similarity between images. In this article, we first present the Force Histogram Decomposition (FHD), a graph-based hierarchical descriptor that allows to characterize the spatial relations and shape information between the pairwise structural subparts of objects. Then, we propose a novel bags-of-features framework based on such descriptors, in order to produce discriminative structural features that are tailored for particular object classification tasks. An advantage of this learning procedure is its compatibility with traditional bags-of-features frameworks, allowing for hybrid representations gathering structural and local features. Experimental results obtained both on the recognition of structured objects from color images and on a parts-based scene recognition task highlight the interest of this approach.
An equivalent definition of the histogram of forces: Theoretical and algorithmic implications
2010, Pattern Recognition
The relative position between two 2-D spatial regions is often represented quantitatively by a force histogram. In the case of raster data, force histograms are usually computed in $O (KN \sqrt N)$ time, where N is the number of pixels in the image and K is the number of directions in which forces are considered. When the regions are defined as fuzzy sets instead of crisp sets, the complexity also depends on the number M of possible membership degrees. In this paper, we show that the force histogram can be defined in a completely different but equivalent way, one which leads to an $O (N \log N)$ algorithm, with complexity independent of K and M. Moreover, the equivalent definition is better adapted to the solving of theoretical issues. We use it here to determine the behavior of the force histogram towards any invertible affine transformation.
Combining color and spatial information for object recognition across illumination changes
2007, Pattern Recognition Letters
One of the most widely used approaches in the context of object recognition across illumination changes consists in comparing the images by means of the intersection between invariant histograms. However, this approach does not provide satisfying results with limited image databases. We propose to cope with the problem of illumination changes by analyzing simultaneously the color components of the pixels and their spatial arrangement in the image. For this purpose, we introduce the chromatic co-occurrence matrices to characterize the relationship between the color component levels of neighboring pixels. In order to compare two images acquired under different illuminations, these matrices are transformed into adapted co-occurrence matrices that are determined so that their intersection is higher when the two images contain the same object lit with different illuminations than when they contain different objects.
A new representation method of the relative position between objects in the image based on the histogram of position sensing forces
2024, Scientific Reports
Representation of the relative position between objects in the image: histogram of position sensing forces
2023, Research Square

View all citing articles on Scopus

View full text

Color and grey level object retrieval using a 3D representation of force histogram

Abstract

Introduction

Section snippets

Related woks

Description of the method

Usefulness

Butterfly database

Conclusion

Pattern Recognition

Pattern Recognition

Pattern Recognition Letters

Image and Vision Computing

Pattern Recognition Letters

Orientation radiograms for image retrieval: an alternative to segmentation

IEEE International Conference on Pattern Recognition, Vienna, Austria

Detection of interest point for image indexation

Third International Conference on Visual Information Systems, Amsterdam

Algorithm for computer control of a digital plotter

IBM Systems Journal

Invariant content-based image retrieval using a complete set of Fourier-Mellin descriptors

IEEE International Conference on Multimedia Computing and Systems

Statistical pattern recognition: a review

IEEE Transactions on PAMI

Image Storage and Retrieval Systems

Quantitative analysis of properties and spatial relations of fuzzy image regions

IEEE Transactions on Fuzzy Systems

STAR—a system for trademark archival and retrieval

Proceedings of the Second Asian Conference on Computer Vision, Singapore

A new set of texture features based on the Haar transform

Proceedings of the 11th IAPR International Conference on Pattern Recognition, The Hague, Netherlands

Texture features for browsing and retrieval of image data

IEEE Transactions on PAMI