Content-based trademark retrieval system using a visually salient feature
Introduction
An image retrieval system based on image content is a key area for building and managing large multimedia databases such as trademark and copyright, art galleries and museum, picture archiving and communication system (PACS), to name a few [1]. So, interest in the subject of content-based image retrieval has greatly increased for the past few years.
In this paper, we address the problem of visually similar trademark retrieval from a large trademark database using shape features. Trademarks are considered valuable intellectual properties and a key component of the goodwill of a business, since they represent not only the quality of actual products and services, but also the reputation of the manufacturer or the company. A registered trademark is protected through legal proceedings from misuse or imitation. Until now, since the total number of registered trademarks is over a million, the task of designing and registering a new trademark becomes more difficult without inadvertent infringement of copyright. So far, the current practice to classify trademarks is first by grouping the trademarks into several similar shapes according to a specific class order, followed by performing the matching process manually by human operator [2]. Therefore, the development of an on-line automatic trademark retrieval system for similar shapes becomes crucial.
In this paper, Zernike moment magnitudes (ZMMs) are used as a feature set. ZMMs are robust to noise or small variance of a pattern, and have rotation invariant characteristics. With a proper normalization method, scale invariance has also been achieved [3]. To retrieve similar shapes, we developed the `visually salient feature' that dominantly affects the global shape of the trademarks by ignoring minor details. The visually salient feature was determined by the probabilistic distribution model of a trademark database. To verify the performance of our proposed similar-shaped trademark retrieval system, several trademarks were submitted as a query image to a trademark database that contains 3000 trademarks.
We also considered pseudo-Zernike moments as a feature set. Pseudo-Zernike moments have properties analogous to Zernike moments. The performance of pseudo-Zernike moments was very similar to that of Zernike moments.
A trademark is a complex pattern, consisting of various text and image patterns. Trademarks can be divided into four types as shown in Fig. 1. Word-in-mark is a trademark that contains only characters or words in the mark. Character recognition or manual annotation is required to handle the type because the linguistic property (word structures and phonetics) is the key component of the type. On the other hand, a device-mark contains graphical or figurative elements only. Thus, the geometric shape is the key component for the type. Composite-mark consists of characters or words and graphical elements, while a complex-mark contains a complex image. Our current system focuses on retrieving device-mark types only.
Content-based image retrieval can be categorized into three parts: color-, texture- and shape-based retrieval. A number of techniques have appeared in the literature that deal with retrieval based on shape similarity. The QBIC (Query by Image Content) system allows queries on a large image database using various image contents such as color, texture, shape and position [4]. Jagadish proposed a similar shape retrieval method using the rectangular cover description 5, 6. Bigün et al. proposed an image retrieval system using orientation radiograms which are similar to the histogram of the edge directions [7]. Bimbo et al. presented an image retrieval system using a hierarchical model of the curve which is derived from its multi-scale analysis [8]. Mokhtarian et al. proposed the similar shape retrieval method using the maxima of curvature zero-crossing contours in the curvature scale space [9].
Several researchers have applied shape-based retrieval techniques to trademark images. Kato introduced a content-based similar shaped-trademark retrieval system [10]. This system used graphical features such as spatial outline of the overall figures, spatial, frequency, local correlation measure and local contrast measure. Cortelazzo et al. presented the trademark shape description method using a string matching technique [11]. Jain et al. proposed a hierarchical image retrieval system and tested the system on a trademark database 12, 13, 14. Their system uses a two-stage hierarchy: a fast screening stage using a histogram of the edge directions and invariant moments and a detailed matching stage using deformable template matching [15]. Eakins presented the SAFARI (shape analysis for automatic retrieval of images) system with curvature-based feature [16]. He developed a later version of SAFARI, so called ARTISAN (automatic retrieval of trademark images by shape analysis) that utilized more complex features: circularity, aspect ratio, discontinuity angle irregularity etc. [17]. Lam et al. presented a trademark retrieval system, STAR (system for trademark archival and retrieval) [18]. The system consisted of two parts to handle device-marks and word-in-marks. For device-marks, invariant moments and Fourier descriptors extracted from manually isolated distinct objects were used for shape features and the similarities among the trademarks are measured by a fuzzy thesaurus. For word-in-marks, the system performed sub-string matching and phonetics matching to retrieve trademarks that have similar linguistic properties.
Boundary based techniques such as boundary matching 11, 16, 17, Fourier descriptors [18] and multiscale curve matching 8, 9 may not be suitable for similar-shaped trademark retrieval, as the boundary shape can be changed drastically when there is a small crack like an opening or an object touching neighboring objects. For example, the shapes shown in Fig. 2(a) and (c) are very similar in human perception. The boundaries of these shapes, as shown in Fig. 2(b) and (d), however, are very different whether or not the inner star touches the outer circle. Furthermore, while most Fourier descriptor or curvature-based methods are based on a single boundary, a trademark consists of a complex pattern that has more than one boundary. Morphology-based preprocessing can be applied to remedy the problem, but it is not easy to determine the number of operations such as erosion or dilation to yield the optimum result for all trademark images. In addition, the resulting number of contours may also be very sensitive to the number of preprocessing steps.
A histogram of the edge directions 7, 12, 13, 14 has also been used in many systems. The drawback of this technique lies in the lack of discernment, because the histogram alone does not contain the information of edge location. For example, the images shown in Fig. 3(a)–(c), although their shapes are very different, have similar histograms of the edge directions as illustrated in Fig. 3(d).
The rest of this paper is organized as follows. In Section 2, we overview Zernike moments as a feature set. In Section 3, we present a probabilistic distribution model of the feature. Then, our retrieval method is described in Section 4. Experimental results are given in Section 5, and Section 6summarizes the paper.
Section snippets
Zernike moments as a feature set
Zernike moments are complex orthogonal moments whose magnitude has rotational invariant property 19, 20, 21, 22. Teh et al. compared several moments in terms of:
- 1.
sensitivity to image noise;
- 2.
aspects of information redundancy;
- 3.
capability for image representation.
Trademark data collection
Three thousand Korean and world trademarks were collected from the reference 24, 25 by scanner. All trademark images were binarized and normalized to the size of 100×100 pixels by maximum extent circle (MEC) method [26]. Color was not considered in our current system. ZMMs were computed by the lookup-table method [26] and stored in a database up to the order of n=17. The total number of moments corresponding to n=17, is 90 [23].
Distribution model
The distribution model of features plays an important role in our
Retrieving similar trademarks using the most salient feature
With 90 Zernike moment features to use for retrieving similar trademarks from database, one of the common practice is to make use of the Euclidean distance in feature space, along with a proper weight on each feature [22]. The one whose distance to the query is the minimum will be selected. However, when the number of patterns in a database to compare is very large, the number of features should also be increased, and this naive approach may pose a computational problem. In addition, as the
Experimental results
To verify the performance of our proposed similar-shaped trademark retrieval scheme, several trademarks shown in Fig. 9 were submitted as query image to the trademark database that consists of 3000 trademarks. The performance is estimated by the following subjective and objective criteria;
- 1.
How well can similar-shaped trademarks be retrieved in accordance with the human perception.
- 2.
How well can the same trademarks be retrieved in the presence of noise or deformation.
When a query was submitted to
Summary and discussion
In this paper, we presented a new content-based similar shape retrieval method for trademarks using Zernike moments. The advantages of using MSF are twofold: quick retrieval of similar trademarks, and robustness to the minor transformation of the shape.
Since the radial complexity and the degree of circular symmetry of the shape are reflected in the MSF, the retrieved trademarks using the MSF will have similar characteristics. The MSF of the trademark was barely affected by noise or deformation
Acknowledgements
This work was supported by the Electronics and Telecommunications Research Institute under grant 97202.
References (28)
- et al.
Trademark shapes description by stringmatching techniques
Pattern Recog.
(1994) - et al.
Image retrieval using color and shape
Pattern Recog.
(1996) - et al.
A survey of moment-based techniques for unoccluded object representation and recognition
CVGIP: Graph. Models Image Process.
(1992) - et al.
Rotation invariant image recognition using features selected via a systematic method
Pattern Recog.
(1990) - et al.
Content-based image retrieval systems
IEEE Comput.
(1995) - B. Andrews, U.S. Patent and Trademark Office ORBIT Trademark Retrieval System, T-term User Guide, Examining Attorney's...
- Kim W.Y., Yuan P.O., A practical pattern recognition system for translation, scale and rotation invariance, in:...
- et al.
Query by image and video content: the QBIC system
IEEE Comput.
(1995) - H.V. Jagadish, A retrieval technique for similar shapes, in: Proceedings of ACM SIGMOD, 1991, pp....
- et al.
A new method of image compression using irreducible covers of maximal rectangles
IEEE Trans. Software Engng
(1988)
Cited by (109)
Finite multi-dimensional generalized Gamma Mixture Model Learning for feature selection
2020, Learning Control: Applications in Robotics and Complex Dynamical SystemsA deep one-shot network for query-based logo retrieval
2019, Pattern RecognitionCitation Excerpt :Thereafter many logo-related works were carried out with the methods of content-based indexing and retrieval in trademark databases. The main goal is to assist in trademark infringement detection by checking a newly designed trademark with registered logos in archives [30–33]. The task of trademark recognition in videos is inherently harder due to loss of quality of original logos during processing (e.g. color sub-sampling, video interlacing, motion blur, etc.).
Multi-faceted assessment of trademark similarity
2016, Expert Systems with ApplicationsLogo and seal based administrative document image retrieval: A survey
2016, Computer Science ReviewCitation Excerpt :Based on this categorization outline, local features used for logo recognition include: features extracted from local zone [54], differential invariants [5], negative shape features [6], primitives (line segments) [55], curvature and distance from centroid point [12,7], SIFT and SURF descriptors derived from Hessian-affine interest points [56–58], horizontal gaps per total area, vertical gaps per total area, ratio of hole area to total area [59,60], color [61], Delaunay triangulation of components/local features [61,60], bag-of-words features [60], edge based features extracted using GHT [62], Fourier coefficients of segmented boundary curves [63], rectangle features extracted from integral image [64], etc. Global features utilized in literature for logo recognition are: different moments (Zernike, Tchebichef, invariant, radial) [54,11,59,65,2,7], projection profiles [66], bispectral [66], gradient features extracted from contour points [67–69], algebraic invariants [5], wavelet-based features [6,70], circularity, eccentricity and rectangularity [59], geometric topology features extracted from components [61], area, isolation, deviation, symmetry, centralization, complexity and 2-level contour representation strings [71], global shape based features (circle, rectangle, triangle, ellipse, polygon, and B-spline) extracted from Fourier descriptor [62,72], shape context [73,3], features extracted from raw image/vector data [74], curvature [22], template matching [75], etc. Different classification methods employed for logo (well segmented) recognition/ classification can be categorized into non-parametric and parametric classification techniques.
An effective vector model for global-contrast-based saliency detection
2015, Journal of Visual Communication and Image RepresentationCitation Excerpt :This selective visual ability allows brain and visual system to break through the bottleneck of information-processing, because it is hypothesized that human visual system only concentrates on the most unusual parts of the massive sensory incoming information [1]. In the computer vision field, it is critical to simulate this ability to extract saliency maps because the maps are key to the applications in images and videos including perceptual video retargeting [2,3], perceptual object segmentation [4,5], adaptive coding [6], object recognition [7,8], and image retrieval [9]. Visual saliency is a perceptual state or quality that makes an item prominent from its neighborhoods.