Content-based trademark retrieval system using a visually salient feature

Abstract

The ever-increasing number of registered trademarks has created greater demand for an automatic trademark retrieval system. In this paper, we present a method for such a system based on the image content, using a shape feature. Zernike moments of an image are used for a feature set. To retrieve similarly shaped trademarks quickly, we introduced the concept of a `visually salient feature' that dominantly affects the global shape of the trademarks. Experiments have been conducted on a database of 3000 trademark images. The retrieval speed was very fast and similar-shaped trademark retrieval results were very promising.

Introduction

An image retrieval system based on image content is a key area for building and managing large multimedia databases such as trademark and copyright, art galleries and museum, picture archiving and communication system (PACS), to name a few [1]. So, interest in the subject of content-based image retrieval has greatly increased for the past few years.

In this paper, we address the problem of visually similar trademark retrieval from a large trademark database using shape features. Trademarks are considered valuable intellectual properties and a key component of the goodwill of a business, since they represent not only the quality of actual products and services, but also the reputation of the manufacturer or the company. A registered trademark is protected through legal proceedings from misuse or imitation. Until now, since the total number of registered trademarks is over a million, the task of designing and registering a new trademark becomes more difficult without inadvertent infringement of copyright. So far, the current practice to classify trademarks is first by grouping the trademarks into several similar shapes according to a specific class order, followed by performing the matching process manually by human operator [2]. Therefore, the development of an on-line automatic trademark retrieval system for similar shapes becomes crucial.

In this paper, Zernike moment magnitudes (ZMMs) are used as a feature set. ZMMs are robust to noise or small variance of a pattern, and have rotation invariant characteristics. With a proper normalization method, scale invariance has also been achieved [3]. To retrieve similar shapes, we developed the `visually salient feature' that dominantly affects the global shape of the trademarks by ignoring minor details. The visually salient feature was determined by the probabilistic distribution model of a trademark database. To verify the performance of our proposed similar-shaped trademark retrieval system, several trademarks were submitted as a query image to a trademark database that contains 3000 trademarks.

We also considered pseudo-Zernike moments as a feature set. Pseudo-Zernike moments have properties analogous to Zernike moments. The performance of pseudo-Zernike moments was very similar to that of Zernike moments.

A trademark is a complex pattern, consisting of various text and image patterns. Trademarks can be divided into four types as shown in Fig. 1. Word-in-mark is a trademark that contains only characters or words in the mark. Character recognition or manual annotation is required to handle the type because the linguistic property (word structures and phonetics) is the key component of the type. On the other hand, a device-mark contains graphical or figurative elements only. Thus, the geometric shape is the key component for the type. Composite-mark consists of characters or words and graphical elements, while a complex-mark contains a complex image. Our current system focuses on retrieving device-mark types only.

Content-based image retrieval can be categorized into three parts: color-, texture- and shape-based retrieval. A number of techniques have appeared in the literature that deal with retrieval based on shape similarity. The QBIC (Query by Image Content) system allows queries on a large image database using various image contents such as color, texture, shape and position [4]. Jagadish proposed a similar shape retrieval method using the rectangular cover description 5, 6. Bigün et al. proposed an image retrieval system using orientation radiograms which are similar to the histogram of the edge directions [7]. Bimbo et al. presented an image retrieval system using a hierarchical model of the curve which is derived from its multi-scale analysis [8]. Mokhtarian et al. proposed the similar shape retrieval method using the maxima of curvature zero-crossing contours in the curvature scale space [9].

Several researchers have applied shape-based retrieval techniques to trademark images. Kato introduced a content-based similar shaped-trademark retrieval system [10]. This system used graphical features such as spatial outline of the overall figures, spatial, frequency, local correlation measure and local contrast measure. Cortelazzo et al. presented the trademark shape description method using a string matching technique [11]. Jain et al. proposed a hierarchical image retrieval system and tested the system on a trademark database 12, 13, 14. Their system uses a two-stage hierarchy: a fast screening stage using a histogram of the edge directions and invariant moments and a detailed matching stage using deformable template matching [15]. Eakins presented the SAFARI (shape analysis for automatic retrieval of images) system with curvature-based feature [16]. He developed a later version of SAFARI, so called ARTISAN (automatic retrieval of trademark images by shape analysis) that utilized more complex features: circularity, aspect ratio, discontinuity angle irregularity etc. [17]. Lam et al. presented a trademark retrieval system, STAR (system for trademark archival and retrieval) [18]. The system consisted of two parts to handle device-marks and word-in-marks. For device-marks, invariant moments and Fourier descriptors extracted from manually isolated distinct objects were used for shape features and the similarities among the trademarks are measured by a fuzzy thesaurus. For word-in-marks, the system performed sub-string matching and phonetics matching to retrieve trademarks that have similar linguistic properties.

Boundary based techniques such as boundary matching 11, 16, 17, Fourier descriptors [18] and multiscale curve matching 8, 9 may not be suitable for similar-shaped trademark retrieval, as the boundary shape can be changed drastically when there is a small crack like an opening or an object touching neighboring objects. For example, the shapes shown in Fig. 2(a) and (c) are very similar in human perception. The boundaries of these shapes, as shown in Fig. 2(b) and (d), however, are very different whether or not the inner star touches the outer circle. Furthermore, while most Fourier descriptor or curvature-based methods are based on a single boundary, a trademark consists of a complex pattern that has more than one boundary. Morphology-based preprocessing can be applied to remedy the problem, but it is not easy to determine the number of operations such as erosion or dilation to yield the optimum result for all trademark images. In addition, the resulting number of contours may also be very sensitive to the number of preprocessing steps.

A histogram of the edge directions 7, 12, 13, 14 has also been used in many systems. The drawback of this technique lies in the lack of discernment, because the histogram alone does not contain the information of edge location. For example, the images shown in Fig. 3(a)–(c), although their shapes are very different, have similar histograms of the edge directions as illustrated in Fig. 3(d).

The rest of this paper is organized as follows. In Section 2, we overview Zernike moments as a feature set. In Section 3, we present a probabilistic distribution model of the feature. Then, our retrieval method is described in Section 4. Experimental results are given in Section 5, and Section 6summarizes the paper.