SKETCHify – An Adaptive Prominent Edge Detection Algorithm for Optimized Query-by-Sketch Image Retrieval

Abstract

Query-by-Sketch image retrieval, unlike content based image retrieval following a Query-by-Example approach, uses human-drawn binary sketches as query objects, thereby eliminating the need for an initial query image close enough to the users’ information need. This is particularly important when the user is looking for a known image, i.e., an image that has been seen before. So far, Query-by-Sketch has suffered from two main limiting factors. First, users tend to focus on the objects’ main contours when drawing binary sketches, while ignoring any texture or edges inside the object(s) and in the background. Second, the users’ limited ability to sketch the known item being searched for in the correct position, scale and/or orientation. Thus, effective Query-by-Sketch systems need to allow users to concentrate on the main contours of the main object(s) they are searching for and, at the same time, tolerate such inaccuracies. In this paper, we present SKETCHify, an adaptive algorithm that is able to identify and isolate the prominent objects within an image. This is achieved by applying heuristics to detect the best edge map thresholds for each image by monitoring the intensity, spatial distribution and sudden spike increase of edges with the intention of generating edge maps that are as close as possible to human-drawn sketches. We have integrated SKETCHify into QbS, our system for Query-by-Sketch image retrieval, and the results show a significant improvement in both retrieval rank and retrieval time when exploiting the prominent edges for retrieval, compared to Query-by-Sketch relying on normal edge maps. Depending on the quality of the query sketch, SKETCHify even allows to provide invariances with regard to position, scale and rotation in the retrieval process. For the evaluation, we have used images from the MIRFLICKR-25K dataset and a free clip art collection of similar size.