Skip to main content
SpringerLink
Log in

Robust Sketch-Based Image Retrieval by Saliency Detection

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9516))

Included in the following conference series:

Abstract

Sketch-based image retrieval (SBIR) has been extensively studied for decades because sketch is one of the most intuitive ways to describe ideas. However, the large expressional gap between hand-drawn sketches and natural images with small-scale complex structures is the fundamental challenge for SBIR systems. We present a novel framework to efficiently retrieve images with a query sketch based on saliency detection. In order to extract primary contours of the scene and depress textures, a hierarchical saliency map is computed for each image. Object contours are extracted from the saliency map instead of the original natural image. Histograms of oriented gradients (HOG) are extracted at multiple scales on a dense gradient field. Using a bag-of-visual-words representation and an inverted index structure, our system efficiently retrieves images by sketches. The experimental results conducted on a dataset of 15 k photographs demonstrate that our method performs well for a wide range of natural scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Canny, J.: A computational approach to edge detection. IEEE TPAMI 6, 679–698 (1986)

    Article  Google Scholar 

  2. Cao, Y., Wang, H., Wang, C., Li, Z., Zhang, L., Zhang, L.: Mindfinder: interactive sketch-based image search on millions of images. In: ACMMM (2010)

    Google Scholar 

  3. Cao, Y., Wang, C., Zhang, L., Zhang, L.: Edgel index for large-scale sketch-based image search. In: IEEE CVPR, pp. 761–768 (2011)

    Google Scholar 

  4. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, pp. 886–893 (2005)

    Google Scholar 

  5. Dance, C., Willamowski, J., Fan, L., Bray, C., Csurka, G.: Visual categorization with bags of keypoints. In: ECCV International Workshop on Statistical Learning in Computer Vision (2004)

    Google Scholar 

  6. Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: A descriptor for large scale image retrieval based on sketched feature lines. In: Eurographics Symposium on Sketch-Based Interfaces and Modeling, pp. 29–38 (2009)

    Google Scholar 

  7. Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: An evaluation of descriptors for large-scale image retrieval from sketched feature lines. Comput. Graph. 34(5), 482–498 (2010)

    Article  Google Scholar 

  8. Hu, R., Barnard, M., Collomosse, J.: Gradient field descriptor for sketch based retrieval and localization. ICIP 10, 1025–1028 (2010)

    Google Scholar 

  9. Hu, R., Collomosse, J.: A performance evaluation of gradient field hog descriptor for sketch based image retrieval. CVIU 117(7), 790–806 (2013)

    Google Scholar 

  10. Jacobs, C.E., Finkelstein, A.: Fast multiresolution image querying. In: ACM Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp. 277–286 (1995)

    Google Scholar 

  11. Parui, S., Mittal, A.: Similarity-Invariant sketch-based image retrieval in large databases. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part VI. LNCS, vol. 8694, pp. 398–414. Springer, Heidelberg (2014)

    Google Scholar 

  12. Saavedra, J.M., Bustos, B.: Sketch-based image retrieval using keyshapes. Multimedia Tools Appl. 73(3), 2033–2062 (2014)

    Article  Google Scholar 

  13. Smith, J.R., Chang, S.F.: Visualseek: a fully automated content-based image query system. In: Proceedings of the Fourth ACM International Conference on Multimedia, pp. 87–98 (1997)

    Google Scholar 

  14. Tencer, L., Renakova, M., Cheriet, M.: Sketch-based retrieval of document illustrations and regions of interest. In: 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 728–732 (2013)

    Google Scholar 

  15. Wang, S., Zhang, J., Xu, T., Miao, Z.: Sketch-based image retrieval through hypothesis driven object boundary selection with HLR descriptor. IEEE Trans. Multimedia 17, 1045–1057 (2015)

    Article  Google Scholar 

  16. Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: CVPR (2013)

    Google Scholar 

  17. Zhou, R., Chen, L., Zhang, L.: Sketch-based image retrieval on a large scale database. In: Proceedings of the 20th ACM international conference on Multimedia, pp. 973–976 (2012)

    Google Scholar 

Download references

Acknowledgments

We would like to thank the anonymous reviewers. This work was supported by the National Natural Science Foundation of China (NSFC) under Nos. 61472377 and 61331017, and the Fundamental Research Funds for the Central Universities under No. WK2100060011.

Author information

Authors and Affiliations

  1. CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, University of Science and Technology of China, Hefei, China

    Xiao Zhang & Xuejin Chen

Authors
  1. Xiao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuejin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuejin Chen .

Editor information

Editors and Affiliations

  1. University of Texas at San Antonio, San Antonio, USA

    Qi Tian

  2. Dept. of Information Engineering, University of Trento, Povo, Trento, Italy

    Nicu Sebe

  3. EECS, University of Central Florida, Orlando, Florida, USA

    Guo-Jun Qi

  4. EURECOM, Sophia-Antipolis, France

    Benoit Huet

  5. Hefei University of Technology, Hefei, Anhui, China

    Richang Hong

  6. School of Computing and Information, Hefei University of Technology, Hefei, Anhui, China

    Xueliang Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Zhang, X., Chen, X. (2016). Robust Sketch-Based Image Retrieval by Saliency Detection. In: Tian, Q., Sebe, N., Qi, GJ., Huet, B., Hong, R., Liu, X. (eds) MultiMedia Modeling. MMM 2016. Lecture Notes in Computer Science(), vol 9516. Springer, Cham. https://doi.org/10.1007/978-3-319-27671-7_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27671-7_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27670-0

  • Online ISBN: 978-3-319-27671-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation