Abstract
A major challenge when dealing with large collections of digital images is to find relevant objects, especially when no metadata on the objects is available. Content-based image retrieval (CBIR) addresses this problem but usually lacks query images that are good enough to express the user’s information need. Therefore, in Query-by-Sketch, CBIR has been considered with user provided sketches as query objects – but so far, this has suffered from the limitations of existing user interfaces. In this paper, we present a novel user interface for query by sketch that exploits emergent interactive paper and digital pen technology. Users can draw sketches on paper in a user-friendly way. Search can be started interactively from the paper front-end, due to a streaming interface from the digital pen to the underlying CBIR system. We present the implementation of the interactive paper/digital pen interface on top of QbS, our system for CBIR using sketches, and we present in detail the evaluation of the system on the basis of the MIRFLICKR-25000 image collection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agosti, M., Berretti, S., Brettlecker, G., del Bimbo, A., Ferro, N., Fuhr, N., Keim, D.A., Klas, C.-P., Lidy, T., Milano, D., Norrie, M.C., Ranaldi, P., Rauber, A., Schek, H.-J., Schreck, T., Schuldt, H., Signer, B., Springmann, M.: DelosDLMS: The Integrated DELOS Digital Library Management System. In: Digital Libraries: Research and Development – 1st Int. DELOS Conf., pp. 36–45 (February 2007)
Chalechale, A., Mertins, A., Naghdy, G.: Edge Image Description using Angular Radial Partitioning. IEE Vision, Image & Signal Processing 151(2) (2004)
Chen, T., Cheng, M.-M., Tan, P., Shamir, A., Hu, S.-M.: Sketch2photo: internet image montage. ACM Trans. Graph. 28(5) (2009)
del Bimbo, A., Pala, P.: Visual Image Retrieval by Elastic Matching of User Sketches. IEEE Trans. on Pattern Analysis & Machine Intelligence 19 (1997)
Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: PhotoSketch: A Sketch based Image Query and Compositing System. In: ACM SIGGRAPH (August 2009)
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., Yanker, P.: Query by Image and Video Content: The QBIC System. Computer 28(9), 23–32 (1995)
Hirata, K., Kato, T.: Query by Visual Example. In: Pirotte, A., Delobel, C., Gottlob, G. (eds.) EDBT 1992. LNCS, vol. 580, pp. 56–71. Springer, Heidelberg (1992)
Huiskes, M.J., Lew, M.S.: The MIR Flickr retrieval evaluation. In: ACM Int. Conf. on Multimedia Information Retrieval, MIR 2008 (2008)
Keysers, D., Deselaers, T., Gollan, C., Hermann, N.: Deformation models for image recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 29(8), 1422–1435 (2007)
Koutamanis, A.: Sketching with Digital Pen and Paper. In: Computer Aided Architectural Design Futures 2005 (June 2005)
Kurtenbach, G.: Pen-based computing. ACM Crossroads 16(4), 14–20 (2010)
Norrie, M.C., Signer, B., Weibel, N.: General framework for the rapid development of interactive paper applications. In: CoPADD 2006, Workshop on Collaborating Over Paper and Digital Documents, vol. 6, pp. 9–12 (2006)
Sikora, T.: The MPEG-7 Visual Standard for Content Description – an Overview. IEEE Trans. on Circuits & Systems for Video Technology 11(6), 696–702 (2001)
Springmann, M., Al Kabary, I., Schuldt, H.: Experiences with QbS: Challenges and Evaluation of Known Image Search based on User-Drawn Sketches. CS Technical Report CS-2010-001, University of Basel, Switzerland (2010), http://informatik.unibas.ch/research/publications_tec_report.html
Springmann, M., Al Kabary, I., Schuldt, H.: Image retrieval at memory’s edge: Known image search based on user-drawn sketches. In: Conference on Information and Knowledge Management (CIKM 2010), pp. 1465–1468 (October 2010)
Veltkamp, R.C., Tanase, M.: Content-Based Image Retrieval Systems: A Survey. Technical Report UU-CS-2000-34, Utrecht University (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kreuzer, R., Springmann, M., Kabary, I.A., Schuldt, H. (2012). An Interactive Paper and Digital Pen Interface for Query-by-Sketch Image Retrieval. In: Baeza-Yates, R., et al. Advances in Information Retrieval. ECIR 2012. Lecture Notes in Computer Science, vol 7224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28997-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-28997-2_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28996-5
Online ISBN: 978-3-642-28997-2
eBook Packages: Computer ScienceComputer Science (R0)