research-article

Image search by concept map

Authors:

Xian-Sheng Hua,

Shipeng LiAuthors Info & Claims

SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

Pages 275 - 282

https://doi.org/10.1145/1835449.1835497

Published: 19 July 2010 Publication History

Abstract

In this paper, we present a novel image search system, image search by concept map. This system enables users to indicate not only what semantic concepts are expected to appear but also how these concepts are spatially distributed in the desired images. To this end, we propose a new image search interface to enable users to formulate a query, called concept map, by intuitively typing textual queries in a blank canvas to indicate the desired spatial positions of the concepts. In the ranking process, by interpreting each textual concept as a set of representative visual instances, the concept map query is translated into a visual instance map, which is then used to evaluate the relevance of the image in the database. Experimental results demonstrate the effectiveness of the proposed system.

References

[1]

J. Sivic, B. C. Russell, A. Efros, A. Zisserman, and W. T. Freeman. Discovering object categories in image collections. In ICCV '05, 2005.

[2]

M. B. Blaschko and C. H. Lampert. Learning to localize objects with structured output regression. In ECCV '08, pages 2--15, 2008.

Digital Library

[3]

T. Chen, M.-M. Cheng, P. Tan, A. Shamir, and S.-M. Hu. Sketch2photo: internet image montage. In SIGGRAPH Asia '09, pages 1--10, 2009.

Digital Library

[4]

J. Cui, F. Wen, and X. Tang. Intentsearch: interactive on-line image search re-ranking. In MM '08, pages 997--998, 2008.

Digital Library

[5]

C. Desai, D. Ramanan, and C. Fowlkes. Discriminative models for multi-class object layout. In ICCV '09, 2009.

[6]

D. Dueck and B. J. Frey. Non-metric affinity propagation for unsupervised image categorization. In ICCV '07, 2007.

[7]

M. J. Egenhofer. Spatial-query-by-sketch. In IEEE Symposium on Visual Languages, pages 60--67, 1996.

Digital Library

[8]

Flickr. http://www.flickr.com/.

[9]

B. J. Frey and D. Dueck. Clustering by passing messages between data points. Science, 315(5814):972, 2007.

[10]

Google Image Search. http://images.google.com/.

[11]

K. Jarvelin and J. Kekalainen. IR evaluation methods for retrieving highly relevant documents. In SIGIR '00, pages 41--48, 2000.

Digital Library

[12]

M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: State of the art and challenges. TOMCCAP, 2(1):1--19, 2006.

Digital Library

[13]

T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, and H.-Y. Shum. Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell., to appear.

Digital Library

[14]

X. Liu, B. Cheng, S. Yan, J. Tang, T. S. Chua, and H. Jin. Label to region by bi-layer sparsity priors. In MM '09, pages 115--124, 2009.

Digital Library

[15]

Y. Luo, W. Liu, J. Liu, and X. Tang. Mqsearch: image search by multi-class query. In CHI '08, pages 49--52, 2008.

Digital Library

[16]

Microsoft Bing Image Search. http://images.bing.com/.

[17]

K. Mikolajczyk and C. Schmid. Scale & affine invariant interest point detectors. IJCV, 60(1):63--86, 2004.

Digital Library

[18]

G. P. Nguyen and M. Worring. Optimization of interactive visual-similarity-based search. TOMCCAP, 4(1), 2008.

Digital Library

[19]

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. LabelMe: a database and web-based tool for image annotation. IJCV, 77(1):157--173, 2008.

Digital Library

[20]

J. R. Smith and S.-F. Chang. Visualseek: A fully automated content-based image query system. In MM '96, pages 87--98, 1996.

Digital Library

[21]

J. Wang, X. Hua, and Y. Zhao. Color-Structured Image Search. MSR-TR-2009-82. July 7, 2009.

[22]

H. Xu, J. Wang, X. Hua, and S. Li. Interactive Image Search by 2D Semantic Map. In WWW '10, 2010.

Digital Library

[23]

R. Yan, A. Natsev, and M. Campbell. Multi-query interactive image and video retrieval -: theory and practice. In CIVR '08, pages 475--484, 2008.

Digital Library

[24]

R. Yan, J. Yang, and A. G. Hauptmann. Learning query-class dependent weights in automatic video retrieval. In MM '04, pages 548--555, 2004.

Digital Library

[25]

E. Zavesky and S.-F. Chang. Cuzero: embracing the frontier of interactive visual search for informed users. In MIR '08, pages 237--244, 2008.

Digital Library

[26]

Z.-J. Zha, L. Yang, T. Mei, M. Wang, and Z. Wang. Visual query suggestion. In MM '09, pages 15--24, 2009.

Digital Library

Cited By

Liu YMedlar AGłowacka D(2024)Sample, Nudge and Rank: Exploiting Interpretable GAN Controls for Exploratory SearchProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645156(582-596)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645156
Barnaby CChen QWang CDillig I(2024)PhotoScout: Synthesis-Powered Multi-Modal Image SearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642319(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642319
Zhu XLiu L(2023)Diverse image search with explanationsMultimedia Tools and Applications10.1007/s11042-023-16393-883:8(23067-23082)Online publication date: 10-Aug-2023
https://doi.org/10.1007/s11042-023-16393-8
Show More Cited By

Index Terms

Image search by concept map
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing systems and tools
2. Information systems
  1. Information retrieval
    1. Document representation
  2. World Wide Web

Recommendations

Interactive Image Search by Color Map

The availability of large-scale images from the Internet has made the research on image search attract a lot of attention. Text-based image search engines, for example, Google/Microsoft Bing/Yahoo! image search engines using the surrounding text, have ...
Using a concept map to evaluate exploratory search
IIiX '10: Proceedings of the third symposium on Information interaction in context

Users often conduct exploratory searches on the Web to solve complex problems, but how for evaluating the effectiveness of a search, especially in a user-centered approach, remains an open question. We propose a user-centered method to evaluate the ...
Interactive image search by 2D semantic map
WWW '10: Proceedings of the 19th international conference on World wide web

In this demo, we present a novel interactive image search system, image search by 2D semantic map. This system enables users to indicate what semantic concepts are expected to appear and even how these concepts are spatially distributed in the desired ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

July 2010

944 pages

ISBN:9781450301534

DOI:10.1145/1835449

General Chairs:
Fabio Crestani
University of Lugano, CH
,
Stéphane Marchand-Maillet
University of Geneva, CH
,
Program Chairs:
Hsin-Hsi Chen
National Taiwan University, TW
,
Efthimis N. Efthimiadis
University of Washington, USA
,
Jacques Savoy
University of Neuchatel, CH

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '10

Sponsor:

SIGIR

SIGIR '10: The 33rd International ACM SIGIR conference on research and development in Information Retrieval

July 19 - 23, 2010

Geneva, Switzerland

Acceptance Rates

SIGIR '10 Paper Acceptance Rate 87 of 520 submissions, 17%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
1,002
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YMedlar AGłowacka D(2024)Sample, Nudge and Rank: Exploiting Interpretable GAN Controls for Exploratory SearchProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645156(582-596)Online publication date: 18-Mar-2024
Barnaby CChen QWang CDillig I(2024)PhotoScout: Synthesis-Powered Multi-Modal Image SearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642319(1-15)Online publication date: 11-May-2024
Zhu XLiu L(2023)Diverse image search with explanationsMultimedia Tools and Applications10.1007/s11042-023-16393-883:8(23067-23082)Online publication date: 10-Aug-2023
Yan WLi YYang HHuang BPan Z(2022)Semantic-aware multi-task learning for image aesthetic quality assessmentConnection Science10.1080/09540091.2022.214790234:1(2689-2713)Online publication date: 22-Nov-2022
Liu JShah C(2022)Interactive IR User Study Design, Evaluation, and ReportingundefinedOnline publication date: 10-Mar-2022
Kilickaya MSmeulders A(2021)Structured Visual Search via Composition-aware Learning2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00174(1700-1709)Online publication date: Jan-2021
Ma JPang SYang BZhu JLi Y(2020)Spatial-Content Image Search in Complex Scenes2020 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV45572.2020.9093427(2492-2500)Online publication date: Mar-2020
Liu JShah C(2019)Interactive IR User Study Design, Evaluation, and ReportingSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S00923ED1V01Y201905ICR06711:2(i-75)Online publication date: 3-Jun-2019
Mailthody VQureshi ZLiang WFeng Zde Gonzalo SLi YFranke HXiong JHuang JHwu W(2019)DeepStoreProceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3352460.3358320(224-238)Online publication date: 12-Oct-2019
Furuta RInoue NYamasaki T(2019)Efficient and interactive spatial-semantic image retrievalMultimedia Tools and Applications10.1007/s11042-018-7148-178:13(18713-18733)Online publication date: 1-Jul-2019
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten