research-article

A bag-of-objects retrieval model for web image search

Authors:

Shipeng LiAuthors Info & Claims

MM '12: Proceedings of the 20th ACM international conference on Multimedia

Pages 49 - 58

https://doi.org/10.1145/2393347.2393362

Published: 29 October 2012 Publication History

Abstract

Image search reranking has been an active research topic in recent years to boost the performance of the existing web image search engine which is mostly based on textual metadata of images. Various approaches have been proposed to rerank images for general queries and argue that, they may not necessarily be optimal for queries in specific domain, e.g., object queries, since the reranking algorithms are operated on whole images, instead of the relevant parts of images. In this paper, we propose a novel bag-of-objects retrieval model for image search reranking of object queries. Firstly, we employ a common object discovery algorithm to discover query-relevant objects from the search results returned by text-based image search engine. Then, the query and its result images are represented as a language model on the query relevant object vocabulary, based on which the ranking function can be derived. As the common object discovery is unreliable and may introduce noises, we propose to incorporate the attributes of the discovered objects, e.g., size, position, etc., into the ranking function through a linear model, and the weights on the object attributes can be learned. The experiments on two subsets of Web Queries dataset comprising object queries demonstrate that our approach can significantly outperform the existing reranking methods on object queries.

References

[1]

A. Bosch, A. Zisserman, and X. Muoz. Image classification using random forests and ferns. In CVPR, pages 1--8, 2007.

[2]

S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems, 30(1--7):107--117, 1998.

Digital Library

[3]

G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray. Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, volume 1, page 22, 2004.

[4]

T. Deselaers, B. Alexe, and V. Ferrari. Localizing objects while learning their appearance. ECCV, pages 452--466, 2010.

Digital Library

[5]

J. Feng, Y. Wei, L. Tao, C. Zhang, and J. Sun. Salient object detection by composition. In ICCV, pages 1028--1035, 2011.

Digital Library

[6]

R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman. Learning object categories from google's image search. In ICCV, volume 2, pages 1816--1823, 2005.

Digital Library

[7]

M. Fritz and B. Schiele. Decomposition, discovery and detection of visual categories using topic models. In CVPR, pages 1--8, 2008.

[8]

D. Hochbaum and V. Singh. An efficient algorithm for co-segmentation. In ICCV, pages 269--276, 2009.

[9]

L. Hohl, F. Souvannavong, B. Merialdo, and B. Huet. Enhancing latent semantic analysis video object retrieval with structural information. In ICIP, volume 3, pages 1609--1612, 2004.

[10]

W. Hsu, L. Kennedy, and S. Chang. Video search reranking through random walk over document-level context graph. In ACM Multimedia, pages 971--980, 2007.

Digital Library

[11]

Y. Jing and S. Baluja. Visualrank: Applying pagerank to large-scale image search. IEEE Trans. on PAMI, 30(11):1877--1890, 2008.

Digital Library

[12]

T. Joachims. Making large-scale svm learning practical. Advances in Kernel Methods Support Vector Learning, pages 169--184, 1999.

Digital Library

[13]

T. Joachims. Training linear svms in linear time. In ACM SIGKDD, pages 217--226, 2006.

Digital Library

[14]

A. Joulin, F. Bach, and J. Ponce. Discriminative clustering for image co-segmentation. In CVPR, pages 1943--1950, 2010.

[15]

G. Kim and A. Torralba. Unsupervised Detection of Regions of Interest using Iterative Link Analysis. In NIPS, 2009.

[16]

G. Kim, E. Xing, L. Fei-Fei, and T. Kanade. Distributed cosegmentation via submodular optimization on anisotropic diffusion. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 169--176, 2011.

Digital Library

[17]

J. Krapac, M. Allan, J. Verbeek, and F. Juried. Improving web image search results using query-relative classifiers. In CVPR, pages 1094--1101, 2010.

[18]

J. Lafferty and C. Zhai. Document language models, query models, and risk minimization for information retrieval. In ACM SIGIR, pages 111--119, 2001.

Digital Library

[19]

Y. Lee and K. Grauman. Object-graphs for context-aware category discovery. In CVPR, pages 1--8, 2010.

[20]

L. Li, H. Su, E. Xing, and L. Fei-Fei. Object bank: A high-level image representation for scene classification and semantic feature sparsification. NIPS, 2010.

[21]

Y. Liu, T. Mei, X. Hua, J. Tang, X. Wu, and S. Li. Learning to video search rerank via pseudo preference feedback. In ICME, pages 297--300, 2008.

[22]

D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91--110, 2004.

Digital Library

[23]

E. Oomoto and K. Tanaka. Ovid: Design and implementation of a video-object database system. IEEE Trans. on KDE, 5(4):629--643, 19

Digital Library

[24]

E. Parzen. On estimation of a probability density function and mode. The annals of mathematical statistics}, 33(3):1065--1076, 1962.

[25]

C. Rother, T. Minka, A. Blake, and V. Kolmogorov. Cosegmentation of image pairs by histogram matching-incorporating a global constraint into mrfs. In CVPR, volume 1, pages 993--1000, 2006.

Digital Library

[26]

S. Sav, G. Jones, H. Lee, N. O'Connor, and A. Smeaton. Interactive experiments in object-based retrieval. Image and Video Retrieval, pages 1--10, 2006.

Digital Library

[27]

S. Sav, H. Lee, A. Smeaton, N. O'Connor, and N. Murphy. Using video objects and relevance feedback in video retrieval. 6015:353--364, 2005.

[28]

F. Schroff, A. Criminisi, and A. Zisserman. Harvesting image databases from the web. In ICCV, pages 1--8, 2007.

[29]

X. Tian, Y. Lu, L. Yang, and Q. Tian. Learning to judge image search results. In ACM Multimedia, pages 363--372, 2011.

Digital Library

[30]

X. Tian, L. Yang, J. Wang, Y. Yang, X. Wu, and X. Hua. Bayesian video search reranking. In ACM Multimedia, pages 131--140, 2008.

Digital Library

[31]

X. Tian, L. Yang, X. Wu, and X. Hua. Visual reranking with local learning consistency. Advances in Multimedia Modeling, pages 163--173, 2010.

Digital Library

[32]

S. Vicente, V. Kolmogorov, and C. Rother. Cosegmentation revisited: Models and optimization. ECCV, pages 465--479, 2010.

Digital Library

[33]

R. Yan, A. Hauptmann, and R. Jin. Multimedia search with pseudo-relevance feedback. Image and Video Retrieval, pages 649--654, 2003.

Digital Library

[34]

L. Yang, B. Geng, Y. Cai, A. Hanjalic, and X. Hua. Object retrieval using visual query context. IEEE Trans. on Multimedia, (99):1--1, 2011.

Digital Library

[35]

L. Yang and A. Hanjalic. Supervised reranking for web image search. In ACM Multimedia, pages 183--192, 2010.

Digital Library

[36]

L. Yang and A. Hanjalic. learning from search engine and human supervision web image search. In ACM Multimedia, pages 1365--1368, 2011.

Digital Library

[37]

S. Zhang, Q. Tian, G. Hua, Q. Huang, and S. Li. Descriptive visual words and visual phrases for image applications. In ACM Multimedia, pages 75--84, 2009.

Digital Library

Cited By

Putzu LPiras LGiacinto G(2020)Convolutional neural networks for relevance feedback in content based image retrievalMultimedia Tools and Applications10.1007/s11042-020-09292-9Online publication date: 21-Jul-2020
https://doi.org/10.1007/s11042-020-09292-9
Niu YChen JGuo W(2018)Meta-metric for saliency detection evaluation metrics based on application preferenceMultimedia Tools and Applications10.1007/s11042-018-5863-277:20(26351-26369)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s11042-018-5863-2
Su ZZeng KLi HLuo X(2017)A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous CounterpartsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/306842713:2(1-21)Online publication date: 26-Apr-2017
https://dl.acm.org/doi/10.1145/3068427
Show More Cited By

Index Terms

A bag-of-objects retrieval model for web image search
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Learning from search engine and human supervision for web image search
MM '11: Proceedings of the 19th ACM international conference on Multimedia

Visual reranking aims at improving the precision of text-based Web image search. In this paper we propose to combine two learning strategies for deriving the reranking model: learning from search engine and learning from human supervision. The first ...
Supervised reranking for web image search
MM '10: Proceedings of the 18th ACM international conference on Multimedia

Visual search reranking that aims to improve the text-based image search with the help from visual content analysis has rapidly grown into a hot research topic. The interestingness of the topic stems mainly from the fact that the search reranking is an ...
Attribute-assisted reranking for web image retrieval
MM '12: Proceedings of the 20th ACM international conference on Multimedia

Image search reranking is an effective approach to refine the text-based image search result. Most existing reranking approaches are based on low-level visual features. In this paper, we propose to exploit semantic attributes for image search reranking. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '12: Proceedings of the 20th ACM international conference on Multimedia

October 2012

1584 pages

ISBN:9781450310895

DOI:10.1145/2393347

General Chairs:
Noboru Babaguchi
Osaka University, Japan
,
Kiyoharu Aizawa
The University of Tokyo, Japan
,
John Smith
IBM, USA
,
Program Chairs:
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Thomas Plagemann
University of Oslo, Norway
,
Xian-Sheng Hua
Microsoft, USA
,
Rong Yan
Facebook, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '12

Sponsor:

SIGMM

MM '12: ACM Multimedia Conference

October 29 - November 2, 2012

Nara, Japan

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
655
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Putzu LPiras LGiacinto G(2020)Convolutional neural networks for relevance feedback in content based image retrievalMultimedia Tools and Applications10.1007/s11042-020-09292-9Online publication date: 21-Jul-2020
https://doi.org/10.1007/s11042-020-09292-9
Niu YChen JGuo W(2018)Meta-metric for saliency detection evaluation metrics based on application preferenceMultimedia Tools and Applications10.1007/s11042-018-5863-277:20(26351-26369)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s11042-018-5863-2
Su ZZeng KLi HLuo X(2017)A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous CounterpartsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/306842713:2(1-21)Online publication date: 26-Apr-2017
https://dl.acm.org/doi/10.1145/3068427
Yanai K(2015)[Invited Paper] A Review of Web Image MiningITE Transactions on Media Technology and Applications10.3169/mta.3.1563:3(156-169)Online publication date: 2015
https://doi.org/10.3169/mta.3.156
Xiangyang Xu Ling Ge Tongwei Ren Gangshan Wu (2015)Adaptive integration of depth and color for objectness estimation2015 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME.2015.7177498(1-6)Online publication date: Jun-2015
https://doi.org/10.1109/ICME.2015.7177498
Ju RLiu YRen TGe LWu G(2015)Depth-aware salient object detection using anisotropic center-surround differenceImage Communication10.1016/j.image.2015.07.00238:C(115-126)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1016/j.image.2015.07.002
Zhao SMa JCui C(2015)Multimodal-Based Supervised Learning for Image Search RerankingWeb-Age Information Management10.1007/978-3-319-21042-1_11(135-147)Online publication date: 6-Jun-2015
https://doi.org/10.1007/978-3-319-21042-1_11
Yang YYang LWu GLi S(2014)Image Relevance Prediction Using Query-Context Bag-of-Object Retrieval ModelIEEE Transactions on Multimedia10.1109/TMM.2014.232683616:6(1700-1712)Online publication date: Oct-2014
https://doi.org/10.1109/TMM.2014.2326836
Huang SWang WZhang H(2014)Retrieving images using saliency detection and graph matching2014 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP.2014.7025624(3087-3091)Online publication date: Oct-2014
https://doi.org/10.1109/ICIP.2014.7025624

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten