Article

Scalable search-based image annotation of personal images

Authors:

Hong-Jiang ZhangAuthors Info & Claims

MIR '06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval

Pages 269 - 278

https://doi.org/10.1145/1178677.1178714

Published: 26 October 2006 Publication History

Abstract

With the prevalence of digital cameras, more and more people have considerable digital images on their personal devices. As a result, there are increasing needs to effectively search these personal images. Automatic image annotation may serve the goal, for the annotated keywords could facilitate the search processes. Although many image annotation methods have been proposed in recent years, their effectiveness on arbitrary personal images is constrained by their limited scalability, i.e. limited lexicon of small-scale training set. To be scalable, we propose a search-based image annotation (SBIA) algorithm that is analogous to Web page search. First, content-based image retrieval (CBIR) technology is used to retrieve a set of visually similar images from a large-scale Web image set. Then, a text-based keyword search (TBKS) technique is used to obtain a ranked list of candidate annotations for each retrieved image. Finally, a fusion algorithm is used to combine the ranked lists into the final annotation list. The application of both efficient search technologies and Web-scale image set guarantees the scalability of the proposed algorithm. Experimental results on U. Washington dataset show not only the effectiveness and efficiency of the proposed algorithm but also the advantage of image retrieval using annotation results over that using visual features.

References

[1]

http://www.cs.washington.edu/research/imagedatabase/groundtruth/

[2]

http://www.photosig.com

[3]

Baeza-Yates, R. A. and Ribeiro-Neto, B. 1999 Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc.

Digital Library

[4]

Blei, D. M. and Jordan, M. I. 2003. Modeling annotated data. In Proceedings of the 26th Annual international ACM SIGIR Conference on Research and Development in informaion Retrieval. New York, NY, 127--134.

Digital Library

[5]

Brown, P. F., deSouza, P. V., Mercer, R. L., Pietra, V. J., and Lai, J. C. 1992. Class-based n-gram models of natural language. Comput. Linguist. 18, 4 (Dec. 1992), 467--479.

Digital Library

[6]

Chang, E., Kingshy, G., Sychay, G., and Wu, G. CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines. IEEE Trans. on CSVT, 13(1):26--38, Jan. 2003.

Digital Library

[7]

Cusano, C., Ciocca, G., and Schettini, R. Image annotation using SVM. In Proc. Of Internet imaging IV, Vol. SPIE, 2004.

[8]

Duygulu, P. and Barnard, K. Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In Seventh European Conference on Computer Vision, 4:97--112, 2002.

Digital Library

[9]

Fan, X., Xie, X., Li, Z., Li, M., and Ma, W. Y. Photo-to- Search: Using Multimodal Queries to Search the Web from Mobile Devices. 7th ACM SIGMM Workshop on MIR, 2005.

Digital Library

[10]

Feng, S. L., Manmatha, R., and Lavrenko, V. Multiple bernoulli relevance models for image and video annotation. In The International Conference on Computer Vision and Pattern Recognition, Washington, DC, June, 2004.

Digital Library

[11]

Ferhatosmanoglu, H., Tuncel, E., Agrawal, D., and Abbadi, A. E. Approximate nearest neighbor searching in multimedia databases. In Proceedings of the 17th IEEE Int'l. Conference on Data Engineering, Heidelberg, Germany, April, 2001, 2-6, pp. 503--511.

Digital Library

[12]

Jeon, J., Lavrenko, V., and Manmatha, R. Automatic Image Annotation and Retrieval Using Cross-media Relevance Models. In Proc. of ACM SIGIR conference on Research and development in information retrieval, pp. 119--126, July, 2003.

Digital Library

[13]

Lavrenko, V., Manmatha, R., and Jeon, J. A Model for Learning the Semantics of Pictures. In Proc. of the 17th Annual Conf. on Neural Information Processing Systems, 2003.

[14]

Li, J. and Wang, J. Z. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. on PAMI, 25(10), Oct. 2003.

Digital Library

[15]

Miller, G. A. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (Nov. 1995), 39--41.

Digital Library

[16]

Mori, Y., Takahashi, H., and Oka, R. Image-to-word transformation based on dividing and vector quantizing images with words. In MISRM'99 First International Workshop on Multimedia Intelligent Storage and Retrieval Management, 1999.

[17]

Page, L., Brin, S., Motwani, R., and Winograd, T. The Pagerank Citation Ranking: Bringing Order to the web, technical report, Stanford University, Stanford, CA, 1998.

[18]

Robertson, S. E. and Walker, S. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 345--354. Springer-Verlag, 1994.

Digital Library

[19]

Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., and Jain, R. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349--1380, 2000.

Digital Library

[20]

Wang, X. J., Zhang, L., Jing, F., and Ma, W. Y. AnnoSearch: Image Auto-Annotation by Search. In The International Conference on Computer Vision and Pattern Recognition, New York, June, 2006.

Digital Library

[21]

Yanai, K. and Barnard, K. 2005. Image region entropy: a measure of "visualness" of web images associated with one concept. In Proceedings of the 13th Annual ACM international Conference on Multimedia. New York, NY, 419--422.

Digital Library

[22]

Yeh, T., Tollmar, K., and Darrell, T. Searching the Web with Mobile Images for Location Recognition. In The International Conference on Computer Vision and Pattern Recognition, 2004, pp. 76--81.

Digital Library

[23]

Zeng, H., He, Q., Chen, Z., Ma, W., and Ma, J. 2004. Learning to cluster web search results. SIGIR, 2004. New York, NY, 210--217.

Digital Library

[24]

Zhang, L., Hu, Y., Li, M., Ma, W., and Zhang, H. 2004. Efficient propagation for face annotation in family albums. In Proceedings of the 12th Annual ACM international Conference on Multimedia. New York, NY, 716--723.

Digital Library

Cited By

Budikova PBatko MZezula P(2018)ConceptRank for search-based image annotationMultimedia Tools and Applications10.1007/s11042-017-4777-877:7(8847-8882)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1007/s11042-017-4777-8
Kuric EBielikova M(2015)ANNORComputers and Graphics10.1016/j.cag.2014.09.03547:C(1-15)Online publication date: 1-Apr-2015
https://dl.acm.org/doi/10.1016/j.cag.2014.09.035
Wang LZhou TLee YCheoi KRyu K(2015)An efficient refinement algorithm for multi-label image annotation with correlation modelTelecommunications Systems10.1007/s11235-015-0030-960:2(285-301)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1007/s11235-015-0030-9
Show More Cited By

Index Terms

Scalable search-based image annotation of personal images
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

ConceptRank for search-based image annotation

Multimedia information is becoming an ubiquitous part of our lives, which brings an equally ubiquitous need for efficient multimedia retrieval. One of the possible solutions to this problem is to attach text descriptions to multimedia data objects, thus ...
Image annotation by large-scale content-based image retrieval
MM '06: Proceedings of the 14th ACM international conference on Multimedia

Image annotation has been an active research topic in recent years due to its potentially large impact on both image understanding and Web image search. In this paper, we target at solving the automatic image annotation problem in a novel search and ...
Learning to reduce the semantic gap in web image retrieval and annotation
SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

We study in this paper the problem of bridging the semantic gap between low-level image features and high-level semantic concepts, which is the key hindrance in content-based image retrieval. Piloted by the rich textual information of Web images, the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MIR '06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval

October 2006

344 pages

ISBN:1595934952

DOI:10.1145/1178677

General Chairs:
James Z. Wang
The Pennsylvania State University
,
Nozha Boujemaa
INRIA Rocquencourt, France
,
Yixin Chen
The University of Mississippi

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM06

Sponsor:

MM06: The 14th ACM International Conference on Multimedia 2006

October 26 - 27, 2006

California, Santa Barbara, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
728
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Budikova PBatko MZezula P(2018)ConceptRank for search-based image annotationMultimedia Tools and Applications10.1007/s11042-017-4777-877:7(8847-8882)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1007/s11042-017-4777-8
Kuric EBielikova M(2015)ANNORComputers and Graphics10.1016/j.cag.2014.09.03547:C(1-15)Online publication date: 1-Apr-2015
https://dl.acm.org/doi/10.1016/j.cag.2014.09.035
Wang LZhou TLee YCheoi KRyu K(2015)An efficient refinement algorithm for multi-label image annotation with correlation modelTelecommunications Systems10.1007/s11235-015-0030-960:2(285-301)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1007/s11235-015-0030-9
Wang XDu JWu SLi XXin HZhang YLi F(2015)High-level semantic image annotation based on hot Internet topicsMultimedia Tools and Applications10.1007/s11042-013-1742-z74:6(2055-2084)Online publication date: 1-Mar-2015
https://dl.acm.org/doi/10.1007/s11042-013-1742-z
Sun XWang CXu CZhang LJaimes ASebe NBoujemaa NGatica-Perez DShamma DWorring MZimmermann R(2013)Indexing billions of images for sketch-based retrievalProceedings of the 21st ACM international conference on Multimedia10.1145/2502081.2502281(233-242)Online publication date: 21-Oct-2013
https://dl.acm.org/doi/10.1145/2502081.2502281
Pawar CPatil S(2013)Interpretation of user intention via textual and visual resemblance2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT)10.1109/ICCCNT.2013.6726850(1-6)Online publication date: Jul-2013
https://doi.org/10.1109/ICCCNT.2013.6726850
Karlsen RAndersen A(2013)NFC-Based Image AnnotationTrends in Mobile Web Information Systems10.1007/978-3-319-03737-0_9(72-85)Online publication date: 2013
https://doi.org/10.1007/978-3-319-03737-0_9
Tang XLiu KCui JWen FWang X(2012)IntentSearchIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2011.24234:7(1342-1353)Online publication date: 1-Jul-2012
https://dl.acm.org/doi/10.1109/TPAMI.2011.242
Zhang DIslam MLu G(2012)A review on automatic image annotation techniquesPattern Recognition10.1016/j.patcog.2011.05.01345:1(346-362)Online publication date: 1-Jan-2012
https://dl.acm.org/doi/10.1016/j.patcog.2011.05.013
Bieliková MKuric E(2011)Automatic Image Annotation Using Global and Local FeaturesProceedings of the 2011 Sixth International Workshop on Semantic Media Adaptation and Personalization10.1109/SMAP.2011.14(33-38)Online publication date: 1-Dec-2011
https://dl.acm.org/doi/10.1109/SMAP.2011.14
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten