skip to main content
10.1145/1873951.1874031acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Image retagging

Published: 25 October 2010 Publication History

Abstract

Online social media repositories such as Flickr and Zooomr allow users to manually annotate their images with freely-chosen tags, which are then used as indexing keywords to facilitate image search and other applications. However, these tags are frequently imprecise and incomplete, though they are provided by human beings, and many of them are almost only meaningful for the image owners (such as the name of a dog). Thus there is still a gap between these tags and the actual content of the images, and this significantly limits tag-based applications, such as search and browsing. To tackle this issue, this paper proposes a social image "retagging" scheme that aims at assigning images with better content descriptors. The refining process, including denoising and enriching, is formulated as an optimization framework based on the consistency between "visual similarity" and "semantic similarity" in social images, that is, the visually similar images tend to have similar semantic descriptors, and vice versa. An effective iterative bound optimization algorithm is applied to learn the improved tag assignment. In addition, as many tags are intrinsically not closely-related to the visual content of the images, we employ knowledge based method to differentiate visual content related tags from unrelated ones and then constrain the tagging vocabulary of our automatic algorithm within the content related tags. Finally, to improve the coverage of the tags, we further enrich the tag set with appropriate synonyms and hypernyms based on an external knowledge base. Experimental results on a Flickr image collection demonstrate the effectiveness of this approach. We will also show the remarkable performance improvements brought by retagging via two applications, i.e., tag-based search and automatic annotation.

References

[1]
P. Anderson. What is web 2.0? Ideas, technologies and implications for education. JISC Technical Report, 2007.
[2]
M. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: State of the art and challenges. TOMCCAP, 2(1):1--19, 2006.
[3]
S. Golder and B. Huberman. Usage patterns of collaborative tagging systems. JIS, 32(2):198--208, 2006.
[4]
K. Matusiak. Towards user-centered indexing in digital image collections. OCLC Systems and Service, 22(4):283--298, 2006.
[5]
J. Li and J. Wang. Real-time computerized annotation of pictures. TPAMI, 30(6):985--1002, 2008.
[6]
X.-S. Hua and G. Qi. Online multi-label active annotation: Towards large-scale content-based video search. In MM, pages 141--150, 2008.
[7]
C. Fellbaum. Wordnet: An electronic lexical database. Bradford Books, 1998.
[8]
D. Liu, X.-S. Hua, L. Yang, M. Wang, and H.-J. Zhang. Tag ranking. In WWW, pages 351--360, 2009.
[9]
D. Liu, X.-S. Hua, M. Wang, and H.-J. Zhang. Boost search relevance for tag-based social image retrieval. In ICME, pages 1636--1639, 2009.
[10]
Z.-J. Zha, L. Yang, T. Mei, M. Wang, and Z. Wang. Visual query suggestion. In MM, pages 15--24, 2009.
[11]
M. Ames and M. Naaman. Why we tag: Motivations for annotation in mobile and online media. In CHI, pages 971--980, 2007.
[12]
B. Sigurbj-ornsson and R. Zwol. Flickr tag recommendation based on collective knowledge. In WWW, pages 327--336, 2008.
[13]
L. Kennedy, S.-F. Chang, and I. Kozintsev. To search or to label? Predicting the performance of search-based automatic image classifiers. In MIR, pages 249--258, 2006.
[14]
R. Yan, A. Natsev, and M. Campbell. A learning-based hybrid tagging and browsing approach for e±cient manual image annotation. In CVPR, pages 1--8, 2008.
[15]
K. Weinberger, M. Slaney, and R. Zwol. Resolving tag ambiguity. In MM, pages 111--120, 2008.
[16]
D. Liu, X.-S. Hua, M. Wang, and H.-J. Zhang. Retagging social images based on visual and semnatic consistecy. In WWW, pages 1149--1150, 2010.
[17]
D. Liu, M. Wang, J. Yang, X.-S. Hua, and H.-J. Zhang. Tag quality improvement for social images. In ICME, pages 350--353, 2009.
[18]
Y. Jin, L. Khan, L. Wang, and M. Awad. Image annotation by combining multiple evidence & wordNet. In MM, pages 706--715, 2005.
[19]
C. Wang, F. Jing, L. Zhang, and H.-J. Zhang. Content-based image annotation refinement. In CVPR, pages 1--8, 2007.
[20]
B. Dennis. Foragr: Collaboratively tagged photographs and social information visualization. In WWW, 2006.
[21]
Y. Lu, L. Zhang, Q. Tian, and W. Ma. What are the high-level concepts with small semantic gaps? In CVPR, pages 1--8, 2008.
[22]
K. Yanai and K. Barnard. Image region entropy: A measure of visualness of web images associated with one concept. In MM, pages 419--422, 2005.
[23]
S. Overell, B. Sigurbj-ornsson, and R. Zwol. Classifying tags uing open content resources. In WSDM, pages 64--73, 2009.
[24]
A. Torralba, R. Fergus, and W. Freeman. 80 million tiny images: A large dataset for non-parametric object and scene recognition. TPAMI, 30(11):1958--1970, 2008.
[25]
D. Lin. Using syntatic dependency as local context to resolve word sense ambiguity. In ACL, pages 64--71, 1997.
[26]
R. Cilibrasi and P. Vitanyi. The google similarity distance. TKDE, 19(3):370--383, 2007.
[27]
Y. Liu, R. Jin, and L. Yang. Semi-supervised multi-label learning by constrained non-negative matrix factorization. In AAAI, pages 421--426, 2006.
[28]
S.-F. Chang, J. He, Y. Jiang, E. Khoury, C. Ngo, A. Yanagawa, and E. Zavesky. Columbia University/VIREO-CityU/IRIT TRECVID2008 high-level feature extraction and interactive video search. In NIST TRECVID Workshop, 2008.
[29]
D. Lee and H. Seung. Algorithms for non-negative matrix factorization. In NIPS, pages 556--562, 2001.

Cited By

View all
  • (2022)Automatic Tagging by Leveraging Visual and Annotated Features in Social MediaIEEE Transactions on Multimedia10.1109/TMM.2021.305503724(2218-2229)Online publication date: 2022
  • (2020)Shuffled ImageNet Banks for Video Event Detection and SearchACM Transactions on Multimedia Computing, Communications, and Applications10.1145/337787516:2(1-21)Online publication date: 22-May-2020
  • (2020)Enhancing the Quality of Image Tagging Using a Visio-Textual Knowledge BaseIEEE Transactions on Multimedia10.1109/TMM.2019.293718122:4(897-911)Online publication date: Apr-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '10: Proceedings of the 18th ACM international conference on Multimedia
October 2010
1836 pages
ISBN:9781605589336
DOI:10.1145/1873951
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. image search
  2. image tagging
  3. retagging
  4. tag refinement

Qualifiers

  • Research-article

Conference

MM '10
Sponsor:
MM '10: ACM Multimedia Conference
October 25 - 29, 2010
Firenze, Italy

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Automatic Tagging by Leveraging Visual and Annotated Features in Social MediaIEEE Transactions on Multimedia10.1109/TMM.2021.305503724(2218-2229)Online publication date: 2022
  • (2020)Shuffled ImageNet Banks for Video Event Detection and SearchACM Transactions on Multimedia Computing, Communications, and Applications10.1145/337787516:2(1-21)Online publication date: 22-May-2020
  • (2020)Enhancing the Quality of Image Tagging Using a Visio-Textual Knowledge BaseIEEE Transactions on Multimedia10.1109/TMM.2019.293718122:4(897-911)Online publication date: Apr-2020
  • (2020)A review on visual content-based and users’ tags-based image annotation: methods and techniquesMultimedia Tools and Applications10.1007/s11042-020-08862-1Online publication date: 9-May-2020
  • (2019)A Collaborative Learning Framework to Tag Refinement for Points of InterestProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3292500.3330698(1752-1761)Online publication date: 25-Jul-2019
  • (2019)Social Anchor-Unit Graph Regularized Tensor Completion for Large-Scale Image RetaggingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2019.290660341:8(2027-2034)Online publication date: 1-Aug-2019
  • (2019)Deep Collaborative Embedding for Social Image UnderstandingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2018.285275041:9(2070-2083)Online publication date: 1-Sep-2019
  • (2019)Cauchy Matrix Factorization for Tag-Based Social Image RetrievalIEEE Access10.1109/ACCESS.2019.29405987(132302-132310)Online publication date: 2019
  • (2019)Multi‐label automatic image annotation approach based on multiple improvement strategiesIET Image Processing10.1049/iet-ipr.2018.537113:4(623-633)Online publication date: 7-Mar-2019
  • (2019)Crowdsourced object-labeling based on a game-based mobile applicationMultimedia Tools and Applications10.1007/s11042-018-6944-y78:13(18137-18168)Online publication date: 1-Jul-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media