skip to main content
10.1145/1631272.1631369acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
short-paper

What is a complete set of keywords for image description & annotation on the web

Published: 19 October 2009 Publication History

Abstract

Does there exist a compact set of keywords that can completely and effectively cover the image annotation problem by expanding from it? In this paper, we answer this question by presenting a complete set framework for image annotation, which is motivated by the existence of semantic ontology. To generate this set, we propose a cross model optimization strategy from both textual and visual information for topic decomposition, based on a so-called Bipartite LSA model, which minimize multimodal error energy functions in a probabilistic Latent Semantic Analysis model. To achieve complete set based annotation, we present a Gaussian-Kernel-Generative process based keyword generation procedure, which analogizes keyword annotation in a probabilistic generative manner. A group of experiments is performed on Washington University image database and 80,000 Flickr images with comparisons to the state-of-the-arts. Finally, potential advantages and future improvements of our framework are discussed outside the scope of topic modeling.

References

[1]
S. Zinger, C. Millet, B. Mathieu, G. Grefenstette, P. Hède, and P.-A. Moëllic. 2005. Extracting an Ontology of Portrayable Objects from WordNet. In Proceedings of the MUSCLE/ImageCLEF Workshop on Image and Video Retrieval Evaluation, 2005.
[2]
T.K. Landauer, P.W. Foltz and D. Laham. 1998. An Introduction to Latent Semantic Analysis. Discourse Processes, 1998.
[3]
T. Hofmann. 1999. Probabilistic Latent Semantic Indexing. In Proceedings of the 22nd annual international ACM SIGIR, 1999.
[4]
D. Blei, A. Ng and M. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research, 2003, 3:993--1022.
[5]
K. Barnard, P. Duygulu, N. de Freitas, D. Forsyth, D. Blei and M. Jordan. 2003. Matching words and pictures. Journal of Machine Learning Research, 2003.
[6]
D. Blei and M. I. Jordan. 2003. Modeling annotated data. In Proceedings of the 26th Intl. ACM SIGIR, 2003.
[7]
X. Rui, M. Li, Z. Li, W.Y. Ma and N. Yu. 2007. Bipartite graph reinforcement model for web image annotation. In Proceedings of the ACM International Conference on Multimedia, 2007.
[8]
Y. Lu, L. Zhang, Q. Tian and W.Y. Ma. 2008. What are the High-Level Concepts with Small Semantic Gaps?. CVPR, 2008.
[9]
Xianming Liu, Rongrong Ji, Hongxun Yao, Pengfei Xu, Xiaoshuai Sun, Tianqiang Liu. "Cross-Media Manifold Learning for Image Retrieval&Annotation". ACM MIR 2008, pp: 141--148, 2008.
[10]
Rongrong Ji, Hongxun Yao, "Visual&Textual Fusion for Region Retrieval: From Both Fuzzy Matching and Bayesian Reasoning Aspects," ACM Conference on Multimedia Information Retrieval (MIR), pp.159--168, 2007.

Cited By

View all
  • (2018)Bidirectional-isomorphic manifold learning at image semantic understanding & representationMultimedia Tools and Applications10.1007/s11042-011-0947-264:1(53-76)Online publication date: 30-Dec-2018
  • (2010)Visual topic model for web image annotationProceedings of the Second International Conference on Internet Multimedia Computing and Service10.1145/1937728.1937758(126-130)Online publication date: 30-Dec-2010
  • (2010)Exploring statistical properties for semantic annotation: sparse distributed and convergent assumptions for keywords2010 IEEE International Conference on Acoustics, Speech and Signal Processing10.1109/ICASSP.2010.5494954(802-805)Online publication date: Mar-2010

Index Terms

  1. What is a complete set of keywords for image description & annotation on the web

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '09: Proceedings of the 17th ACM international conference on Multimedia
    October 2009
    1202 pages
    ISBN:9781605586083
    DOI:10.1145/1631272
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 October 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. image annotation
    2. keyword selection
    3. ontology
    4. semantic items

    Qualifiers

    • Short-paper

    Conference

    MM09
    Sponsor:
    MM09: ACM Multimedia Conference
    October 19 - 24, 2009
    Beijing, China

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)1
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 27 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2018)Bidirectional-isomorphic manifold learning at image semantic understanding & representationMultimedia Tools and Applications10.1007/s11042-011-0947-264:1(53-76)Online publication date: 30-Dec-2018
    • (2010)Visual topic model for web image annotationProceedings of the Second International Conference on Internet Multimedia Computing and Service10.1145/1937728.1937758(126-130)Online publication date: 30-Dec-2010
    • (2010)Exploring statistical properties for semantic annotation: sparse distributed and convergent assumptions for keywords2010 IEEE International Conference on Acoustics, Speech and Signal Processing10.1109/ICASSP.2010.5494954(802-805)Online publication date: Mar-2010

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media