Rui Shi
ABSTRACT
Automatic image annotation (AIA) has been a hot research topic in recent years since it can be used to support concept-based image retrieval. However, most existing AIA models depend heavily on the availability of a large number of labeled training samples, which require significant human labeling efforts. In this paper, we propose a novel learning framework which integrates text-based Bayesian model (TBM) and concept ontology to effectively expand the training set of each concept class without the need of additional human labeling efforts or collecting additional training images from other data sources. The basic idea lies in exploiting the text information from training set to provide additional effective annotations for training images so that training data for each concept class can be augmented. In this study we employ Bayesian Hierarchical Multinomial Mixture Models (BHMMMs) as our baseline AIA model. By combining additional annotations obtained from TBM into each concept class in the training phase, the performance of BHMMMs can be significantly improved on Corel image dataset with 263 testing concepts as compared to the state-of-the-art AIA models under the same experimental configurations.
- K. Barnard, P. Duygulu and D. Forsyth, "Clustering Art", In Proc. Of IEEE Computer Vision and Pattern Recognition, 2001.Google Scholar
Cross Ref
- G. Carneiro and N. Vasconcelos, "Formulating Semantic Image Annotation as a Supervised Learning Problem", In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. Google ScholarDigital Library
- J. P. Fan, H. Z. Luo and Y. L. Gao, "Learning the Semantics of Images by Using Unlabeled Samples", Proceedings CVPR, 2005. Google ScholarDigital Library
- H. M. Feng. R. Shi and T. S. Chua, "A Bootstrapping Framework for Annotating and Retrieving WWW Images", In ACM Multimedia'04, pp. 960--967, New York, 2004. Google ScholarDigital Library
- S. L. Feng, R. Manmatha and V. Lavrenko, "Multiple Bernoulli Relevance Models for Image and Video Annotation", Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR'04. Google ScholarDigital Library
- S. Gao, D.-H. Wang and C.-H. Lee, "Automatic Image Annotation through Multi-Topic Text Categorization", Proc. ICASSP, Toulouse, France, May 2006.Google Scholar
- J. Jeon, V. Lavrenko, and R. Manmatha, "Automatic Image Annotation and Retrieval Using Cross-Media Relevance Models", Proc. of the 26th ACM SIGIR, 2003. Google ScholarDigital Library
- V. Lavrenko, R. Manmatha and J. Jeon, "A Model for Learning the Semantics of Pictures", NIPS, 2003.Google Scholar
- G. A. Miller, R. Beckwith, C. Fellbaum, D. Gross and K. J. Miller, "Introduction to WordNet: an on-line lexical database", Intl. Jour. Of Lexicography, pp. 235--244, 1990.Google ScholarCross Ref
- J. Novovicova and A. Malik, "Application of Multinomial Mixture Model to Text Classification", Pattern Recognition and Image Analysis, pp. 646--653, 2003.Google ScholarCross Ref
- M. Srikanth, J. Varner, M. Bowden and D. Moldovan, "Exploiting Ontologies for Automatic Image Annotation", Proceedings of the 28th ACM SIGIR, 2005. Google ScholarDigital Library
- R. Shi, T. S. Chua, C. H. Lee and S. Gao, "Bayesian Learning of Hierarchical Multinomial Mixture Models of Concepts for Automatic Image Annotation", In Proc. of CIVR'06, pp. 102--112, Arizona, United States, 2006. Google ScholarDigital Library
- S. Tong and E. Chang, "Support Vector Machine Active Learning for Image Retrieval", In ACM Multimedia'01, pp.107--118, Ottawa, Canada, 2001. Google ScholarDigital Library
- R. Yan, and A. G. Hauptmann, "Multi-class Active Learning for Video Semantic Feature Extraction", In Proc. of ICME'04, pp. 69--72, 2004.Google Scholar
- C. X. Zhai and J. Lafferty, "A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval", SIGIR'01, 2001. Google ScholarDigital Library
Index Terms
- Enhancing image annotation by integrating concept ontology and text-based bayesian learning model
Recommendations
A Novel Region-based Image Annotation Using Multi-instance Learning
WKDD '09: Proceedings of the 2009 Second International Workshop on Knowledge Discovery and Data MiningIn this paper, we formulate image annotation as a semi-supervised learning problem under multi-instance learning framework. A novel graph based semi-supervised learning approach to image annotation using multiple instances is presented, which extends ...
Transductive Multi-Instance Multi-Label learning algorithm with application to automatic image annotation
Automatic image annotation has emerged as an important research topic due to its potential application on both image understanding and web image search. Due to the inherent ambiguity of image-label mapping and the scarcity of training examples, the ...
Multi-label learning by Image-to-Class distance for scene classification and image annotation
CIVR '10: Proceedings of the ACM International Conference on Image and Video RetrievalIn multi-label learning, an image containing multiple objects can be assigned to multiple labels, which makes it more challenging than traditional multi-class classification task where an image is assigned to only one label. In this paper, we propose a ...
Comments