skip to main content
10.1145/1180639.1180840acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
Article

Automatic image annotation by incorporating feature hierarchy and boosting to scale up SVM classifiers

Published: 23 October 2006 Publication History

Abstract

The performance of image classifiers largely depends on two inter-related issues:(1)suitable frameworks for image content representation and automatic feature extraction;(2) effective algorithms for image classifier training and feature subset selection. To address the first issue, a multiresolution grid-based framework is proposed for image content representation and feature extraction to bypass the time-consuming and erroneous process for image segmentation. To address the second issue, a hierarchical boosting algorithm is proposed by incorporating feature hierarchy and boosting to scale up SVM image classifier training in high-dimensional feature space. The high-dimensional multi-modal heterogeneous visual features are partitioned into multiple low-dimensional single-modal homogeneous feature subsets and each of them characterizes certain visual property of images. For each homogeneous feature subset, principal component analysis (PCA)is performed to exploit the feature correlations and a weak classifier is learned simultaneously. After the weak classifiers for different feature subsets and grid sizes are available, they are combined to boost an optimal classifier for the given object class or image concept, and the most representative feature subsets and grid sizes are selected. Our experiments on a specific domain of natural images have obtained very positive results.

References

[1]
Y. Rui, T.S. Huang, and S.-F. Chang, "Image Retrieval: Current Techniques, Promising Directions and Open Issues ", Journal of Visual Communication and Image Representation Vol. 10, pp. 39--62, 1999.
[2]
F. Monay, D. Gatica-Perez, "On image auto-annotation with latent space models", ACM Multimedia, 2003.
[3]
A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta and R. Jain, "Content-based image retrieval at the end of the early years", IEEE Trans. on PAMI 2000.
[4]
R. Zhao, W.I. Grosky, "Negotiating the semantic gap: from feature maps to semantic landscapes", Pattern Recognition vol. 5, no. 3, pp. 593--600, 2002.
[5]
X. He, W.-Y. Ma, O. King, M. Liand H.J. Zhang, "Learning and inferring a semantic space from user's relevance feedback", ACM Multimedia, 2002.
[6]
R. Lienhart and A. Hartmann, "Classifying images on the web automatically", Journal of Electronic Imaging vol. 11, no. 4, pp. 445--454, 2002.
[7]
C. Carson, S. Belongie, H. Greenspan, J. Malik, "Blobworld: Image segmentation using expectation-maximization and its application to image querying", IEEE Trans. PAMI 2002.
[8]
Y. Gong, "Advancing Content-Based Image Retrieval by Exploiting Image Color and Region Features", Multimedia Systems vol. 7, no. 6, pp. 449--457, 1999.
[9]
K. Vu, K.A. Hua, W. Tavanapong, "Image Retrieval Based on Regions of Interest", IEEE Trans. TKDE vol. 15, no. 4, pp. 1045--1049, 2003.
[10]
J.R. Smith and C.-S. Li, "Image classification and querying using composite region template", Journal of CVIU 1999.
[11]
B. Manjunath, W.-Y.Ma, "Texture features for browsing and retrieval of image data", IEEE Trans. PAMI vol. 18, pp. 837--842, 1996.
[12]
B. Li, K. Goh, E. Chang, "Confidence-based dynamic ensamble for image annotation and semantic discovery", ACM Multimedia, 2003.
[13]
A.B. Benitez, S.-F. Chang, "Image classification using multimedia knowledge networks", ICIP, pp. 613--616, 2003.
[14]
Y. Li, C. Dorai, R. Farrell, "Creating MAGIC: system for generating learning object metadata for instructional content", ACM Multimedia, pp. 367--370, 2005.
[15]
C. Dorai, S. Venkatesh, "Bridging the Semantic Gap with Computational Media Aesthetics", IEEE MultiMedia vol. 10, no. 2, pp. 15--17, 2003.
[16]
S. Fischer, R. Lienhart, W.E fielsberg, "Automatic Recognition of Film Genres", ACM Multimedia, pp. 295--304, 1995.
[17]
P. Quelhas, F. Monay, J.-M. Odobez, D. Gatica-Perez, T. Tuytelaars, L.J. Van Gool, "Modeling Scenes with Local Descriptors and Latent Aspects", IEEE ICCV, 883--890, 2005.
[18]
F. Monay, D. Gatica-Perez, "PLSA-based image auto-annotation:constraining the latent space", ACM Multimedia, pp. 348--351, 2004.
[19]
J. Luo, M.R. Boutell, R.T. Gray, C.M. Brown, "Image transform bootstrapping and its applications to semantic scene classification", IEEE Trans. on SMC vol. 35, no. 3, pp. 563--570, 2005.
[20]
N. Serrano, A.E. Savakis, J. Luo, "Improved scene classification using efficient low-level features and semantic cues", Pattern Recognition vol. 37, no. 9, pp. 1773--1784, 2004.
[21]
M.R. Naphade, J.R. Smith, "On the detection of semantic ocncepts at TRECVID", ACM Multimedia, 2004.
[22]
J.Z. Wang, J. Li and G. Wiederhold, "SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture Libraries", IEEE Trans. on PAMI vol. 23, no. 9, pp. 947--963, 2001.
[23]
J. Li and J.Z. Wang, "Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach", IEEE Trans. on PAMI vol. 25, no. 9, pp. 1075--1088, 2003.
[24]
R. Jin, A.G. Hauptmann, "Using a probabilistic source model for comparing images", ICIP, pp. 941--944, 2002.
[25]
J. Fan, Y. Gao, H. Luo, "Multi-level annotation of natural scenes using dominant image compounds and semantic concepts", ACM Multimedia, 2004.
[26]
A. Vailaya, M. Figueiredo, A.K. Jain, H.J. Zhang, "Image classification for content-based indexing", IEEE Trans. on Image Processing vol. 10, pp. 117--130, 2001.
[27]
K. Barnard and D. Forsyth, "Learning the semantics of words and pictures", Proc. ICCV, pp. 408--415, 2001.
[28]
K. Tieu, P. Viola, "Boosting image retrieval", Proc. CVPR, 2000.
[29]
D. Zhang, S.Z. Li, D. Gatica-Perez, "Real-Time Face Detection Using Boosting in Hierarchical Feature Spaces", ICPR, pp. 411--414, 2004.
[30]
J. Fan, H. Luo, Y. Gao, "Learning the semantics of images by using unlabeled samples", IEEE CVPR, pp. 704--710, 2005.
[31]
D.-D. Le, Shin'ichi Satoh, "An Efficient Feature Selection Method for Object Detection", ICAPR, pp. 461--468, 2005.
[32]
J. O'Sullivan, J. Langford, R. Caruana, A. Blum, "FeatureBoost: A meta learning algorithm that improves model robustness", ICML, pp. 703--710, 2000.
[33]
P. Viola, M. Jones, "Robust real-time face detection", Intl. J. ComputerVision vol. 57, no. 2, pp. 137--154, 2004.
[34]
J.C. Platt, "Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods", in Adavances in Large Margin Classifiers MIT Press,1999.
[35]
B. Heisele, T. Serre, S. Prentice, T. Poggio, "Hierarchical classification and feature reduction for fast face detection with SVM", Pattern Recognition vol. 36, pp. 2007--2017, 2003.
[36]
L. Breiman, "Bagging predictors", Machine Learning vol. 24, no. 2, pp. 123--140, 1996.
[37]
Y. Freund, R.E. Schapire, "Experiments with a new boosting algorithm", Proc. ICML, pp. 148--156, 1996.
[38]
R. Lienhart, A. Kuranov, V. Pisarevsky, "Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection", DAGM-Symposium, pp. 297--304, 2003.
[39]
L. Zhang, M. Li, H.-J. Zhang, "Boosting image orientation detection with indoor vs. outdoor classification", WACV 2003.
[40]
A. Torralba, K. Murphy, W. Freeman, "Sharing features: efficient boosting procedures for multiclass object detection", CVPR, 2004.
[41]
N. Panda, E. Chang, "Exploiting geometric property for support vector machine indexing", SIAM Data Mining, 2005.
[42]
G.H. John, R. Kohavi, K.P eger, "Irrelevant features and the subset selection problem", ICML, pp. 121--129, 1994.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '06: Proceedings of the 14th ACM international conference on Multimedia
October 2006
1072 pages
ISBN:1595934472
DOI:10.1145/1180639
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. hierarchical boosting
  2. image annotation

Qualifiers

  • Article

Conference

MM06
MM06: The 14th ACM International Conference on Multimedia 2006
October 23 - 27, 2006
CA, Santa Barbara, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)1
Reflects downloads up to 24 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)A Comprehensive Survey on the Process, Methods, Evaluation, and Challenges of Feature SelectionIEEE Access10.1109/ACCESS.2022.320561810(99595-99632)Online publication date: 2022
  • (2019)CNN-feature based automatic image annotation methodMultimedia Tools and Applications10.1007/s11042-018-6038-x78:3(3767-3780)Online publication date: 1-Feb-2019
  • (2018)Visual attention mechanism and support vector machine based automatic image annotationPLOS ONE10.1371/journal.pone.020697113:11(e0206971)Online publication date: 6-Nov-2018
  • (2018)Structured Scene Decoding with Finite State Machines2018 25th IEEE International Conference on Image Processing (ICIP)10.1109/ICIP.2018.8451821(485-489)Online publication date: Oct-2018
  • (2018)Kernel Based Approaches for Context Based Image AnnotatıonComputational Vision and Bio Inspired Computing10.1007/978-3-319-71767-8_21(249-258)Online publication date: 20-Feb-2018
  • (2017)Automatic Image Annotation Based on Particle Swarm Optimization and Support Vector ClusteringMathematical Problems in Engineering10.1155/2017/84932672017:1Online publication date: 18-May-2017
  • (2017)Robust Web Image Annotation via Exploring Multi-Facet and Structural KnowledgeIEEE Transactions on Image Processing10.1109/TIP.2017.271718526:10(4871-4884)Online publication date: Oct-2017
  • (2017)Automatic image annotation by combining generative and discriminant modelsNeurocomputing10.1016/j.neucom.2016.09.108236:C(48-55)Online publication date: 2-May-2017
  • (2016)Sparse Penalized Forward Selection for Support Vector ClassificationJournal of Computational and Graphical Statistics10.1080/10618600.2015.102339525:2(493-514)Online publication date: 10-May-2016
  • (2015)Improving tag matrix completion for image annotation and retrieval2015 Visual Communications and Image Processing (VCIP)10.1109/VCIP.2015.7457871(1-4)Online publication date: Dec-2015
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media