skip to main content
10.1145/1816041.1816060acmconferencesArticle/Chapter ViewAbstractPublication PagescivrConference Proceedingsconference-collections
poster

Multi-label learning by Image-to-Class distance for scene classification and image annotation

Published: 05 July 2010 Publication History

Abstract

In multi-label learning, an image containing multiple objects can be assigned to multiple labels, which makes it more challenging than traditional multi-class classification task where an image is assigned to only one label. In this paper, we propose a multi-label learning framework based on Image-to-Class (I2C) distance, which is recently shown useful for image classification. We adjust this I2C distance to cater for the multi-label problem by learning a weight attached to each local feature patch and formulating it into a large margin optimization problem. For each image, we constrain its weighted I2C distance to the relevant class to be much less than its distance to other irrelevant class, by the use of a margin in the optimization problem. Label ranks are generated under this learned I2C distance framework for a query image. Thereafter, we employ the label correlation information to split the label rank for predicting the label(s) of this query image. The proposed method is evaluated in the applications of scene classification and automatic image annotation using both the natural scene dataset and Microsoft Research Cambridge (MSRC) dataset. Experiment results show better performance of our method compared to previous multi-label learning algorithms.

References

[1]
J. W. A. Elisseeff. A kernel methods for multi-labelled classification. In Advances in Neural Information Processing Systems 14, pages 681--687, Cambridge, MA, 2002. MIT Press.
[2]
O. Boiman, E. Shechtman, and M. Irani. In defense of nearest-neighbor based image classification. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2008.
[3]
M. R. Boutell, J. Luo, X. Shen, and C. M. Brown. Learning multi-label scene classification. Pattern Recognition, 37(9):1751--1771, 2004.
[4]
S. L. Feng, R. Manmatha, and V. Lavrenko. Multiple bernoulli relevance models for image and video annotation. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1002--1009, 2004.
[5]
A. Frome, Y. Singer, F. Sha, and J. Malik. Learning globally-consistent local distance functions for shape-based image retrieval and classification. In Proceedings of IEEE International Conference on Computer Vision, October 2007.
[6]
F. Kang, R. Jin, and R. Sukthankar. Correlated label propagation with application to multi-label learning. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1719--1726, 2006.
[7]
S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 2169--2178, 2006.
[8]
T. Li, T. Mei, S. Yan, I.-S. Kweon, and C. Lee. Contextual decomposition of multi-label images. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 2270--2277, June 2009.
[9]
D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91--110, 2004.
[10]
Z. Lu, H. H. Ip, and Q. He. Context-based multi-label image annotation. In Proceeding of the ACM International Conference on Image and Video Retrieval, July 2009.
[11]
R. E. Schapire and Y. Singer. BoosTexter: a boosting-based system for text categorization. Machine Learning, 39(2--3):135--168, 2000.
[12]
G. Tsoumakas, A. Dimou, E. Spyromitros, V. Mezaris, I. Kompatsiaris, and I. Vlahavas. Correlation-based pruning of stacked binary relevance models for multi-label learning. In ECML PKDD Workshop on Learning From Multi-Label Data, pages 101--116, Sepember 2009.
[13]
S. Vijayanarasimhan and K. Grauman. What's it going to cost you?: predicting effort vs. informativeness for multi-label image annotations. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 2262--2269, June 2009.
[14]
C. Wang, S. Yan, L. Zhang, and H.-J. Zhang. Multi-label sparse coding for automatic image annotation. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1643--1650, June 2009.
[15]
H. Wang, H. Huang, and C. Ding. Image annotation using multi-label correlated green's function. In Proceedings of IEEE International Conference on Computer Vision, 2009.
[16]
M. Wang, X. Zhou, and T.-S. Chua. Automatic image annotation via local multi-label classification. In Proceeding of the ACM International Conference on Image and Video Retrieval, pages 17--26, July 2008.
[17]
Z. Wang, Y. Hu, and L.-T. Chia. Learning instance-to-class distance for human action recognition. In IEEE International Conference on Image Processing, pages 3545--3548, November 2009.
[18]
Z.-J. Zha, X.-S. Hua, T. Mei, J. Wang, G.-J. Qi, and Z. Wang. Joint multi-label multi-instance learning for image classification. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2008.
[19]
Z.-J. Zha, T. Mei, J. Wang, Z. Wang, and X.-S. Hua. Graph-based semi-supervised learning with multi-label. In IEEE International Conference on Multimedia and Expo, pages 1321--1324, 2008.
[20]
M.-L. Zhang and Z.-H. Zhou. Ml-knn: a lazy learning approach to multi-label learning. Pattern Recognition, 40(7):2038--2048, 2007.
[21]
M.-L. Zhang and Z.-H. Zhou. Multi-label learning by instance differentiation. In AAAI'07: Proceedings of the 22nd national conference on Artificial intelligence, pages 669--674, 2007.
[22]
M.-L. Zhang and Z.-H. Zhou. M3MIML: a maximum margin method for multi-instance multi-label learning. In Proceedings of IEEE International Conference on Data Mining, pages 688--697, December 2008.
[23]
Z.-H. Zhou and M.-L. Zhang. Multi-instance multi-label learning with application to scene classification. In Advances in Neural Information Processing Systems 19, pages 1634--1641, Cambridge, MA, 2007. MIT Press.

Cited By

View all
  • (2016)Sparsity-Induced Similarity Measure and Its ApplicationsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2012.222591126:4(613-626)Online publication date: Apr-2016
  • (2015)Semantic Annotation Model for objects Classification2015 IEEE Student Conference on Research and Development (SCOReD)10.1109/SCORED.2015.7449439(87-92)Online publication date: Dec-2015
  • (2015)Fuzzy Rough Decision Trees for Multi-label ClassificationRough Sets, Fuzzy Sets, Data Mining, and Granular Computing10.1007/978-3-319-25783-9_19(207-217)Online publication date: 8-Nov-2015
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIVR '10: Proceedings of the ACM International Conference on Image and Video Retrieval
July 2010
492 pages
ISBN:9781450301176
DOI:10.1145/1816041
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Image-to-Class distance
  2. automatic image annotation
  3. multi-label learning
  4. scene classification

Qualifiers

  • Poster

Conference

CIVR' 10
Sponsor:

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)1
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2016)Sparsity-Induced Similarity Measure and Its ApplicationsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2012.222591126:4(613-626)Online publication date: Apr-2016
  • (2015)Semantic Annotation Model for objects Classification2015 IEEE Student Conference on Research and Development (SCOReD)10.1109/SCORED.2015.7449439(87-92)Online publication date: Dec-2015
  • (2015)Fuzzy Rough Decision Trees for Multi-label ClassificationRough Sets, Fuzzy Sets, Data Mining, and Granular Computing10.1007/978-3-319-25783-9_19(207-217)Online publication date: 8-Nov-2015
  • (2014)Kernelized pyramid nearest-neighbor search for object categorizationMachine Vision and Applications10.1007/s00138-014-0608-325:4(931-941)Online publication date: 1-May-2014
  • (2012)Deep representations and codes for image auto-annotationProceedings of the 26th International Conference on Neural Information Processing Systems - Volume 110.5555/2999134.2999236(908-916)Online publication date: 3-Dec-2012

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media