skip to main content
10.1145/2043674.2043677acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicimcsConference Proceedingsconference-collections
research-article

Multiple instance learning with missing object tags

Published: 05 August 2011 Publication History

Abstract

In this paper, we have addressed two key issues for leveraging large-scale loosely-tagged images for object classifier training: (a) loose object tags, e.g., multiple object tags are loosely given at the image level without identifying object locations in the images; (b) missing object tags, e.g., some object tags are missed and thus negative bags may contain positive instances. To address both the issues of loose object tags and missing object tags jointly, a novel multiple instance learning (MIL) algorithm is developed and it consists of the following key components: (1) An agreement value is defined for characterizing instance-tag relatedness; (2) Automatic instance clustering is performed and inter-cluster correlations are leveraged for agreement value refinement; (3) An automatic instance-tag alignment algorithm is developed for assigning multiple object tags into the most relevant image instances with maximum agreement values and determining uncertain image instances whose object tags are not available on the tag list (missing object tags); (4) Object co-occurrence contexts are leveraged to predict missing object tags for the uncertain image instances. Our experiments on large-scale loosely-tagged images have provided very positive results.

References

[1]
R. Fergus, L. Fei-Fei, P. Perona, A. Zisserman, "Learning object categories from Google's image search", CVPR, 2006.
[2]
T. Berg, A. Berg, J. Edwards, M. Mair, R. White, Y. Yeh, E. Learned-Miller, D. Forsyth, "Names and faces in the news", CVPR, 2004.
[3]
F. Schroff, A. Criminisi, A. Zisserman, "Harvesting image databases from the web", ICCV, 2007.
[4]
A. Quattoni, M. Collins, T. Darrell, "Learning visual representations using images with captions", CVPR 2007.
[5]
N. Ben-Haim, B. Babenko, S. Belongie, "Improving image search via content based clustering", SLAM 2006.
[6]
Y. Deng, B. S. Manjunath, "Color image segmentation", CVPR, 1999.
[7]
J Shi, J Malik, "Normalized cuts and image segmentation", IEEE Trans. on PAMI, 2000.
[8]
B. J. Frey, D. Dueck, "Clustering by Passing Messages Between Data Points", Science, vol.315, pp.972--976, Feb. 2007
[9]
S. Vijayanarasimhan, K. Grauman, "Keywords to visual categories: Multiple-instance learning for weakly supervised object categorization", CVPR 2008.
[10]
S. Vijayanarasimhan, K. Grauman, "What's it going to cost you?: Predicting effort vs. informativeness for multi-label image annotations", CVPR 2009.
[11]
C. Galleguillos, B. Babenko, A. Rabinovich, S. J. Belongie, "Weakly supervised object localization with stable segmentations", ECCV, pp.193--207, 2008.
[12]
T. Cour, B. Sapp, C. Jordan, B. Taskar, "Learning from ambiguously labeled images", CVPR, 2009.
[13]
C. R. Rosenberg, M. Hebert, "Training object detection models with weakly labeled data", BMVC 2002.
[14]
U. Syed, B. Taskar, "Semi-supervised learning with adversarially missing label information", NIPS, 2010.
[15]
Q. Zhang, W. Yu, S. A. Goldman, J. E. Fritts, "Content-based image retrieval using multiple-instance learning", ICML, 2002.
[16]
O. Maron, A. L. Ratan, "Multiple-instance learning for natural scene classification", ICML, 1998.
[17]
Y. Chen, J. Bi, J. Z. Wang, "MILES: multiple instance learning via embedded instance selection", IEEE Trans. PAMI, vol.28, no.12, pp.1931--1947, 2006.
[18]
P. Viola, J. C. Platt, C. Zhang, "Multiple instance boosting for object detection", ICML, 2006.
[19]
J. Tang, X. Hua, M. Wang, Z. Gu, G. Qi, X. Wu, "Correlative linear neighborhood propagation for video annotation", IEEE Trans. on SMC, vol. 39, no.2, pp.409--416, 2009.
[20]
G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, T. Mei, H.-J. Zhang, "Correlative multi-label video annotation", ACM Multimedia, pp.17--26, 2007.
[21]
Z. Zha, X.-S. Hua, T. Mei, J. Wang, G.-J. Qi, Z. Wang, "Joint multi-label multi-instance learning for image classification", CVPR, 2008.
[22]
Z.-H. Zhu, M.-L. Zhang, "Multi-instance multi-label learning with application to scene classification", NIPS, 2006.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICIMCS '11: Proceedings of the Third International Conference on Internet Multimedia Computing and Service
August 2011
208 pages
ISBN:9781450309189
DOI:10.1145/2043674
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • Sichuan University
  • Chinese Academy of Sciences
  • SCF: Sichuan Computer Federation
  • Southwest Jiaotong University
  • Beijing ACM SIGMM Chapter

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 August 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. multi-instance learning
  2. object recognition

Qualifiers

  • Research-article

Conference

ICIMCS '11
Sponsor:
  • SCF

Acceptance Rates

Overall Acceptance Rate 163 of 456 submissions, 36%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 24 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media