skip to main content
10.1145/2063576.2063674acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Semi-supervised SVMs for classification with unknown class proportions and a small labeled dataset

Published: 24 October 2011 Publication History

Abstract

In the design of practical web page classification systems one often encounters a situation in which the labeled training set is created by choosing some examples from each class; but, the class proportions in this set are not the same as those in the test distribution to which the classifier will be actually applied. The problem is made worse when the amount of training data is also small. In this paper we explore and adapt binary SVM methods that make use of unlabeled data from the test distribution, viz., Transductive SVMs (TSVMs) and expectation regularization/constraint (ER/EC) methods to deal with this situation. We empirically show that when the labeled training data is small, TSVM designed using the class ratio tuned by minimizing the loss on the labeled set yields the best performance; its performance is good even when the deviation between the class ratios of the labeled training set and the test set is quite large. When the labeled training data is sufficiently large, an unsupervised Gaussian mixture model can be used to get a very good estimate of the class ratio in the test set; also, when this estimate is used, both TSVM and EC/ER give their best possible performance, with TSVM coming out superior. The ideas in the paper can be easily extended to multi-class SVMs and MaxEnt models.

References

[1]
D. P. Bertsekas and J. N. Tsitsiklis. Parallel and Distributed Computation: Numerical Methods. Athena Scientific, 1997.
[2]
S. Bickel, M. Brckner, and T. Scheffer. Discriminative learning for differing training and test distributions. In ICML, pages 81--88, 2007.
[3]
L. Bruzzone, M. Chi, and M. Marconcini. A novel transductive SVM for semisupervised classification of remote-sensing images. IEEE-GRSS, 3363--3373, 2006.
[4]
O. Chapelle, B. Schölkopf, and A. Zien. Semi-Supervised Learning. MIT Press, 2006.
[5]
O. Chapelle, V. Sindhwani, and S. S. Keerthi. Optimization techniques for semi-supervised support vector machines. JMLR, 9:203--233, 2008.
[6]
O. Chapelle and A. Zien. Semi-supervised classification by low density separation. In AISTATS, pages 57--64, 2005.
[7]
Y. Chen, G. Wang, and S. Dong. Learning with progressive transductive support vector machine. Pattern Recognition Letters, 24:1845--1855, 2003.
[8]
T. G\"artner, Q. V. Le, S. Burton, A. J. Smola, and S. V. N. Vishwanathan. Large-scale multiclass transduction. In NIPS, 2005.
[9]
Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. In NIPS, 2004.
[10]
T. Joachims. Transductive inference for text classification using support vector machines. In ICML, 1999.
[11]
T. Joachims. Training linear SVMs in linear time. In KDD, 2006.
[12]
G. S. Mann and A. McCallum. Generalized expectation criteria for semi-supervised learning with weakly labeled data. JMLR, 11:955--984, 2010.
[13]
V. Sindhwani and S. S. Keerthi. Large scale semi-supervised linear SVMs. In SIGIR, pages 477--484, 2006.
[14]
Y. Wang and S.-T. Huang. Training TSVM with the proper number of positive samples. Pattern Recognition Letters, 26:2187--2194, 2005.
[15]
T. Zhang and F. J. Oles. A probability analysis on the value of unlabeled data for classification problems. In ICML, pages 1191--1198, 2000.

Cited By

View all
  • (2024)Graph Receptive Transformer Encoder for Text ClassificationIEEE Transactions on Signal and Information Processing over Networks10.1109/TSIPN.2024.3380362(1-13)Online publication date: 2024
  • (2021)HeteGCNProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441746(860-868)Online publication date: 8-Mar-2021
  • (2017)Inferring Individual Attributes from Search Engine Queries and Auxiliary InformationProceedings of the 26th International Conference on World Wide Web10.1145/3038912.3052629(293-301)Online publication date: 3-Apr-2017

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management
October 2011
2712 pages
ISBN:9781450307178
DOI:10.1145/2063576
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. classification
  2. support vector machines
  3. transductive and semi-supervised learning

Qualifiers

  • Research-article

Conference

CIKM '11
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Graph Receptive Transformer Encoder for Text ClassificationIEEE Transactions on Signal and Information Processing over Networks10.1109/TSIPN.2024.3380362(1-13)Online publication date: 2024
  • (2021)HeteGCNProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441746(860-868)Online publication date: 8-Mar-2021
  • (2017)Inferring Individual Attributes from Search Engine Queries and Auxiliary InformationProceedings of the 26th International Conference on World Wide Web10.1145/3038912.3052629(293-301)Online publication date: 3-Apr-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media