research-article

Semi-supervised learning using label mean

Authors:

Zhi-Hua ZhouAuthors Info & Claims

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

Pages 633 - 640

https://doi.org/10.1145/1553374.1553456

Published: 14 June 2009 Publication History

Abstract

Semi-Supervised Support Vector Machines (S3VMs) typically directly estimate the label assignments for the unlabeled instances. This is often inefficient even with recent advances in the efficient training of the (supervised) SVM. In this paper, we show that S3VMs, with knowledge of the means of the class labels of the unlabeled data, is closely related to the supervised SVM with known labels on all the unlabeled data. This motivates us to first estimate the label means of the unlabeled data. Two versions of the meanS3VM, which work by maximizing the margin between the label means, are proposed. The first one is based on multiple kernel learning, while the second one is based on alternating optimization. Experiments show that both of the proposed algorithms achieve highly competitive and sometimes even the best performance as compared to the state-of-the-art semi-supervised learners. Moreover, they are more efficient than existing S3VMs.

References

[1]

Bach, R. R., Lanckriet, G. R. G., & Jordan, M. I. (2004). Multiple kernel learning, conic duality, and the SMO algorithm. Proceeding of the 21st International Conference on Machine Learning (pp. 41--48).

Digital Library

[2]

Belkin, M., Niyogi, P., & Sindhwani, V. (2006). Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7, 2399--2434.

Digital Library

[3]

Bennett, K., & Demiriz, A. (1999). Semi-supervised support vector machines. In Advances in Neural Information Processing Systems 11, 368--374.

Digital Library

[4]

Chapelle, O., Schölkopf, B., & Zien, A. (Eds.). (2006). Semi-supervised learning. Cambridge, MA: MIT Press.

[5]

Chapelle, O., Sindhwani, V., & Keerthi, S. S. (2008). Optimization techniques for semi-supervised support vector machines. Journal of Machine Learning Research, 9, 203--233.

Digital Library

[6]

Chapelle, O., & Zien, A. (2005). Semi-supervised learning by low density separation. Proceeding of the 8th International Conference on Artificial Intelligence and Statistics (pp. 57--64).

[7]

Grandvalet, Y., Rakotomamonjy, A., Keshet, J., & Canu, S. (2009). Support vector machines with a reject option. In Advances in Neural Information Processing Systems 21, 537--544.

[8]

Gretton, A., Borgwardt, K. M., Rasch, M., Schölkopf, B., & Smola, A. J. (2006). A kernel method for the two-sample-problem. In Advances in Neural Information Processing Systems 19, 513--520.

[9]

Joachims, T. (1999). Transductive inference for text classification using support vector machines. Proceeding of the 16th International Conference on Machine Learning (pp. 200--209).

Digital Library

[10]

Joachims, T., Finley, T., & Yu, C.-N. (2009). Cutting-plane training of structural SVMs. Machine Learning, to appear.

[11]

Kelly, J. E. (1960). The cutting plane method for solving convex programs. Journal of Society for Industrial and Applied Mathematics, 8, 703--712.

[12]

Lanckriet, G. R. G., Cristianini, N., Bartlett, P., Ghaoui, L. E., & Jordan, M. I. (2004). Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research, 5, 27--72.

Digital Library

[13]

Li, Y.-F., Tsang, I. W., Kwok, J. T., & Zhou, Z.-H. (2009). Tighter and convex maximum margin clustering. Proceeding of the 12th International Conference on Artificial Intelligence and Statistics (pp. 344--351).

[14]

Mallapragada, P., Jin, R., Jain, A., & Liu, Y. (2009). Semi-Boost: Boosting for semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, to appear.

Digital Library

[15]

Rakotomamonjy, A., Bach, F. R., Canu, S., & Grandvalet, Y. (2008). SimpleMKL. Journal of Machine Learning Research, 9, 2491--2521.

[16]

Sonnenburg, S., Rätsch, G., Schäfer, C., & Schöölkopf, B. (2006). Large scale multiple kernel learning. Journal of Machine Learning Research, 7, 1531--1565.

Digital Library

[17]

Tsochantaridis, I., Joachims, T., Hofmann, T., & Altun, Y. (2005). Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research, 6, 1453--1484.

Digital Library

[18]

Weston, J., Leslie, C., Ie, E., Zhou, D., Elisseeff, A., & Noble, W. S. (2005). Semi-supervised protein classification using cluster kernels. Bioinformatics, 21, 3241--3247.

Digital Library

[19]

Xu, Z., Jin, R., King, I., & Lyu, M. R. (2009). An extended level method for efficient multiple kernel learning. In Advances in Neural Information Processing Systems 21, 1825--1832.

Cited By

Zhang YLiu LQiao QLi F(2025)A Lie group Laplacian Support Vector Machine for semi-supervised learningNeurocomputing10.1016/j.neucom.2025.129728630(129728)Online publication date: May-2025
https://doi.org/10.1016/j.neucom.2025.129728
Qian YZhang ZZhao PZhou Z(2024)Learning with Asynchronous LabelsACM Transactions on Knowledge Discovery from Data10.1145/366218618:8(1-27)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1145/3662186
Pourpanah FLim CEtemad AWu Q(2024)An Ensemble Semi-Supervised Adaptive Resonance Theory Model With Explanation Capability for Pattern ClassificationIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2023.32859328:1(814-827)Online publication date: Feb-2024
https://doi.org/10.1109/TETCI.2023.3285932
Show More Cited By

Index Terms

Semi-supervised learning using label mean
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
      2. Unsupervised learning
        Cluster analysis
    2. Machine learning approaches
      1. Classification and regression trees
  2. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
      2. Modeling methodologies

Recommendations

Semi-supervised multi-label classification using incomplete label information
Highlights
- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
Abstract
Classifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...
Semi-supervised partial label learning algorithm via reliable label propagation
Abstract
Partial label learning (PLL) is a weakly supervised learning method that is able to predict one label as the correct answer from a given candidate label set. In PLL, when all possible candidate labels are as signed to real-world training examples, ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

In multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

June 2009

1331 pages

ISBN:9781605585161

DOI:10.1145/1553374

General Chair:
Andrea Danyluk
Williams College
,
Program Chairs:
Léon Bottou
NEC Laboratories America
,
Michael Littman
Rutgers University

Copyright © 2009 Copyright 2009 by the author(s)/owner(s).

Sponsors

NSF
Microsoft Research: Microsoft Research
MITACS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Conference

ICML '09

Sponsor:

Microsoft Research

ICML '09: The 26th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 14 - 18, 2009

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

74
Total Citations
View Citations
572
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YLiu LQiao QLi F(2025)A Lie group Laplacian Support Vector Machine for semi-supervised learningNeurocomputing10.1016/j.neucom.2025.129728630(129728)Online publication date: May-2025
https://doi.org/10.1016/j.neucom.2025.129728
Qian YZhang ZZhao PZhou Z(2024)Learning with Asynchronous LabelsACM Transactions on Knowledge Discovery from Data10.1145/366218618:8(1-27)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1145/3662186
Pourpanah FLim CEtemad AWu Q(2024)An Ensemble Semi-Supervised Adaptive Resonance Theory Model With Explanation Capability for Pattern ClassificationIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2023.32859328:1(814-827)Online publication date: Feb-2024
https://doi.org/10.1109/TETCI.2023.3285932
Luo YLiu JZhang C(2023)Detecting genuine multipartite entanglement via machine learningPhysical Review A10.1103/PhysRevA.108.052424108:5Online publication date: 21-Nov-2023
https://doi.org/10.1103/PhysRevA.108.052424
Merkurjev E(2023)Efficient graph-based spectral techniques for data with few labeled samplesInternational Journal of Data Science and Analytics10.1007/s41060-023-00403-x18:2(113-138)Online publication date: 1-Jul-2023
https://doi.org/10.1007/s41060-023-00403-x
Sun JQu W(2022)DCA for Sparse Quadratic Kernel-Free Least Squares Semi-Supervised Support Vector MachineMathematics10.3390/math1015271410:15(2714)Online publication date: 1-Aug-2022
https://doi.org/10.3390/math10152714
Zhu LHill DLu C(2022)Semi-Supervised Ensemble Learning Framework for Accelerating Power System Transient Stability Knowledge Base GenerationIEEE Transactions on Power Systems10.1109/TPWRS.2021.311740237:3(2441-2454)Online publication date: May-2022
https://doi.org/10.1109/TPWRS.2021.3117402
Gong CYang JYou JSugiyama M(2022)Centroid Estimation With Guaranteed Efficiency: A General Framework for Weakly Supervised LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.304499744:6(2841-2855)Online publication date: 1-Jun-2022
https://doi.org/10.1109/TPAMI.2020.3044997
Jia WLiu XWang YPedrycz WZhou J(2022)Semisupervised Learning via Axiomatic Fuzzy Set Theory and SVMIEEE Transactions on Cybernetics10.1109/TCYB.2020.303270752:6(4661-4674)Online publication date: Jun-2022
https://doi.org/10.1109/TCYB.2020.3032707
Zhong JZeng XCao WWu SLiu CYu ZWong H(2022)Semisupervised Multiple Choice Learning for Ensemble ClassificationIEEE Transactions on Cybernetics10.1109/TCYB.2020.301604852:5(3658-3668)Online publication date: May-2022
https://doi.org/10.1109/TCYB.2020.3016048
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten