skip to main content
10.1145/1143844.1143868acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmlConference Proceedingsconference-collections
Article

A continuation method for semi-supervised SVMs

Published: 25 June 2006 Publication History

Abstract

Semi-Supervised Support Vector Machines (S3VMs) are an appealing method for using unlabeled data in classification: their objective function favors decision boundaries which do not cut clusters. However their main problem is that the optimization problem is non-convex and has many local minima, which often results in suboptimal performances. In this paper we propose to use a global optimization technique known as continuation to alleviate this problem. Compared to other algorithms minimizing the same objective function, our continuation method often leads to lower test errors.

References

[1]
Astorino, A., & Fuduli, A. (2005). Nonsmooh optimization techniques for semi-supervised classification (Technical Report). U. Pisa, Dipartimento di Matematica. www.dm.unipi.it/mat_dia_med/PAPER_Astorino.pdf.]]
[2]
Bennett, K., & Demiriz, A. (1998). Semi-supervised support vector machines. Advances in Neural Information processing systems 12.]]
[3]
Chapelle, O. (2006). Training a support vector machine in the primal (Technical Report 147). Max Planck Institute. www.kyb.tuebingen.mpg.de/bs/people/chapelle/primal.]]
[4]
Chapelle, O., Weston, J., & Schöölkopf, B. (2002). Cluster kernels for semi-supervised learning. Advances in Neural Information Processing Systems 15.]]
[5]
Chapelle, O., & Zien, A. (2005). Semi-supervised classification by low density separation. Tenth International Workshop on Artificial Intelligence and Statistics.]]
[6]
Collobert, R., Sinz, F., Weston, J., & Bottou, L. (2006). Large scale transductive SVMs. Journal of Machine Learning Research. Submitted, www.kyb.tuebingen.mpg.de/bs/people/fabee/universvm.html.]]
[7]
Cortes, C., & Vapnik, V. (1995). Support vector network. Machine learning, 20, 1--25.]]
[8]
Fung, G., & Mangasarian, O. (2001). Semi-supervised support vector machines for unlabeled data classification. Optimization Methods and Software, 29--44.]]
[9]
Joachims, T. (1999). Transductive inference for text classification using support vector machines. International Conference on Machine Learning.]]
[10]
Joachims, T. (2003). Transductive learning via spectral graph partitioning. International Conference on Machine Learning.]]
[11]
Keerthi, S. S., & DeCoste, D. M. (2005). A modified finite Newton method for fast solution of large scale linear SVMs. Journal of Machine Learning Research, 6, 341--361.]]
[12]
Kimeldorf, G., & Wahba, G. (1971). Some results on tchebycheffian spline functions. J. Math. Anal. Applic., 33, 82--95.]]
[13]
Ng, A. Y., Jordan, M. I., & Weiss, Y. (2001). On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems.]]
[14]
Schölkopf, B., & Smola, A. (2002). Learning with kernels. Cambridge, MA: MIT Press.]]
[15]
Seeger, M. (2006). A taxonomy of semi-supervised learning methods. In O. Chapelle, B. Schölkopf and A. Zien (Eds.), Semi-supervised lerning. MIT Press.]]
[16]
Sindhwani, V., Keerthi, S., & Chapelle, O. (2006). Deterministic annealing for semi-supervised kernel machines. International Conference on Machine Learning.]]
[17]
Sindhwani, V., Niyogi, P., & Belkin, M. (2005). Beyond the point cloud: From transductive to semi-supervised learning. International Conference on Machine Learning.]]
[18]
Vapnik, V., & Sterin, A. (1977). On structural risk minimization or overall risk in a problem of pattern recognition. Automation and Remote Control, 10, 1495--1503.]]
[19]
Vapnik, V. N. (1998). Statistical learning theory. New York: John Wiley & Sons, Inc.]]
[20]
Wu, Z. (1996). The effective energy transformation scheme as a special continuation approach to global optimization with application to molecular conformation. SIAM Journal on Optimization, 6, 748--768.]]

Cited By

View all
  • (2024)Weakly Supervised AUC Optimization: A Unified Partial AUC ApproachIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.335781446:7(4780-4795)Online publication date: Jul-2024
  • (2024)UPMatch: Enhancing Semi-Supervised Medical Image Classification through Contrastive Learning with Unreliable Pseudo Labels2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)10.1109/EMBC53108.2024.10782507(1-5)Online publication date: 15-Jul-2024
  • (2024)Multi-head co-trainingKnowledge-Based Systems10.1016/j.knosys.2024.112325302:COnline publication date: 25-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICML '06: Proceedings of the 23rd international conference on Machine learning
June 2006
1154 pages
ISBN:1595933832
DOI:10.1145/1143844
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2006

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Article

Acceptance Rates

ICML '06 Paper Acceptance Rate 140 of 548 submissions, 26%;
Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)25
  • Downloads (Last 6 weeks)5
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Weakly Supervised AUC Optimization: A Unified Partial AUC ApproachIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.335781446:7(4780-4795)Online publication date: Jul-2024
  • (2024)UPMatch: Enhancing Semi-Supervised Medical Image Classification through Contrastive Learning with Unreliable Pseudo Labels2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)10.1109/EMBC53108.2024.10782507(1-5)Online publication date: 15-Jul-2024
  • (2024)Multi-head co-trainingKnowledge-Based Systems10.1016/j.knosys.2024.112325302:COnline publication date: 25-Oct-2024
  • (2024)Mixed-integer quadratic optimization and iterative clustering techniques for semi-supervised support vector machinesTOP10.1007/s11750-024-00668-w32:3(391-428)Online publication date: 16-May-2024
  • (2024)Machine learning meets enzyme engineering: examples in the design of polyethylene terephthalate hydrolasesFrontiers of Chemical Science and Engineering10.1007/s11705-024-2500-718:12Online publication date: 10-Sep-2024
  • (2024)Optimization meets machine learning: an exact algorithm for semi-supervised support vector machinesMathematical Programming10.1007/s10107-024-02175-zOnline publication date: 19-Dec-2024
  • (2024)Semi-supervised fuzzy broad learning system based on mean-teacher modelPattern Analysis and Applications10.1007/s10044-024-01217-827:1Online publication date: 28-Feb-2024
  • (2023)Continuation path learning for homotopy optimizationProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619285(21288-21311)Online publication date: 23-Jul-2023
  • (2023)A Systematic Review of Application Progress on Machine Learning-Based Natural Language Processing in Breast Cancer over the Past 5 YearsDiagnostics10.3390/diagnostics1303053713:3(537)Online publication date: 1-Feb-2023
  • (2023)Semi-Supervised Graph Learning Meets Dimensionality Reduction2023 International Conference on Machine Learning and Applications (ICMLA)10.1109/ICMLA58977.2023.00303(2006-2013)Online publication date: 15-Dec-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media