Abstract
SVM-related feature selection has shown to be effective, while feature selection with transductive SVMs has been less studied. This paper investigates the use of transductive SVMs for feature selection, based on three SVM-related feature selection methods: filtering scores + SVM wrapper, recursive feature elimination (RFE) and multiplicative updates(MU). We show transductive SVMs can be tailored to feature selection by embracing feature scores for feature filtering, or acting as wrappers and embedded feature selectors. We conduct experiments on the feature selection competition tasks to demonstrate the performance of Transductive SVMs in feature selection and classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
K. Bennett and A. Demiriz. Semi-supervised support vector machines. In NIPS, 1998.
Bernhard E. Boser, Isabelle Guyon, and Vladimir Vapnik. A training algorithm for optimal margin classifiers. In Computational Learing Theory, pages 144–152, 1992.
Janez Brank, Marko Grobelnik, N. Milic-Frayling, and Dunja Mladenic. Feature selection using linear support vector machines. Technical report, 2002.
C.-C. Chang and C.-J. Lin. Libsvm: a library for svms, 2001.
George Forman. An extensive empirical study of feature selection metrics for text classification (kernel machines section). JMLR, pages 1289–1305, 2003.
G. Fung and O. Mangasarian. Semi-supervised support vector machines for unlabeled data classification. In Optimization Methods and Software, 2001.
Isabelle Guyon, Hans-Marcus Bitter, Zulfikar Ahmed, Michael Brown, and Jonathan Heller. Multivariate non-linear feature selection with kernel multiplicative updates and gram-schmidt relief. In BISC FLINT-CIBI 2003 workshop, 2003.
Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46(1–3):389–422, 2002.
T. Jebara and T. Jaakkola. Feature selection and dualities in maximum entropy discrimination. In Uncertainity In Artificial Intellegence, 2000.
T. Joachims. Making large-scale support vector machine learning practical. In Advances in Kernel Methods: Support Vector Machines, 1999a.
Thorsten Joachims. Transductive inference for text classification using support vector machines. In Proceedings of ICML-99, pages 200–209, 1999b.
D. Mladenic and M. Grobelnik. Feature selection for unbalanced class distribution and naive bayes. In ICML, 1999.
Simon Perkins, Kevin Lacker, and James Theiler. Grafting: Fast, incremental feature selection by gradient descent in function space (kernel machines section). JMLR, pages 1333–1356, 2003.
J. Platt. Sequential minimal optimization: A fast algorithm for training support vector machines, 1998.
V. Vapnik. Statistical Learning Theory. Wiley, 1998.
J. Weston, F. Perez-Cruz, O. Bousquet, O. Chapelle, A. Elisseeff, and B. Schoelkopf. Feature selection and transduction for prediction of molecular bioactivity for drug design. Bioinformatics, 2002.
“Jason Weston, André Elisseeff, Bernhard Schöllkopf, and Mike Tipping”. Use of the zero-norm with linear models and kernel methods. JMLR, 2003.
Jason Weston, Sayan Mukherjee, Olivier Chapelle, Massimiliano Pontil, Tomaso Poggio, and Vladimir Vapnik. Feature selection for SVMs. In NIPS, pages 668–674, 2000.
S. Wu and P. A. Flach. Feature selection with labelled and unlabelled data. In ECML/PKDD’02, 2002.
E. Yu and S. Cho. Ga-svm wrapper approach for feature subset selection in keystroke dynamics identity verification. In INNS-IEEE International Joint Conference on Neural Networks, 2003.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Wu, Z., Li, C. (2006). Feature Selection with Transductive Support Vector Machines. In: Guyon, I., Nikravesh, M., Gunn, S., Zadeh, L.A. (eds) Feature Extraction. Studies in Fuzziness and Soft Computing, vol 207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-35488-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-540-35488-8_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35487-1
Online ISBN: 978-3-540-35488-8
eBook Packages: EngineeringEngineering (R0)