Summary
Support Vector Machines (SVMs) are a set of related methods for supervised learning, applicable to both classification and regression problems. A SVM classifiers creates a maximum-margin hyperplane that lies in a transformed input space and splits the example classes, while maximizing the distance to the nearest cleanly split examples. The parameters of the solution hyperplane are derived from a quadratic programming optimization problem. Here, we provide several formulations, and discuss some key concepts.
Key words
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bazaraa M. S., Sherali H. D., and Shetty C. M. Nonlinear programming: theory and algorithms. Wiley, second edition, 1993.
Bertsekas D.P. Nonlinear Programming. Athena Scientific, MA, 1995.
Chang C.-C. and Lin C.-J. Training support vector classifiers: Theory and algorithms. Neural Computation 2001; 13(9):2119–2147.
Chang C.-C. and Lin C.-J. (2001). LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm.
Chen P.-H., Lin C. -J., and Scholkopf B. A tutorial on nu-support vector machines. 2003.
Chew H. G., Lim C. C., and Bogner R. E. An implementation of training dual-nu support vector machines. In Qi, Teo, and Yang, editors, Optimization and Control with Applications. Kluwer, 2003.
Chu W. Bayesian approach to support vector machines. PhD thesis, National University of Singapore , 2003; Available online http://citeseer.ist.psu.edu/chu03bayesian.html
Chung K.-M., Kao W.-C., Sun C.-L., and Lin C.-J. Decomposition methods for linear support vector machines. Neural Computation 2004; 16(8):1689-1704).
Cortes C. and Vapnik V. Support vector networks. Machine Learning 1995; 20:273–297.
Cristianini N. and Shawe-Taylor J. An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge Univ. Press, 2000.
Dumais S. Using SVMs for text categorization. IEEE Intelligent Systems 1998; 13(4).
Hsu C.-W. and Lin C.-J. A comparison of methods for multi-class support vector machines IEEE Transactions on Neural Networks 2002; 13(2); 415–425.
Hsu C.-W. Chang C.-C and Lin C.-J. A practical guide to support vector classification. 2003. Available Online: www.csie.ntu.edu.tw/∼cjlin/papers/guide/guide.pdf
Isabelle 2004, (a collection of SVM applications) Available Online: http://www.clopinet.com/isabelle/Projects/SVWM/applist.html
Joachims T. Making large–scale SVM learning practical. In Scholkopf B., Burges C. J. C., and Smola A. J., editors, Advances in Kernel Methods — Support Vector Learning, pages 169–184, Cambridge, MA, MIT Press, 1999.
Joachims T. Learning to Classify Text using Support Vector Machines Methods, Theory, and Algorithms. Kluwer Academic Publishers, 2002.
Joachims T. 2004, SVMlight, available online http://www.cs.cornell.edu/People/tj/svmlight/
Kernel 2004, (a collection of literature, software and Web pointers dealing with SVM and Gaussian processes) Available Online http://www.kernel-machines.org.
Law M. H. and Kwok J. T. Bayesian support vector regression. Proceedings of the 8th International Workshop on Artificial Intelligence and Statistics (AISTATS) pages 239-244, Key-West, Florida, USA, January 2000.
Lin C.-J. Formulations of support vector machines: a note from an optimization point of view. Neural Computation 2001; 13(2):307–317.
Lin C.-J. On the convergence of the decomposition method for support vector machines. IEEE Transactions on Neural Networks 2001; 12(6):1288–1298.
Martin D. R., Fowlkes C. C., and Malik J. Learning to detect natural image boundaries using brightness and texture. In Advances in Neural Information Processing Systems, volume 14, 2002.
Mukherjee S., Osuna E., and Girosi F. Nonlinear prediction of chaotic time series using a support vector machine. In Principe J., Gile L., Morgan N. and Wilson E. editors, Neural Networks for Signal Processing VII - proceedings of the 1997 IEEE Workshop, pages 511–520, New-York, IEEE Press, 1997.
Muller K.-R., Mika S., Ratsch G., Tsuda K., and Scholkopf B., An introduction to kernel-based learning algorithms. IEEE Neural Networks 2001; 12(2):181-201.
Osuna E., Freund R., and Girosi F. An improved training algorithm for support vector machines. In Principe J., Gile L., Morgan N. and Wilson E. editors, Neural Networks for Signal Processing VII - proceedings of the 1997 IEEE Workshop, pages 276-285, New-York, IEEE Press, 1997.
Platt J. C. Fast training of support vector machines using sequential minimal optimization. In Scholkopf B., Burges C. J. C., and Smola A. J., editors, Advances in Kernel Methods - Support Vector Learning, Cambridge, MA, MIT Press, 1998.
Ratsch G., Onoda T., and Muller K.R. Soft margins for AdaBoost. Machine Learning 2001; 42(3):287–320.
Rifkin R. and Klautau A.. In Defense of One-vs-All Classification, Journal of Machine Learning Research 2004; 5:101-141.
Scholkopf B., Support Vector Learning. Oldenbourg Verlag, Munich, 1997.
Scholkopf B., Statistical learning and kernel methods, Technical Report MSRTR-2000-23, Available Online http://research.microsoft.com/research/pubs/view.aspx?msr tr id= MSR-TR-2000-23
Scholkopf B., Burges C.J.C., and Vapnik V.N. Extracting support data for a given task. In Fayyad U.M. and Uthurusamy R., Editors, Proceedings, First International Conference on Knowledge Discovery and Data Mining. AAAI Press, Menlo Park, CA, 1995.
Scholkopf B., Simard P.Y., Smola A.J., and Vapnik V.N.. Prior knowledge in support vector kernels. In Jordan M., Kearns M., and Solla S., Editors, Advances in Neural Information Processing Systems 10, pages 640–646. MIT Press, Cambridge, MA, 1998.
Scholkopf B., Burges C. J. C., and Smola A. J., editors, Advances in Kernel Methods - Support Vector Learning, Cambridge, MA, MIT Press, 1999.
Scholkopf B. and Smola A. J. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
Scholkopf B., Smola A. J., Williamson R. C., and Bartlett P. L. New support vector algorithms. Neural Computation 2000; 12:1207–1245.
Shawe-Taylor J. and Cristianini N. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
Smola A. J., Bartlett P. L., Scholkopf B. and Schuurmans D. Advances in Large Margin Classifiers. MIT Press, Cambridge, MA, 2000.
Smola A.J. and Scholkopf B.. A tutorial on support vector regression. Statistics and Computing 2004; 14(13):199-222.
Smola A.J., Scholkopf B. and Ratsch G. Linear programs for automatic accuracy control in regression. Proceedings of International Conference on Artificial Neural Networks ICANN’99, Berlin, Springer 1999.
Steinwart I. On the optimal parameter choice for nu-support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003; 25: 1274-1284.
Steinwart I. Sparseness of support vector machines. Journal of Machine Learning Research 2004; 4(6):1071-1105.
Suykens J.A.K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J. Least Squares Support Vector Machines. World Scientific Publishing, Singapore, 2002.
Vapnik V. The Nature of Statistical Learning Theory . Springer Verlag, New York, 1995.
Vapnik V. Statistical Learning Theory. Wiley, NY, 1998.
Vapnik V. and Chapelle O. Bounds on error expectation for support vector machines. Neural Computation 2000; 12(9):2013–2036.
Weston J. and Herbrich R., Adaptive margin support vector machines. In Smola A.J., Bartlett P.L., Scholkopf B., and Schuurmans D., Editors, Advances in Large Margin Classifiers, pages 281–296, MIT Press, Cambridge, MA, 2000,.
Williamson R. C., Smola A. J., and Scholkopf B., Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information Theory 2001; 47(6):2516–2532.
Wolfe P. A duality theorem for non-linear programming. Quartely of Applied Mathematics 1961; 19:239–244.
Zien A., Ratsch G., Mika S., Scholkopf B., Lengauer T. and Muller K.R. Engineering support vector machine kernels that recognize translation initiation sites. Bio-Informatics 248 Armin Shmilovici 16(9):799–807.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Shmilovici, A. (2009). Support Vector Machines. In: Maimon, O., Rokach, L. (eds) Data Mining and Knowledge Discovery Handbook. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-09823-4_12
Download citation
DOI: https://doi.org/10.1007/978-0-387-09823-4_12
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-09822-7
Online ISBN: 978-0-387-09823-4
eBook Packages: Computer ScienceComputer Science (R0)