Abstract
Support Vector Machines (SVMs) are traditionally used for multi-class classification by introducing for each class one SVM trained to distinguish the associated class from all the others. In a recent experiment, we attempted to solve a K-class problem using a similar decomposition with K feedforward binary neural networks. The disappointing results were explained by the fact that neural networks suffer from datasets with a strongly unbalanced class distribution. By opposition to one-per-class, pairwise coupling introduces one binary classifier for each pair of classes and does not degrade the original class distribution. A few papers report evidences that pairwise coupling gives better results for SVMs than one-per-class. This issue is revisited in this paper where one-per-class class and pairwise coupling decomposition schemes used with both, SVMs and neural networks, are compared on a real life problem. Various methods for aggregating the results of pairwise classifiers are experimented. Beside our online handwriting application, experiments on some databases of the Irvine repository are also reported.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ethem Alpaydin. Combined 5 × 2 cv f test for comparing supervised classification learning algorithms. Neural Computation, 11(8):1885–1892, 1999.
Hervé A. Bourlard and Nelson Morgan. Links between markov models and multilayer perceptrons. In Ed. D. S. Touretzky, editor, Advances in Neural Information Processing Systems 1 (NIPS*88), volume 1, pages 502–510, San Mateo, CA, 1989. Morgan Kaufmann.
Pierre J. Castellano, Stefan Slomka, and Sridha Sridharan. Telephone based speaker recognition using multiplt binary classifier and Gaussian mixture models. In ICASSP, volume 2, pages 1075–1078. IEEE Computer Society Press, 1997.
Ronan Collobert and Samy Bengio. Support vector machines for large-scale regression problems. Journal of Machine Learning Research, 1:143–160, 2001. http://www.idiap.ch/learning/SVMTorch.html.
T. G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 7, 1998.
T. G. Dietterich and G. Bakiri. Error-correcting output codes: A general method for improving multiclass inductive learning programs. In Proceedings of AAAI-91, pages 572–577. AAAI Press / MIT Press, 1991.
Dominique Genoud, Miguel Moreira, and Eddy Mayoraz. Text dependent speaker verification using binary classifiers. IDIAP-RR 8, IDIAP, 1997. To appear in the Proceedings of the International Conference on Automatic Speech and Signal Processing, ICASSP’98.
Stefan Knerr, Leon Personnaz, and Gerard Dreyfus. Handwritten digit recognition by neural networks with single-layer training. IEEE Trans. on Neural Networks, 3(6):962–968, November 1992.
Ulrich Kreßel. Pairwise classification and support vector machines. In B. Schlkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods — Support Vector Learning. MIT Press, 1999.
Eddy Mayoraz and Miguel Moreira. On the decomposition of polychotomies into dichotomies. In Douglas H. Fisher, editor, The Fourteenth International Conference on Machine Learning, pages 219–226, 1997.
Eddy Mayoraz and Ethem Alpaydin. Support vector machine for multiclass classification. In Proceedings of the International Workshop on Artificial Neural Networks (IWANN’99), 1998. ftp://ftp.idiap.ch/pub/reports/1998/rr98-06.ps.gz.
C. J. Merz and P. M. Murphy. UCI repository of machine learning databases. Technical report, Irvine, CA: University of California, Department of Information and Computer Science, 1998. Machine-readable data repository http://www.ics.uci.edu/$¡m$mlearn/MLRepository.html.
J. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin dags for multiclass classification. In S.A. Solla, T.K. Leen, and K.-R. Muller, editors, Advances in Neural Information Processing Systems 12 (NIPS*1999). MIT Press, 2000.
David Price, Stefan Knerr, Leon Personnaz, and Gerard Dreyfus. Pairwise neural network classifiers with probabilistic outputs. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems 7 (NIPS*94), volume 7, pages 1109–1116. The MIT Press, 1995.
M. Schmidt and H. Gish. Speaker identification via support vector classifiers. In Proceedings of ICASSP’96, pages 105–108, Atlanta, GA, 1996.
B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In U. M. Fayyad and R. Uthurusamy, editors, Proceedings of the First International Conference on Knowledge Discovery and Data Mining, pages 252–257. AAAI Press, 1995.
Fusi Wang, Louis Vuurpijl, and Lambert Schomaker. Support vector machines for the classification of western handwritten capitals. In Seventh International Workshop on Frontiers in Handwriting Recognition Proceedings, pages 167–176, 2000.
H. White. Learning in artificial neural networks: A statistical perspective. Neural Computation, 1(4):425–464, 1989.
Stephen A. Zahorian, Peter Silsbee, and Xihong Wang. Phone classification with segmental features and a binary-pair partitioned neural network classifier. In ICASSP, volume 2, pages 1011–1014. IEEE Computer Society Press, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mayoraz, E.N. (2001). Multiclass Classification with Pairwise Coupled Neural Networks or Support Vector Machines. In: Dorffner, G., Bischof, H., Hornik, K. (eds) Artificial Neural Networks — ICANN 2001. ICANN 2001. Lecture Notes in Computer Science, vol 2130. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44668-0_45
Download citation
DOI: https://doi.org/10.1007/3-540-44668-0_45
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42486-4
Online ISBN: 978-3-540-44668-2
eBook Packages: Springer Book Archive