Abstract
Multi-class classification can often be constructed as a generalization of binary classification. The approach that we use for solving this kind of classification problem is SVM based Binary Decision Tree architecture (SVM-BDT). It takes advantage of both the efficient computation of the decision tree architecture and the high classification accuracy of SVMs. The hierarchy of binary decision subtasks using SVMs is designed with a clustering algorithm. In this work, we are investigating how different distance measures for the clustering influence the predictive performance of the SVM-BDT. The distance measures that we consider include Euclidian distance, Standardized Euclidean distance and Mahalanobis distance. We use five different datasets to evaluate the performance of the SVM based Binary Decision Tree architecture with different distances. Also, the performance of this architecture is compared with four other SVM based approaches, ensembles of decision trees and neural network. The results from the experiments suggest that the performance of the architecture significantly varies depending of applied distance measure in the clustering process.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1999)
Burges, C.J.C.: A tutorial on support vector machine for pattern recognition. Data Min. Knowl. Disc. 2, 121 (1998)
Joachims, T.: Making large scale SVM learning practical. In: Scholkopf, B., Bruges, C., Smola, A. (eds.) Advances in kernel methods-support vector learning. MIT Press, Cambridge (1998)
Madzarov, G., Gjorgjevikj, D., Chorbev, I.: A multi-class SVM classifier utilizing binry decision tree. An International Journal of Computing and Informatics, Informatica 33(2), 233–241 (2009)
Mahalanobis, P.: On tests and measures of group divergence I. Theoretical formulae, J. and Proc. Asiat. Soc. of Bengal 26, 541–588 (1930)
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Friedman, J.H.: Another approach to polychotomous classification. Technical report, Department of Statistics, Stanford University (1997)
Xu, P., Chan, A.K.: Support vector machine for multi-class signal classification with unbalanced samples. In: Proceedings of the IJCNN 2003, Portland, pp. 1116–1119 (2003)
Platt, J., Cristianini, N., Shawe-Taylor, J.: Large margin DAGSVMs for multiclass classification. Advances in Neural Information Processing Sys. 12, 547–553 (2000)
Fei, B., Liu, J.: Binary Tree of SVM: A New Fast Multiclass Training and Classification Algorithm. IEEE Transaction on neural net. 17(3) (May 2006)
Kocev, D., Vens, C., Struyf, J., Dzeroski, S.: Ensembles of multi-objective decision trees. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 624–631. Springer, Heidelberg (2007)
Blockeel, H., Struyf, J.: Efficient Algorithms for Decision Tree Cross-validation. Journal of Machine Learning Research 3, 621–650 (2002)
Collobert, R., Bengio, S., Mariethoz, J.: Torch: a modular machine learning software library, Technical Report IDIAP-RR 02-46, IDIAP (2002)
MNIST, MiniNIST, USA, http://yann.lecun.com/exdb/mnist
Gorgevik, D., Cakmakov, D.: An Efficient Three-Stage Classifier for Handwritten Digit Recognition. In: Proceedings of 17th ICPR 2004, August 23-26, vol. 4, pp. 507–510. IEEE Computer Society, Cambridge (2004)
Blake, C., Keogh, E., Merz, C.: UCI Repository of Machine Learning Databases (1998), http://archive.ics.uci.edu/ml/datasets.html
Statlog, http://archive.ics.uci.edu/ml/datasets/Letter+Recognition
Martinez, J.M. (ed.) MPEG Requirements Group, ISO/MPEG N4674, Overview of the MPEG-7 Standard, v 6.0, Jeju (March 2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Madzarov, G., Gjorgjevikj, D. (2010). Evaluation of Distance Measures for Multi-class Classification in Binary SVM Decision Tree. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2010. Lecture Notes in Computer Science(), vol 6113. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13208-7_55
Download citation
DOI: https://doi.org/10.1007/978-3-642-13208-7_55
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13207-0
Online ISBN: 978-3-642-13208-7
eBook Packages: Computer ScienceComputer Science (R0)