Abstract
This paper describes an extension of a symbolic knowledge extraction approach for Linear Support Vector Machine [1]. The proposed method retrieves a set of concise and interpretable IF-THEN rules from a novel polytope classifier, which can be described as a Piecewise-Linear Support Vector Machine with the successful application for linearly non-separable classification problems. Recent major achievements in rule extraction for kernelized classifiers left some reasonable and unresolved problems in knowledge discovery from nonlinear SVMs. The most comprehensible methods imply constraints that strictly enforce convexity of the searched-through half-space of inducted SVM classifier [2]. Obviously non-convex hyper-surfaces couldn’t be effectively described by a finite set of IF-THEN rules without violating bounds of a constrained non-convex area. In this paper we describe two different approaches for ”learning” a polytope classifier. One of them uses Multi-Surface Method Tree [3] to generate decision half-spaces, while the other one enables clustering-based decomposition of target classes and initiates a separate Linear SVM for every pair of clusters. We claim that the proposed polytope classifier achieves classification rates comparable to a nonlinear SVM and corresponding rule extraction approach helps to extract better rules from linearly non-separable cases in comparison with decision trees and C4.5 rule extraction algorithm.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fung, G., Sandilya, S., Bharat Rao, R.: Rule Extraction from Linear Support Vector Machines. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, Illinois, USA, April 21-24, pp. 32–40 (2005)
Ren, L., Garcez, A.: Symbolic Knowledge Extraction from Support Vector Machines: A Geometric Approach. In: Köppen, M., Kasabov, N., Coghill, G. (eds.) ICONIP 2008. LNCS, vol. 5507, pp. 335–343. Springer, Heidelberg (2009)
Mangasarian, O.–L.: Mathematical Programming in Neural Networks. ORSA Journal on Computing 5(4), 349–360 (1993)
Nunez, H., Angulo, C., Catala, A.: Rule extraction from support vector machines. In: Proceedings of the 10th European Symposium on Artificial Neural Networks, Bruges, Belgium, April 24-26, pp. 107–112 (2002)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995)
Lloyd, S.–P.: Least squares quantization in PCM. IEEE Transactions on Information Theory 28(2), 129–137 (1982)
Grunbaum B.: Convex Polytopes, 2nd edn., prepared by Kaibel, V., Klee, V., Ziegler, G.M. (2003)
Bennet K.–P.: Desicion Tree Construction via Linear Programming. In: Proceedings of the 4th Midwest Artificial Intelligence and Cognitive Science Society Conference, pp. 97–101 (1992)
Frank, A., Asuncion, A.: UCI Machine Learning Repository, School of Information and Computer Science, Irvine, University of California, USA (2010), http://archive.ics.uci.edu/ml
Thompson M.–E.: NDCC: Normally Distributed Clustered Datasets on Cubes, Computer Sciences Department, University of Wisconsin, Madison, USA (2010), http://www.cs.wisc.edu/dmi/svm/ndcc/
Rakotomamonjy, A., et al.: SimpleMKL. Journal of Machine Learning Research 9, 2491–2521 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jumutc, V., Bondarenko, A. (2011). Polytope Classifier: A Symbolic Knowledge Extraction from Piecewise-Linear Support Vector Machine. In: König, A., Dengel, A., Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based and Intelligent Information and Engineering Systems. KES 2011. Lecture Notes in Computer Science(), vol 6881. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23851-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-23851-2_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23850-5
Online ISBN: 978-3-642-23851-2
eBook Packages: Computer ScienceComputer Science (R0)