Abstract
Next to prediction accuracy, the interpretability of models is one of the fundamental criteria for machine learning algorithms. While high accuracy learners have intensively been explored, interpretability still poses a difficult problem, largely because it can hardly be formalized in a general way. To circumvent this problem, one can often find a model in a hypothesis space that the user regards as understandable or minimize a user-defined measure of complexity, such that the obtained model describes the essential part of the data. To find interesting parts of the data, unsupervised learning has defined the task of detecting local patterns and subgroup discovery. In this paper, the problem of detecting local classification models is formalized. A multi-classifier algorithm is presented that finds a global model that essentially describes the data, can be used with almost any kind of base learner and still provides an interpretable combined model.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Chan, P.K., Stolfo, S.: Experiments in multistrategy learning by meta-learning. In: Proceedings of the second international conference on informationand knowledge management, Washington, DC, pp. 314–323 (1993)
Dempster, P., Laird, N.M., Rubin, D.B.: Maximum-likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Ser. B 39, 1–38 (1977)
Freund, Y., Schapire, R.E.: Game theory, on-line prediction and boosting. In: Proceedings of the 9th Annual Conference on Computational Learning Theory, pp. 325–332 (1996)
Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: A statistical view of boosting. Technical report, Departement of Statistics, Stanford University, Stanford, California 94305, July 23(1998)
Fürnkranz, J.: From local to global patterns: Evaluation issues in rule learning algorithms. In: Morik, K., Boulicaut, J.-F., Siebes, A. (eds.) Detecting Local Patterns. Springer, Heidelberg (2005)
Gama, J., Brazdil, P.: Cascade generalization. Machine Learning 41(3), 315–343 (2000)
Garczarek, U.: Classification Rules in Standardized Partition Spaces. PhD thesis, Universität Dortmund (2002)
Guyon, I., Matic, N., Vapnik, V.: Discovering informative patterns and data cleaning. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, ch. 2, pp. 181–204. AAAI Press/The MIT Press, Menlo Park (1996)
Hand, D.: Pattern detection and discovery. In: Hand, D.J., Adams, N.M., Bolton, R.J. (eds.) Pattern Detection and Discovery. LNCS (LNAI), vol. 2447, p. 1. Springer, Heidelberg (2002)
Miller, G.: The magical number seven, plus or minus two: Some limits to our capacity for processing information. Psychol. Rev. 63, 81–97 (1956)
Murphy, P.M., Aha, D.W.: UCI repository of machine learning databases (1994)
Platt, J.: Advances in Large Margin Classifiers, chapter Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. MIT Press, Cambridge (1999)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Machine Learning. Morgan Kaufmann, San Mateo (1993)
Quinlan, R.J.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)
Rousseeuw, P.J.: Least median of squares regression. Journal of the American Statistical Association 79, 871–880 (1984)
Rüping, S.: A simple method for estimating conditional probabilities in SVMs. In: Abecker, A., Bickel, S., Brefeld, U., Drost, I., Henze, N., Herden, O., Minor, M., Scheffer, T., Stojanovic, L., Weibelzahl, S. (eds.) LWA 2004 - Lernen - Wissensentdeckung - Adaptivität. Humboldt-Universität Berlin (2004)
Smyth, P., Gray, A., Fayyad, U.M.: Retrofitting decision tree classifiers using kernel density estimation. In: International Conference on Machine Learning, pp. 506–514 (1995)
Sommer, E.: Theory Restructering: A Perspective on Design & Maintenance of Knowledge Based Systems. PhD thesis, University of Dortmund (1996)
Todorovski, L., Dzeroski, S.: Combining multiple models with meta decision trees. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 54–64. Springer, Heidelberg (2000)
Todorovski, L., Dzeroski, S.: Experiments in meta-level learning with ILP. In: Żytkow, J.M., Rauch, J. (eds.) PKDD 1999. LNCS (LNAI), vol. 1704, pp. 98–106. Springer, Heidelberg (1999)
Tumer, K., Ghosh, J.: Order statistics combiners for neural classifiers. In: Proceedings of the World Congress on Neural Networks (1995)
Vapnik, V.: Statistical Learning Theory. Wiley, Chichester (1998)
Wolpert, D.: Stacked generalizations. Neural Networks 5, 241–259 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rüping, S. (2005). Learning with Local Models. In: Morik, K., Boulicaut, JF., Siebes, A. (eds) Local Pattern Detection. Lecture Notes in Computer Science(), vol 3539. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11504245_10
Download citation
DOI: https://doi.org/10.1007/11504245_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26543-6
Online ISBN: 978-3-540-31894-1
eBook Packages: Computer ScienceComputer Science (R0)