Abstract
Many problems encountered in practice involve the prediction of a continuous attribute associated with an example. This problem, known as regression, requires that samples of past experience with known continuous answers are examined and generalized in a regression model to be used in predicting future examples. Regression algorithms deeply investigated in statistics, machine learning and data mining usually lack measures to give an indication of how “good” the predictions are. Tolerance regions, i.e., a range of possible predictive values, can provide a measure of reliability for every bare prediction. In this paper, we focus on tree-based prediction models, i.e., model trees, and resort to the inductive inference to output tolerance regions in addition to bare prediction. In particular, we consider model trees mined by SMOTI (Stepwise Model Tree Induction) that is a system for data-driven stepwise construction of model trees with regression and splitting nodes and we extend the definition of trees to build tolerance regions to be associated with each leaf. Experiments evaluate validity and quality of output tolerance regions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Breiman, L., Friedman, J., Olshen, R., Stone, J.: Classification and regression tree. Wadsworth & Brooks (1984)
Chaudhuri, P., Huang, M.C., Loh, W.Y., Yao, R.: Piecewise-polynomial regression trees. Statistica Sinica 4, 143–167 (1994)
Cowan, G.: Statistical Data Analysis. Oxford University Press, USA (1998)
Dobra, A., Gehrke, J.E.: SECRET: A scalable linear regression tree algorithm. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Edmonton, Alberta, Canada (2002)
Draper, N.R., Smith, H.: Applied regression analysis. John Wiley, Chichester (1982)
Fraser, D.A.S.: Non-parametric Methods in Statistics. Wiley, New York (1957)
Gammerman, A., Vovk, V., Vapnik, V.: Learning by transduction. In: 14th Conference on Uncertainty in Artificial Intelligence, pp. 148–155 (1998)
Hahn, G.J., Meeker, W.Q.: Statistical Intervals: A Guide for Practitioners. Series in Probability and Statistics. John Wiley & Sons, Chichester (1991)
Nouretdinov, V.V.I., Gammerman, A.: Transductive confidence machine is universal. In: Gavaldá, R., Jantke, K.P., Takimoto, E. (eds.) ALT 2003. LNCS (LNAI), vol. 2842, p. 283. Springer, Heidelberg (2003)
Karalic, A.: Linear regression in regression tree leaves. In: Proceedings of Int. School for Synthesis of Expert Knowledge, ISSEK 1992, pp. 151–163 (1992)
Loh, W.Y.: Regression trees with unbiased variable selection and interaction detection. Statistica Sinica 12, 361–386 (2002)
Malerba, D., Esposito, F., Ceci, M., Appice, A.: Top down induction of model trees with regression and splitting nodes. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(5), 612–625 (2004)
Melluish, T., Saunders, C., Nouretdinov, I., Vovk, V.: Comparing the bayes and typicalness frameworks. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, Springer, Heidelberg (2001)
Papadopoulos, H., Proedrou, K., Vovk, V., Gammerman, A.: Inductive confidence machines for regression. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 345–356. Springer, Heidelberg (2002)
Proedrou, K., Nouretdinov, I., Vovk, V., Gammerman, A.: Transductive confidence machines for pattern recognition (2001)
Quinlan, J.R.: Learning with continuous classes. In: Adams, Sterling (eds.) Proceedings of the 5th Australian Joint Conference on Artificial Intelligence, pp. 343–348. World Scientific, Singapore (1992)
Torgo, L.: Functional models for regression tree leaves. In: Fisher, D. (ed.) Proceedings of the Fourteenth International Conference on Machine Learning, ICML 1997, Nashville, Tennessee, pp. 385–393 (1997)
Torgo, L.: Inductive Learning of Tree-based Regression Models. PhD thesis, Department of Computer Science, University of Porto, Porto, Portugal (1999)
Wang, Y., Witten, I.H.: Inducing model trees for continuous classes. In: van Someren, M., Widmer, G. (eds.) ECML 1997. LNCS, vol. 1224, pp. 128–137. Springer, Heidelberg (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Appice, A., Ceci, M. (2006). Mining Tolerance Regions with Model Trees. In: Esposito, F., Raś, Z.W., Malerba, D., Semeraro, G. (eds) Foundations of Intelligent Systems. ISMIS 2006. Lecture Notes in Computer Science(), vol 4203. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11875604_63
Download citation
DOI: https://doi.org/10.1007/11875604_63
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45764-0
Online ISBN: 978-3-540-45766-4
eBook Packages: Computer ScienceComputer Science (R0)