Definition
Regression trees are supervised learning methods that address multiple regression problems. They provide a tree-based approximation \(\hat{f}\), of an unknown regression function \(Y \,=\,f(\mathbf{x}) + \epsilon\) with Y ∈ ℜ and ε ≈ N(0, σ 2), based on a given sample of data \(D = \{\langle {x}_{i,1},\ldots ,{x}_{i,p},{y}_{i}\rangle {\}}_{i=1}^{n}\). The obtained models consist of a hierarchy of logical tests on the values of any of the p predictor variables. The terminal nodes of these trees, known as the leaves, contain the numerical predictions of the model for the target variable Y .
Motivation and Background
This is a preview of subscription content, log in via an institution to check access.
Recommended Reading
Breiman, L., Friedman, J., Olshen, R., & Stone, C. (1984). Classification and regression trees. Statistics/probability series. Wadsworth & Brooks/Cole Advanced Books & Software.
Breiman, L., & Meisel, W. S. (1976). General estimates of the intrinsic variability of data in nonlinear regression models. Journal of the American Statistical Association, 71, 301–307.
Buja, A., & Lee, Y.-S. (2001). Data mining criteria for tree-based regression and classification. In Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining (pp. 27–36). San Francisco, California, USA.
Friedman, J. H. (1979). A tree-structured approach to nonparametric multiple regression. In T. Gasser & M. Rosenblatt (Eds.), Smoothing techniques for curve estimation. Lecture notes in mathematics (Vol. 757, pp. 5–22). Berlin: Springer.
Gama, J. (2004). Functional trees. Machine Learning, 55(3), 219–250.
Li, K. C., Lue, H., & Chen, C. (2000). Interactive tree-structured regression via principal Hessians direction. Journal of the American Statistical Association, 95, 547–560.
Loh, W. (2002). Regression trees with unbiased variable selection and interaction detection. Statistica Sinica, 12, 361–386.
Lubinsky, D. (1995). Tree structured interpretable regression. In Proceedings of the workshop on AI & statistics.
Malerba, D., Esposito, F., Ceci, M., & Appice, A. (2004). Top-down induction of model trees with regression and splitting nodes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(5), 612–625.
Morgan, J. N., & Sonquist, J. A. (1963). Problems in the analysis of survey data, and a proposal. Journal of American Statistical Association, 58(302), 415–434.
Robnik-Sikonja, M., & Kononenko, I. (1996). Context-sensitive attribute estimation in regression. In Proceedings of the ICML-96 workshop on learning in context-sensitive domains. Brighton, UK.
Robnik-Sikonja, M., & Kononenko, I. (1998). Pruning regression trees with MDL. In Proceedings of ECAI-98. Brighton, UK.
Torgo, L. (1998). Error estimates for pruning regression trees. In C. Nedellec & C. Rouveirol (Eds.), Proceedings of the tenth European conference on machine learning. LNAI (Vol. 1398). London, UK: Springer-Verlag.
Torgo, L. (1999). Inductive learning of tree-based regression models. PhD thesis, Department of Computer Science, Faculty of Sciences, University of Porto.
Torgo, L., & Ribeiro, R. (2003). Predicting outliers. In N. Lavrac, D. Gamberger, L. Todorovski, & H. Blockeel (Eds.), Proceedings of principles of data mining and knowledge discovery (PKDD’03). LNAI (Vol. 2838, pp. 447–458). Berlin/Heidelberg: Springer-Verlag.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this entry
Cite this entry
Torgo, L. (2011). Regression Trees. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-30164-8_711
Download citation
DOI: https://doi.org/10.1007/978-0-387-30164-8_711
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-30768-8
Online ISBN: 978-0-387-30164-8
eBook Packages: Computer ScienceReference Module Computer Science and Engineering