Abstract
Random forest models [1] consist of an ensemble of randomized decision trees. It is one of the best performing classification models. With this idea in mind, in this section we introduced a random split operator based on a Bayesian approach for building a random forest. The convenience of this split method for constructing ensembles of classification trees is justified with an error bias-variance decomposition analysis. This new split operator does not clearly depend on a parameter K as its random forest’s counterpart, and performs better with a lower number of trees.
This work has been jointly supported by Spanish Ministry of Education and Science under project TIN2007-67418-C03-03, by European Regional Development Fund (FEDER), by the Spanish research programme Consolider Ingenio 2010: MIPRCV (CSD2007-00018), and by the FPU scholarship programme (AP2004-4678).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Mingers, J.: An empirical comparison of selection measures for decision-tree induction. Mach. Learn. 3(4), 319–342 (1989)
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)
Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40(2), 139–157 (2000)
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth International Group, Belmont (1984)
Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Machine Learning 63(1), 3–42 (2006)
Breiman, L.: Arcing classifiers. The Annals of Statistics 26(3), 801–824 (1998)
Kohavi, R., Wolpert, D.: Bias plus variance decomposition for zero-one loss functions. In: ICML, pp. 275–283 (1996)
Buntine, W.: Learning classification trees. Statistics and Computing (2), 63–73 (1992)
Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)
Abellán, J., Masegosa, A.R.: Combining decision trees based on imprecise probabilities and uncertainty measures. In: Mellouli, K. (ed.) ECSQARU 2007. LNCS, vol. 4724, pp. 512–523. Springer, Heidelberg (2007)
Andrés Cano, A.M., Moral, S.: A bayesian approach to estimate probabilities in classification trees. In: Jaeger, M., Nielsen, T.D. (eds.) Proceedings of the Fourth European Workshop on Probabilistic Graphical Models, pp. 49–56 (2008)
Consortium, E.: Elvira: An environment for probabilistic graphical models. In: Gámez, J., Salmerón, A. (eds.) Proceedings of the 1st European Workshop on Probabilistic Graphical Models, pp. 222–230 (2002)
Witten, I.H., Frank, E.: Data mining: practical machine learning tools and techniques with Java implementations. Morgan Kaufmann Publishers Inc., San Francisco (2000)
Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proc. of 13th Int. Joint Conf. on AI (1993)
Webb, G.I., Conilione, P.: Estimating bias and variance from data (2006)
Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cano, A., Masegosa, A.R., Moral, S. (2009). A Bayesian Random Split to Build Ensembles of Classification Trees. In: Sossai, C., Chemello, G. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2009. Lecture Notes in Computer Science(), vol 5590. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02906-6_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-02906-6_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02905-9
Online ISBN: 978-3-642-02906-6
eBook Packages: Computer ScienceComputer Science (R0)