Abstract
Random forest induction is a bagging method that randomly samples the feature set at each node in a decision tree. In propositional learning, the method has been shown to work well when lots of features are available. This certainly is the case in first order learning, especially when aggregate functions, combined with selection conditions on the set to be aggregated, are included in the feature space. In this paper, we introduce a random forest based approach to learning first order theories with aggregates. We experimentally validate and compare several variants: first order random forests without aggregates, with simple aggregates, and with complex aggregates in the feature set.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Berka, P.: Guide to the financial data set. In: Siebes, A., Berka, P. (eds.) The ECML/PKDD 2000 Discovery Challenge (2000)
Blockeel, H., Bruynooghe, M.: Aggregation versus selection bias, and relational neural networks. In: IJCAI-2003 Workshop on Learning Statistical Models from Relational Data, SRL 2003, Acapulco, Mexico, August 11 (2003)
Blockeel, H., De Raedt, L.: Lookahead and discretization in ILP. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS, vol. 1297, pp. 77–85. Springer, Heidelberg (1997)
Blockeel, H., De Raedt, L.: Top-down induction of first order logical decision trees. Artificial Intelligence 101(1-2), 285–297 (1998)
Blockeel, H., Dehaspe, L., Demoen, B., Janssens, G., Ramon, J., Vandecasteele, H.: Improving the efficiency of inductive logic programming through the use of query packs. Journal of Artificial Intelligence Research 16, 135–166 (2002)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L.: Out-of-bag estimation. ftp.stat.berkeley.edu/pub/users/breiman/OOBestimation.ps (1996)
Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)
De Raedt, L., Van Laer, W.: Inductive constraint logic. In: Zeugmann, T., Shinohara, T., Jantke, K.P. (eds.) ALT 1995. LNCS, vol. 997, pp. 80–94. Springer, Heidelberg (1995)
Dietterich, T.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)
Ďzeroski, S., Schulze-Kremer, S., Heidtke, K.R., Siems, K., Wettschereck, D., Blockeel, H.: Diterpene structure elucidation from 13C NMR spectra with inductive logic programming. Applied Artificial Intelligence 12(5), 363–384 (1998)
Emde, W., Wettschereck, D.: Relational instance based learning. In: Proceedings of the 1995 Workshop of the GI Special Interest Group on Machine Learning (1995)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Saitta, L. (ed.) Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156. Morgan Kaufmann, San Francisco (1996)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 993–1001 (1990)
Jensen, D., Neville, J., Hay, M.: Avoiding bias when aggregating relational data with degree disparity. In: Proceedings of the 20th International Conference on Machine Learning (2003)
Knobbe, A., Siebes, A., Marseille, B.: Involving aggregate functions in multirelational search. In: Proceedings of the 6th European Conference, Principles of Data Mining and Knowledge Discovery, August 2002, pp. 287–298. Springer, Heidelberg (2002)
Koller, D.: Probabilistic relational models. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 3–13. Springer, Heidelberg (1999)
Krogel, M.-A., Wrobel, S.: Transformation-based learning using multi-relational aggregation. In: Proceedings of the Eleventh International Conference on Inductive Logic Programming, pp. 142–155 (2001)
Lavrač, N., Ďzeroski, S.: Inductive Logic Programming: Techniques and Applications. Ellis Horwood (1994)
Muggleton, S. (ed.): Inductive Logic Programming. Academic Press, London (1992)
Muggleton, S.: Inverse entailment and Progol. New Generation Computing, Special issue on Inductive Logic Programming 13(3-4), 245–286 (1995)
Neville, J., Jensen, D., Friedland, L., Hay, M.: Learning relational probability trees. In: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2003)
Perlich, C., Provost, F.: Aggregation-based feature invention and relational concept classes. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 167–176. ACM Press, New York (2003)
Quinlan, J.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann series in Machine Learning. Morgan Kaufmann, San Francisco (1993)
Srinivasan, A., King, R., Bristol, D.: An assessment of ILP-assisted models for toxicology and the PTE-3 experiment. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 291–302. Springer, Heidelberg (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vens, C., Van Assche, A., Blockeel, H., Džeroski, S. (2004). First Order Random Forests with Complex Aggregates. In: Camacho, R., King, R., Srinivasan, A. (eds) Inductive Logic Programming. ILP 2004. Lecture Notes in Computer Science(), vol 3194. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30109-7_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-30109-7_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22941-4
Online ISBN: 978-3-540-30109-7
eBook Packages: Springer Book Archive