Abstract
In this paper, we consider an algorithm that generates an ensemble of decision rules. A single rule is treated as a specific subsidiary, base classifier in the ensemble that indicates only one of the decision classes. Experimental results have shown that the ensemble of decision rules is as efficient as other machine learning methods. In this paper we concentrate on a common problem appearing in real-life data that is a presence of missing attributes values. To deal with this problem, we experimented with different approaches inspired by rough set approach to knowledge discovery. Results of those experiments are presented and discussed in the paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Michalski, R.S.: A Theory and Methodology of Inductive Learning. In: Michalski, R.S., Carbonell, J.G., Mitchell, T.M. (eds.) Machine Learning: An Artificial Intelligence Approach, Palo Alto, pp. 83–129. Tioga Publishing (1983)
Błaszczyński, J., Dembczyński, K., Kotłowski, W., Słowiński, R., Szeląg, M.: Ensemble of Decision Rules. Research Report RA-011/06, Poznań University of Technology (2006)
Boros, E., Hammer, P.L., Ibaraki, T., Kogan, A., Mayoraz, E., Muchnik, I.: An Implementation of Logical Analysis of Data. IEEE Trans. on Knowledge and Data Engineering 12, 292–306 (2000)
Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth (1984)
Clark, P., Nibbet, T.: The CN2 induction algorithm. Machine Learning 3, 261–283 (1989)
Cohen, W.W., Singer, Y.: A simple, fast, and effective rule learner. In: Proc. of 16th National Conference on Artificial Intelligence, pp. 335–342 (1999)
Düntsch, I., Gediga, G., Orłowska, E.: Relational attribute systems. International Journal of Human-Computer Studies 55, 293–309 (2001)
Friedman, J.H., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Stanford University Technical Report (last access: 1.05.2006) (August 1998), http://www-stat.stanford.edu/~jhf/
Friedman, J.H., Hastie, T., Tibshirani, R.: Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Heidelberg (2003)
Friedman, J.H.: Recent advances in predictive (machine) learning. Stanford University Technical Report (last access: 1.05.2006) (November 2003), http://www-stat.stanford.edu/~jhf/
Friedman, J.H., Popescu, B.E.: Gradient directed regularization. Stanford University Technical Report (last access: 1.05.2006) (February 2004), http://www-stat.stanford.edu/~jhf/
Friedman, J.H., Popescu, B.E.: Predictive Learning via Rule Ensembles. Stanford University Technical Report (last access: 1.05.2006) (February 2005), http://www-stat.stanford.edu/~jhf/
Greco, S., Matarazzo, B., Słowiński, R.: Dealing with missing data in rough set analysis of multi-attribute and multi-criteria decision problems. In: Zanakis, S.H., Doukidis, G., Zopounidis, C. (eds.) Decision Making: Recent Developments and Worldwide Applications, pp. 295–316. Kluwer Academic Publishers, Dordrecht (2000)
Grzymala-Busse, J.W.: LERS — A system for learning from examples based on rough sets. In: Słowiński, R. (ed.) Intelligent Decision Support, Handbook of Applications and Advances of the Rough Sets Theory, pp. 3–18. Kluwer Academic Publishers, Dordrecht (1992)
Grzymala-Busse, J.W., Hu, M.: A Comaprison of Several Approaches in Missing Attribute Values in Data Mining. In: Ziarko, W., Yao, Y. (eds.) RSCTC 2000. LNCS (LNAI), vol. 2005, pp. 378–385. Springer, Heidelberg (2001)
Grzymala-Busse, J.W.: Incomplete Data and Generalization of Indiscernibility Relation, Definability, and Approximation. In: Wang, L., Jin, Y. (eds.) FSKD 2005. LNCS (LNAI), vol. 3614, pp. 244–253. Springer, Heidelberg (2005)
Kryszkiewicz, M.: Rough Set approach to incomplete information systems. Information Sciences 112, 39–49 (1998)
Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: (UCI) Repository of machine learning databases, Dept. of Information and Computer Sciences, University of California, Irvine (1998) (last access: 01.05.2006), http://www.ics.uci.edu/~mlearn/MLRepository.html
Pawlak, Z.: Rough Sets. Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.E.: Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics 26(5), 1651–1686 (1998)
Skowron, A.: Extracting laws from decision tables - a rough set approach. Computational Intelligence 11, 371–388 (1995)
Stefanowki, J.: On rough set based approach to induction of decision rules. In: Polkowski, L., Skowron, A. (eds.) Rough Sets in Knowledge Discovery, pp. 500–529. Physica-Verlag, Heidelberg (1998)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Błaszczyński, J., Dembczyński, K., Kotłowski, W., Słowiński, R., Szeląg, M. (2006). Ensembles of Decision Rules for Solving Binary Classification Problems in the Presence of Missing Values. In: Greco, S., et al. Rough Sets and Current Trends in Computing. RSCTC 2006. Lecture Notes in Computer Science(), vol 4259. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11908029_25
Download citation
DOI: https://doi.org/10.1007/11908029_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-47693-1
Online ISBN: 978-3-540-49842-1
eBook Packages: Computer ScienceComputer Science (R0)