Abstract
In the paper, a new method of decision tree learning for cost-sensitive classification is presented. In contrast to the traditional greedy top-down inducer in the proposed approach optimal trees are searched in a global manner by using an evolutionary algorithm (EA). Specialized genetic operators are applied to modify both the tree structure and tests in non-terminal nodes. A suitably defined fitness function enables the algorithm to minimize the misclassification cost instead of the number of classification errors. The performance of the EA-based method is compared to three well-recognized algorithms on real-life problems with known and randomly generated cost-matrices. Obtained results show that the proposed approach is competitive both in terms of misclassification cost and compactness of the classifier at least for some datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abe, N., Zadrozny, B., Langford, J.: An iterative method for multi-class cost-sensitive learning. In: KDD’04, pp. 3–11. ACM Press, New York (2004)
Blake, C., Keogh, E., Merz, C.: UCI repository of machine learning databases, (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Bradford, J.P., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.E.: Pruning decision trees with misclassification costs. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 131–136. Springer, Heidelberg (1998)
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth Int. Group, Belmont (1984)
Cantu-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Transactions on Evolutionary Computation 7(1), 54–68 (2003)
Chai, B., et al.: Piecewise-linear classifiers using binary tree structure and genetic algorithm. Pattern Recognition 29(11), 1905–1917 (1996)
Domingos, P.: MetaCost: A general method for making classifiers cost-sensitive. In: Proc. of KDD’99, pp. 155–164. ACM Press, New York (1999)
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proc. of IJCAI’93, pp. 1022–1027 (1993)
Frank, E., et al.: Weka 3 - Data Mining with Open Source Machine Learning Software in Java. University of Waikato (2000), http://www.cs.waikato.ac.nz/~ml/weka
Knoll, U., Nakhaeizadeh, G., Tausend, B.: Cost-sensitive pruning of decision trees. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 383–386. Springer, Heidelberg (1994)
Koza, J.: Concept formation and decision tree induction using genetic programming paradigm. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 124–128. Springer, Heidelberg (1991)
Krȩtowski, M.: An evolutionary algorithm for oblique decision tree induction. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 432–437. Springer, Heidelberg (2004)
Krȩtowski, M., Grześ, M.: Global learning of decision trees by an evolutionary algorithm. In: Information Processing and Security Systems, pp. 401–410. Springer, Heidelberg (2005)
Krȩtowski, M., Grześ, M.: Evolutionary learning of linear trees with embedded feature selection. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 400–409. Springer, Heidelberg (2006)
Krȩtowski, M., Grześ, M.: Mixed decision trees: An evolutionary approach. In: Tjoa, A.M., Trujillo, J. (eds.) DaWaK 2006. LNCS, vol. 4081, pp. 260–269. Springer, Heidelberg (2006)
Kwedlo, W., Krȩtowski, M.: An evolutionary algorithm for cost-sensitive decision rule learning. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 288–299. Springer, Heidelberg (2001)
Ling, C., Yang, Q., Wang, J., Zhang, S.: Decision trees with minimal costs. In: Proc. of ICML’04, Article No. 69, ACM Press, New York (2004)
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Heidelberg (1996)
Margineantu, D.D., Dietterich, T.G.: Bootstrap methods for the cost-sensitive evaluation of classifiers. In: Proc. of ICML’2000, pp. 583–590. Morgan Kaufmann, San Francisco (2000)
Papagelis, A., Kalles, D.: Breeding decision trees using evolutionary techniques. In: Proc. of ICML’01, pp. 393–400. Morgan Kaufmann, San Francisco (2001)
Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Ting, K.M.: An instance-weighting method to induce cost-sensitive trees. IEEE Transactions on Knowledge and Data Engineering 14(3), 659–665 (2002)
Turney, P.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research 2, 369–409 (1995)
Turney, P.: Types of cost in inductive concept learning. In: Proc. of ICML’2000 Workshop on Cost-Sensitive Learning, Stanford, CA (2000)
Zadrozny, B., Langford, J., Abe, N.: Cost-sensitive learning by cost-proportionate example weighting concept learning. In: Proc. of ICDM’03, IEEE Computer Society Press, Los Alamitos (2003)
Zhang, S., Qin, Z., Ling, C., Sheng, S.: Missing is usefull: Missing values in cost-sensitive decision trees. IEEE Transactions on Knowledge and Data Engineering 17(12), 1689–1693 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Krȩtowski, M., Grześ, M. (2007). Evolutionary Induction of Decision Trees for Misclassification Cost Minimization. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2007. Lecture Notes in Computer Science, vol 4431. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71618-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-71618-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71589-4
Online ISBN: 978-3-540-71618-1
eBook Packages: Computer ScienceComputer Science (R0)