Skip to main content
SpringerLink
Log in
Book cover

Rough Sets and Intelligent Systems - Professor Zdzisław Pawlak in Memoriam pp 21–40Cite as

Comparison of Greedy Algorithms for Decision Tree Optimization

  • Chapter

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 43))

Abstract

This chapter is devoted to the study of 16 types of greedy algorithms for decision tree construction. The dynamic programming approach is used for construction of optimal decision trees. Optimization is performed relative to minimal values of average depth, depth, number of nodes, number of terminal nodes, and number of nonterminal nodes of decision trees. We compare average depth, depth, number of nodes, number of terminal nodes and number of nonterminal nodes of constructed trees with minimum values of the considered parameters obtained based on a dynamic programming approach. We report experiments performed on data sets from UCI ML Repository and randomly generated binary decision tables. As a result, for depth, average depth, and number of nodes we propose a number of good heuristics.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alkhalid, A., Chikalov, I., Moshkov, M.: On Algorithm for Building of Optimal α-Decision Trees. In: Szczuka, M., Kryszkiewicz, M., Ramanna, S., Jensen, R., Hu, Q. (eds.) RSCTC 2010. LNCS (LNAI), vol. 6086, pp. 438–445. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  2. Alkhalid, A., Chikalov, I., Moshkov, M.: A Tool for Study of Optimal Decision Trees. In: Yu, J., Greco, S., Lingras, P., Wang, G., Skowron, A. (eds.) RSKT 2010. LNCS, vol. 6401, pp. 353–360. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  3. Alkhalid, A., Chikalov, I., Moshkov, M.: Comparison of Greedy Algorithms for α-Decision Tree Construction. In: Yao, J., Ramanna, S., Wang, G., Suraj, Z. (eds.) RSKT 2011. LNCS (LNAI), vol. 6954, pp. 178–186. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  4. Alkhalid, A., Chikalov, I., Moshkov, M.: Decision tree construction using greedy algorithms and dynamic programming – comparative study. In: Szczuka, M., et al. (eds.) Concurrency, Specification and Programming. Proceedings of International Workshop CS&P 2011, Pultusk, Poland, pp. 1–9. Bialystok University of Technology, Poland (2010)

    Google Scholar 

  5. Alkhalid, A., Chikalov, I., Moshkov, M.: Comparison of greedy algorithms for decision tree construction. In: Proceedings of International Conference on Knowledge Discovery and Information Retrieval KDIR 2011, Paris, France (2011) (to appear)

    Google Scholar 

  6. Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine (2011), http://www.ics.uci.edu/ml (cited October 20, 2010)

    Google Scholar 

  7. Breitbart, Y., Reiter, A.: A branch-and-bound algorithm to obtain an optimal evaluation tree for monotonic boolean functions. Acta Inf. 4, 311–319 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  8. Boryczka, U., Kozak, J.: Ant Colony Decision Trees – A New Method for Constructing Decision Trees Based on Ant Colony Optimization. In: Pan, J.-S., Chen, S.-M., Nguyen, N.T. (eds.) ICCCI 2010, Part I. LNCS, vol. 6421, pp. 373–382. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  9. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth & Brooks (1984)

    Google Scholar 

  10. Chai, B.-B., Zhuang, X., Zhao, Y., Sklansky, J.: Binary linear decision tree with genetic algorithm. In: Proceedings of the International Conference on Pattern Recognition (ICPR 1996), vol. 7472, pp. 530–534. IEEE Computer Society Press, Los Alamitos (1996)

    Google Scholar 

  11. Chikalov, I., Hussain, S., Moshkov, M.: Relationships between Depth and Number of Misclassifications for Decision Trees. In: Kuznetsov, S.O., Ślęzak, D., Hepting, D.H., Mirkin, B.G. (eds.) RSFDGrC 2011. LNCS, vol. 6743, pp. 286–292. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  12. Chegis, I.A., Yablonskii, S.V.: Logical methods of electric circuit control. Trudy MIAN SSSR 51, 270–360 (1958) (in Russian)

    MATH  Google Scholar 

  13. Chikalov, I., Moshkov, M., Zelentsova, M.: On Optimization of Decision Trees. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets IV. LNCS, vol. 3700, pp. 18–36. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  14. Fayyad, U.M., Irani, K.B.: The attribute specification problem in decision tree generation. In: Proceedings of the 10th National Conference on Artificial Intelligence (AAAI 1992), San Jose, CA, pp. 104–110 (1992)

    Google Scholar 

  15. Garey, M.R.: Optimal binary identification procedures. SIAM Journal on Applied Mathematics 23, 173–186 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  16. Heath, D., Kasif, S., Salzberg, S.: Introduction to oblique decision trees. In: Ruzena, B. (ed.) The 13th International Joint Conference on Artificial Intelligence (IJCAI 1993), Chambéry, France, pp. 1002–1007 (1993)

    Google Scholar 

  17. Hyafil, L., Rivest, R.: Constructing optimal binary decision trees is NP-complete. Information Processing Letters 5, 15–17 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  18. Kononenko, I.: On biases in estimating multi-valued attributes. In: Mellish, C. (ed.) Proceedings of the 14th International Joint Conference on Artificial intelligence (IJCAI 1995), vol. 2, pp. 1034–1040 (1995)

    Google Scholar 

  19. Martin, J.K.: An exact probability metric for decision tree splitting and stopping. Machine Learning 28, 257–291 (1997)

    Article  Google Scholar 

  20. Mingers, J.: Expert systems — rule induction with statistical data. J. of the Operational Research Society 38, 39–47 (1987)

    Google Scholar 

  21. Moret, B.E., Thomason, R.C., Gonzalez, M.: The activity of a variable and its relation to decision trees. ACM Transactions on Programming Languages and Systems (TOPLAS) 2, 580–595 (1980)

    Article  Google Scholar 

  22. Martelli, A., Montanari, U.: Optimizing decision trees through heuristically guided search. Commun. ACM 21, 1025–1039 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  23. Moshkov, M.: Time Complexity of Decision Trees. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets III. LNCS, vol. 3400, pp. 244–459. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  24. Moshkov, M., Chikalov, I.: Consecutive optimization of decision trees concerning various complexity measures. Fundam. Inform. 61, 87–96 (2003)

    MathSciNet  Google Scholar 

  25. Pawlak, Z.: Rough Sets – Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)

    MATH  Google Scholar 

  26. Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1, 81–106 (1986)

    Google Scholar 

  27. Riddle, P., Segal, R., Etzioni, O.: Representation design and brute-force induction in a Boeing manufacturing domain. Applied Artificial Intelligence 8, 125–147 (1994)

    Article  Google Scholar 

  28. Schumacher, H., Sevcik, K.C.: The synthetic approach to decision table conversion. Commun. ACM 19, 343–351 (1976)

    Article  MATH  Google Scholar 

  29. Skowron, A., Rauszer, C.: The discernibility matrices and functions in information systems. In: Słowiński, R. (ed.) Intelligent Decision Support. Handbook of Applications and Advances of the Rough Set Theory, pp. 331–362. Kluwer Academic Publishers, Dordrecht (1992)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematical and Computer Sciences & Engineering Division, King Abdullah University of Science and Technology, Thuwal, 23955-6990, Saudi Arabia

    Abdulaziz Alkhalid, Igor Chikalov & Mikhail Moshkov

Authors
  1. Abdulaziz Alkhalid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Igor Chikalov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mikhail Moshkov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdulaziz Alkhalid .

Editor information

Editors and Affiliations

  1. Institute of Mathematics, Warsaw University, Banacha 2, Warsaw, 02097, Poland

    Andrzej Skowron

  2. Institute of Computer Science, University of Rzeszów, ul. Dekerta 2, Rzeszów, 35-030, Poland

    Zbigniew Suraj

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Alkhalid, A., Chikalov, I., Moshkov, M. (2013). Comparison of Greedy Algorithms for Decision Tree Optimization. In: Skowron, A., Suraj, Z. (eds) Rough Sets and Intelligent Systems - Professor Zdzisław Pawlak in Memoriam. Intelligent Systems Reference Library, vol 43. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30341-8_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-30341-8_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-30340-1

  • Online ISBN: 978-3-642-30341-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation