Skip to main content
SpringerLink
Log in

Pareto-optimality of oblique decision trees from evolutionary algorithms

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

This paper investigates the performance of evolutionary algorithms in the optimization aspects of oblique decision tree construction and describes their performance with respect to classification accuracy, tree size, and Pareto-optimality of their solution sets. The performance of the evolutionary algorithms is analyzed and compared to the performance of exhaustive (traditional) decision tree classifiers on several benchmark datasets. The results show that the classification accuracy and tree sizes generated by the evolutionary algorithms are comparable with the results generated by traditional methods in all the sample datasets and in the large datasets, the multiobjective evolutionary algorithms generate better Pareto-optimal sets than the sets generated by the exhaustive methods. The results also show that a classifier, whether exhaustive or evolutionary, that generates the most accurate trees does not necessarily generate the shortest trees or the best Pareto-optimal sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Quinlan R.: Induction of decision trees. Mach. Learn. 1, 81–106 (1986)

    Google Scholar 

  2. Loh W., Vanichsetakul N.: Tree structured classification via generalized discriminant analysis. J. Am. Stat. Assoc. 83, 715–725 (1988)

    Article  Google Scholar 

  3. Loh W., Shih Y.: Split selection methods for classification trees. Stat. Sin. 7, 815–840 (1997)

    Google Scholar 

  4. Breiman L., Friedman J., Olshen R., Stone C.: Classification and Regression Trees. Wadsworth, Belmont (1984)

    Google Scholar 

  5. Heath, D., Kasif, S., Salzberg, S.: Induction of oblique decision trees. Proceedings of IJCAI, Morgan Kaufman, Chambery, pp. 1002–1007 (1993)

  6. Murthy S., Kasif S., Salzberg S.: A system for induction of oblique decision trees. J. Artif. Intel. Res. 2, 1–32 (1994)

    Google Scholar 

  7. Vadera S.: Safer oblique trees without costs. Expert Syst. 22, 206–221 (2005)

    Article  Google Scholar 

  8. Street N.: Oblique multi-category decision trees using nonlinear programming. J. Comput. 17, 25–31 (2005)

    Google Scholar 

  9. Moret B.: Decision trees and diagrams. Comput. Surv. 14(4), 593–623 (1982)

    Article  Google Scholar 

  10. Fayyad, U., Irani, K.: What should be minimized in a decision tree? Proceedings of the National Conference on Artificial Intelligence, Boston, pp. 749–754 (1990)

  11. Kononenko I., Bratko I.: Information based evaluation criterion for classifier’s performance. Mach. Learn. 6, 67–80 (1991)

    Google Scholar 

  12. Gelfand S., Ravishankar C., Delp E.: An iterative growing and pruning algorithm for classification tree design. IEEE Trans. Pattern Anal. Mach. Intel. 13, 163–174 (1991)

    Article  Google Scholar 

  13. Wang Q., Suen C.: Analysis and design of a decision tree based on entropy reduction and its application to large character set recognition. IEEE Trans. Pattern Anal. Mach. Intel. 6, 406–417 (1984)

    Article  Google Scholar 

  14. Murthy S.: Automatic construction of decision trees from data: a multi-disciplinary survey. Data Min. Knowl. Discov. 2, 345–398 (1998)

    Article  Google Scholar 

  15. Murthy, S., Salzberg, S.: Look-ahead and pathology in decision tree induction. Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, pp. 1025–1033 (1995)

  16. Koza, J.: Hierarchical genetic algorithms operating in populations of computer programs. Proceedings of the 11th International Joint Conference on Artificial Intelligence, pp. 768–774 (1989)

  17. Lutsko J.F., Kuijpers B.: Simulated annealing in the construction of near-optimal decision trees. In: Chessman, P., Oldford, R.W. (eds) Selecting Models from Data: AI and Stat. IV, pp. 453–462. Springer, Berlin (1994)

    Chapter  Google Scholar 

  18. Folino, G., Pizzuti, C., Spezzano, G., Spezzano, O.: Genetic programming and simulated annealing: a hybrid method to evolve decision trees. Proceedings of the European Conference on Genetic Programming, Edinburgh, pp. 294–303 (2000)

  19. Kwok, S., Carter, C.: Multiple decision trees. Proceedings of the 4th Annual Conference on Uncertainty in Artificial Intelligence, Cambridge, pp. 327–325 (1990)

  20. Shlien S.: Nonparametric classification using matched binary decision trees. Pattern Recognit. Lett. 13, 83–88 (1992)

    Article  Google Scholar 

  21. Buntine W.: Learning classification trees. Stat. Comput. 2, 63–73 (1992)

    Article  Google Scholar 

  22. Heath, D., Kasif, S., Salzberg, S.: k-DT: A multi-tree learning method. Proceedings of the 2nd International Workshop on Multi-strategy Learning, West Virginia, pp. 138–149 (1993)

  23. Cantu-Paz E., Kamath C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Trans. Evol. Comput. 7, 54–68 (2003)

    Article  Google Scholar 

  24. Harik G., Cantu-Paz E., Goldberg D., Miller B.: The gamblers ruin problem, genetic algorithms, and the sizing of populations. Evol. Comput. 7, 231–253 (1999)

    Article  Google Scholar 

  25. Miller B., Goldberg D.: Genetic algorithm selections schemes, and the varying effects of noise. Evol. Comput. 4, 113–131 (1996)

    Article  Google Scholar 

  26. Sörensen K., Janssens G.K.: Data mining with genetic algorithms on binary trees. Eur. J. Oper. Res. 151, 253–264 (2003)

    Article  Google Scholar 

  27. Papagelis, A., Kalles, D.: Breeding decision trees using evolutionary techniques. Proceedings of the 8th International Conference on Machine Learning, Williamstown, pp. 393–400 (2001)

  28. Bot, M., Langdon, W.: Application of genetic programming to induction of linear classification trees. Proceedings of European Conference on Genetic Programming, pp. 247–258. Springer, Edinburgh (2000)

  29. Kim, D.: Structural risk minimization on decision trees using evolutionary multiobjective optimization. Springer Lecture Notes on Computer Science 3003, pp. 338–348 (2004)

  30. Bryson N.: Evaluation of decision trees: a multicriteria approach. Comp. Oper. Res. 31, 1933–1945 (2004)

    Article  Google Scholar 

  31. Zhao H.: A multiobjective genetic programming approach to developing Pareto-optimal decision trees. Decis. Support Syst. 43, 809–826 (2007)

    Article  Google Scholar 

  32. Doumpos M., Chatzi E., Zopounidis C.: An experimental evaluation of some classification methods. J. Glob. Optim. 36(2), 33–50 (2006)

    Article  Google Scholar 

  33. Chinchuluun A., Pardalos P.M.: A survey of recent developments in multiobjective optimization. Ann. Oper. Res. 154(1), 29–50 (2007)

    Article  Google Scholar 

  34. Pappalardo M.: Multiobjective optimization: a brief overview. In: Chichuluun, A., Pardalos, P.M., Migdalas, A., Pitsoulis, L. (eds) Pareto Optimality, Game Theory and Equilibria, pp. 517–528. Springer, Berlin (2008)

    Chapter  Google Scholar 

  35. Zitzler E., Thiele L.: Comparison of multi-objective evolutionary algorithms: empirical results. Evol. Comput. 8, 125–148 (2000)

    Article  Google Scholar 

  36. Zitzler E., Laumanns M., Thiele L.: SPEA2: improving the strength Pareto evolutionary algorithm for multiobjective optimization. In: Giannakoglou, K., Tsahalis, D., Periaux, J., Papailu, K., Fogarty, T. (eds) Evolutionary Methods for Design, Optimization, and Control, pp. 19–26. CIMNE, Barcelona (2002)

    Google Scholar 

  37. Deb K., Agrawal S., Pratap A., Meyarivan T.: A fast elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6, 182–197 (2002)

    Article  Google Scholar 

  38. Gil C., Márquez A., Baños R., Montoya M.G., Gómez J.: A hybrid method for solving multi-objective global optimization problems. J. Glob. Optim. 38(2), 265–281 (2007)

    Article  Google Scholar 

  39. Deb K.: Multiobjective Optimization Using Evolutionary Algorithms. Wiley, UK (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saint Louis University, Baguio City, Philippines

    Jose Maria Pangilinan

  2. IMOB, Hasselt University, Diepenbeek, Belgium

    Gerrit K. Janssens

Authors
  1. Jose Maria Pangilinan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerrit K. Janssens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerrit K. Janssens.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pangilinan, J.M., Janssens, G.K. Pareto-optimality of oblique decision trees from evolutionary algorithms. J Glob Optim 51, 301–311 (2011). https://doi.org/10.1007/s10898-010-9614-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-010-9614-9

Keywords

Search

Navigation