Skip to main content
SpringerLink
Log in

A Multi-objective Evolutionary Approach to Pareto Optimal Model Trees. A Preliminary Study

  • Conference paper
  • First Online:
Theory and Practice of Natural Computing (TPNC 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10071))

Included in the following conference series:

Abstract

Decision tree induction is inherently a multi-objective task. However, most of the conventional learning algorithms can only deal with a single-objective that may possibly aggregate multiple objectives. This paper proposes the multi-objective evolutionary approach to Pareto optimal model trees. We developed a set of non-dominated model trees for a Global Model Tree framework using efficient sort and specialized selection. Performed study covers variants with two and three objectives that relate to the tree error and the tree comprehensibility. Pareto front generated by the GMT system allows the decision maker to select desired output model according to his preferences on the conflicting objectives. Experimental evaluation of the proposed approach is performed on three real-life datasets and is confronted with competitive model tree inducers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Barros, R.C., Ruiz, D.D., Basgalupp, M.P.: Evolutionary model trees for handling continuous classes in machine learning. Inf. Sci. 181(5), 954–971 (2011)

    Article  Google Scholar 

  2. Barros, R.C., Basgalupp, M.P., de Carvalho, A.C.P.L.F., Freitas, A.A.: A survey of evolutionary algorithms for decision-tree induction. IEEE Trans. Syst. Man Cybern. Part C 42(3), 291–312 (2012)

    Article  Google Scholar 

  3. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth and Brooks, Monterey (1984)

    MATH  Google Scholar 

  4. Czajkowski, M., Czerwonka, M., Kretowski, M.: Cost-sensitive global model trees applied to loan charge-off forecasting. Decis. Support Syst. 74, 57–66 (2015)

    Article  MATH  Google Scholar 

  5. Czajkowski, M., Kretowski, M.: Evolutionary induction of global model trees with specialized operators and memetic extensions. Inf. Sci. 288, 153–173 (2014)

    Article  Google Scholar 

  6. Czajkowski, M., Kretowski, M.: The role of decision tree representation in regression problems - an evolutionary perspective. Appl. Soft Comput. 48, 458–475 (2016)

    Article  Google Scholar 

  7. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comp. 6(2), 182–197 (2002)

    Article  Google Scholar 

  8. Esposito, F., Malerba, D., Semeraro, G.: A comparative analysis of methods for pruning decision trees. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 476–491 (1997)

    Article  Google Scholar 

  9. Fan, G., Gray, B.J.: Regression tree analysis using TARGET. J. Comput. Graph. Stat. 14(1), 206–218 (2005)

    Article  MathSciNet  Google Scholar 

  10. Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.): Advances in Knowledge Discovery and Data Mining. American Association for Artificial Intelligence, Menlo Park (1996)

    Google Scholar 

  11. Fortin, F.A., Parizeau, M.: Revisiting the nsga-ii crowding-distance computation. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 623–630. GECCO 2013. ACM, New York, NY, USA (2013)

    Google Scholar 

  12. Friedrich, T., Kroeger, T., Neumann, F.: Weighted preferences in evolutionary multi-objective optimization. Int. J. Mach. Learn. Cybern. 4(2), 139–148 (2013)

    Article  Google Scholar 

  13. Guzman, F.M.O., Valenzuela, O., Prieto, B., Saéz-Lara, M.J., Torres, C., Pomares, H., et al.: Comparing different machine learning and mathematical regression models to evaluate multiple sequence alignments. Neurocomputing 164, 123–136 (2015)

    Article  Google Scholar 

  14. Hazan, A., Ramirez, R., Maestre, E., Perez, A., Pertusa, A.: Modelling expressive performance: a regression tree approach based on strongly typed genetic programming. In: Rothlauf, F., et al. (eds.) EvoWorkshops 2006. LNCS, vol. 3907, pp. 676–687. Springer, Heidelberg (2006). doi:10.1007/11732242_64

    Chapter  Google Scholar 

  15. Hiwa, S., Nishioka, M., Hiroyasu, T., Miki, M.: Novel search scheme for multi-objective evolutionary algorithms to obtain well-approximated and widely spread pareto solutions. Swarm Evol. Comput. 22, 30–46 (2015). (Complete)

    Article  Google Scholar 

  16. Ishibuchi, H., Murata, T.: A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Syst. Man Cybern. Part C 28(3), 392–403 (1998). (Applications and Reviews)

    Article  Google Scholar 

  17. Jin, Y., Sendhoff, B.: Pareto-based multiobjective machine learning: An overview and case studies. IEEE Trans. Syst. Man Cybern. Part C 38(3), 397–415 (2008)

    Article  Google Scholar 

  18. Jurczuk, K., Czajkowski, M., Kretowski, M.: Evolutionary induction of a decision tree for large scale data. a GOU-based approach. Soft Comput. doi:10.1007/s00500-016-2280-1 (in press, 2016)

  19. Kim, D.E.: Structural risk minimization on decision trees using an evolutionary multiobjective optimization. In: Keijzer, M., O’Reilly, U.-M., Lucas, S., Costa, E., Soule, T. (eds.) EuroGP 2004. LNCS, vol. 3003, pp. 338–348. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24650-3_32

    Chapter  Google Scholar 

  20. Kotsiantis, S.B.: Decision trees: a recent overview. Artif. Intell. Rev. 39(4), 261–283 (2013)

    Article  Google Scholar 

  21. Loh, W.Y.: Fifty years of classification and regression trees. Int. Stat. Rev. 82(3), 329–348 (2014)

    Article  MathSciNet  Google Scholar 

  22. Louis, T.: Regression data sets (2016). http://www.dcc.fc.up.pt/ltorgo/Regression/DataSets.html

  23. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 2nd edn. Springer-Verlag, New York (1994)

    Book  MATH  Google Scholar 

  24. Pangilinan, J., Janssens, G.: Pareto-optimality of oblique decision trees from evolutionary algorithms. J. Global Optim. 51(2), 301–311 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  25. Pappalardo, M.: Multiobjective optimization: a brief overview. In: Chinchuluun, A., Pardalos, P.M., Migdalas, A., Pitsoulis, L. (eds.) Pareto Optimality, Game Theory And Equilibria. Springer Optimization and Its Applications, vol. 17, pp. 517–528. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  26. Potgieter, G., Engelbrecht, A.P.: Evolving model trees for mining data sets with continuous-valued classes. Expert Syst. Appl. 35(4), 1513–1532 (2008)

    Article  Google Scholar 

  27. Quinlan, J.R.: Learning with continuous classes. Proc. Aust. Joint Conf. Artif. Intell. 92, 343–348 (1992). World Scientific, Singapore

    Google Scholar 

  28. Rokach, L., Maimon, O.: Top-down induction of decision trees classifiers - a survey. Trans. Sys. Man Cyber Part C 35(4), 476–487 (2005)

    Article  Google Scholar 

  29. Rokach, L., Maimon, O.: Data Mining with Decision Trees: Theroy and Applications. World Scientific Publishing Co. Inc., River Edge (2008)

    MATH  Google Scholar 

  30. Schwarz, G.: Estimating the dimension of a model. Ann. Statist. 6(2), 461–464 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  31. Zhang, X., Tian, Y., Cheng, R., Jin, Y.: An efficient approach to nondominated sorting for evolutionary multiobjective optimization. IEEE Trans. Evol. Comput. 19(2), 201–213 (2015)

    Article  Google Scholar 

  32. Zhao, H.: A multi-objective genetic programming approach to developing pareto optimal decision trees. Decis. Support Syst. 43(3), 809–826 (2007)

    Article  Google Scholar 

  33. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: A comparative case study and the strength pareto approach. IEEE Trans. Evol. Comp. 3(4), 257–271 (1999)

    Article  Google Scholar 

Download references

Acknowledgments

This project was funded by the Polish National Science Center and allocated on the basis of decision 2013/09/N/ST6/04083. The second author was supported by the grant S/WI/2/13 from Bialystok University of Technology founded by Ministry of Science and Higher Education.

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Bialystok University of Technology, Wiejska 45a, 15-351, Bialystok, Poland

    Marcin Czajkowski & Marek Kretowski

Authors
  1. Marcin Czajkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marek Kretowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcin Czajkowski .

Editor information

Editors and Affiliations

  1. Rovira i Virgili University, Tarragona, Spain

    Carlos Martín-Vide

  2. Tohoku University, Sendai, Japan

    Takaaki Mizuki

  3. University of Extremadura, Cáceres, Spain

    Miguel A. Vega-Rodríguez

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Czajkowski, M., Kretowski, M. (2016). A Multi-objective Evolutionary Approach to Pareto Optimal Model Trees. A Preliminary Study. In: Martín-Vide, C., Mizuki, T., Vega-Rodríguez, M. (eds) Theory and Practice of Natural Computing. TPNC 2016. Lecture Notes in Computer Science(), vol 10071. Springer, Cham. https://doi.org/10.1007/978-3-319-49001-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49001-4_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49000-7

  • Online ISBN: 978-3-319-49001-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation