Abstract
Decision tree induction is inherently a multi-objective task. However, most of the conventional learning algorithms can only deal with a single-objective that may possibly aggregate multiple objectives. This paper proposes the multi-objective evolutionary approach to Pareto optimal model trees. We developed a set of non-dominated model trees for a Global Model Tree framework using efficient sort and specialized selection. Performed study covers variants with two and three objectives that relate to the tree error and the tree comprehensibility. Pareto front generated by the GMT system allows the decision maker to select desired output model according to his preferences on the conflicting objectives. Experimental evaluation of the proposed approach is performed on three real-life datasets and is confronted with competitive model tree inducers.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barros, R.C., Ruiz, D.D., Basgalupp, M.P.: Evolutionary model trees for handling continuous classes in machine learning. Inf. Sci. 181(5), 954–971 (2011)
Barros, R.C., Basgalupp, M.P., de Carvalho, A.C.P.L.F., Freitas, A.A.: A survey of evolutionary algorithms for decision-tree induction. IEEE Trans. Syst. Man Cybern. Part C 42(3), 291–312 (2012)
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth and Brooks, Monterey (1984)
Czajkowski, M., Czerwonka, M., Kretowski, M.: Cost-sensitive global model trees applied to loan charge-off forecasting. Decis. Support Syst. 74, 57–66 (2015)
Czajkowski, M., Kretowski, M.: Evolutionary induction of global model trees with specialized operators and memetic extensions. Inf. Sci. 288, 153–173 (2014)
Czajkowski, M., Kretowski, M.: The role of decision tree representation in regression problems - an evolutionary perspective. Appl. Soft Comput. 48, 458–475 (2016)
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comp. 6(2), 182–197 (2002)
Esposito, F., Malerba, D., Semeraro, G.: A comparative analysis of methods for pruning decision trees. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 476–491 (1997)
Fan, G., Gray, B.J.: Regression tree analysis using TARGET. J. Comput. Graph. Stat. 14(1), 206–218 (2005)
Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.): Advances in Knowledge Discovery and Data Mining. American Association for Artificial Intelligence, Menlo Park (1996)
Fortin, F.A., Parizeau, M.: Revisiting the nsga-ii crowding-distance computation. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 623–630. GECCO 2013. ACM, New York, NY, USA (2013)
Friedrich, T., Kroeger, T., Neumann, F.: Weighted preferences in evolutionary multi-objective optimization. Int. J. Mach. Learn. Cybern. 4(2), 139–148 (2013)
Guzman, F.M.O., Valenzuela, O., Prieto, B., Saéz-Lara, M.J., Torres, C., Pomares, H., et al.: Comparing different machine learning and mathematical regression models to evaluate multiple sequence alignments. Neurocomputing 164, 123–136 (2015)
Hazan, A., Ramirez, R., Maestre, E., Perez, A., Pertusa, A.: Modelling expressive performance: a regression tree approach based on strongly typed genetic programming. In: Rothlauf, F., et al. (eds.) EvoWorkshops 2006. LNCS, vol. 3907, pp. 676–687. Springer, Heidelberg (2006). doi:10.1007/11732242_64
Hiwa, S., Nishioka, M., Hiroyasu, T., Miki, M.: Novel search scheme for multi-objective evolutionary algorithms to obtain well-approximated and widely spread pareto solutions. Swarm Evol. Comput. 22, 30–46 (2015). (Complete)
Ishibuchi, H., Murata, T.: A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Syst. Man Cybern. Part C 28(3), 392–403 (1998). (Applications and Reviews)
Jin, Y., Sendhoff, B.: Pareto-based multiobjective machine learning: An overview and case studies. IEEE Trans. Syst. Man Cybern. Part C 38(3), 397–415 (2008)
Jurczuk, K., Czajkowski, M., Kretowski, M.: Evolutionary induction of a decision tree for large scale data. a GOU-based approach. Soft Comput. doi:10.1007/s00500-016-2280-1 (in press, 2016)
Kim, D.E.: Structural risk minimization on decision trees using an evolutionary multiobjective optimization. In: Keijzer, M., O’Reilly, U.-M., Lucas, S., Costa, E., Soule, T. (eds.) EuroGP 2004. LNCS, vol. 3003, pp. 338–348. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24650-3_32
Kotsiantis, S.B.: Decision trees: a recent overview. Artif. Intell. Rev. 39(4), 261–283 (2013)
Loh, W.Y.: Fifty years of classification and regression trees. Int. Stat. Rev. 82(3), 329–348 (2014)
Louis, T.: Regression data sets (2016). http://www.dcc.fc.up.pt/ltorgo/Regression/DataSets.html
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 2nd edn. Springer-Verlag, New York (1994)
Pangilinan, J., Janssens, G.: Pareto-optimality of oblique decision trees from evolutionary algorithms. J. Global Optim. 51(2), 301–311 (2011)
Pappalardo, M.: Multiobjective optimization: a brief overview. In: Chinchuluun, A., Pardalos, P.M., Migdalas, A., Pitsoulis, L. (eds.) Pareto Optimality, Game Theory And Equilibria. Springer Optimization and Its Applications, vol. 17, pp. 517–528. Springer, Heidelberg (2008)
Potgieter, G., Engelbrecht, A.P.: Evolving model trees for mining data sets with continuous-valued classes. Expert Syst. Appl. 35(4), 1513–1532 (2008)
Quinlan, J.R.: Learning with continuous classes. Proc. Aust. Joint Conf. Artif. Intell. 92, 343–348 (1992). World Scientific, Singapore
Rokach, L., Maimon, O.: Top-down induction of decision trees classifiers - a survey. Trans. Sys. Man Cyber Part C 35(4), 476–487 (2005)
Rokach, L., Maimon, O.: Data Mining with Decision Trees: Theroy and Applications. World Scientific Publishing Co. Inc., River Edge (2008)
Schwarz, G.: Estimating the dimension of a model. Ann. Statist. 6(2), 461–464 (1978)
Zhang, X., Tian, Y., Cheng, R., Jin, Y.: An efficient approach to nondominated sorting for evolutionary multiobjective optimization. IEEE Trans. Evol. Comput. 19(2), 201–213 (2015)
Zhao, H.: A multi-objective genetic programming approach to developing pareto optimal decision trees. Decis. Support Syst. 43(3), 809–826 (2007)
Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: A comparative case study and the strength pareto approach. IEEE Trans. Evol. Comp. 3(4), 257–271 (1999)
Acknowledgments
This project was funded by the Polish National Science Center and allocated on the basis of decision 2013/09/N/ST6/04083. The second author was supported by the grant S/WI/2/13 from Bialystok University of Technology founded by Ministry of Science and Higher Education.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Czajkowski, M., Kretowski, M. (2016). A Multi-objective Evolutionary Approach to Pareto Optimal Model Trees. A Preliminary Study. In: MartÃn-Vide, C., Mizuki, T., Vega-RodrÃguez, M. (eds) Theory and Practice of Natural Computing. TPNC 2016. Lecture Notes in Computer Science(), vol 10071. Springer, Cham. https://doi.org/10.1007/978-3-319-49001-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-49001-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49000-7
Online ISBN: 978-3-319-49001-4
eBook Packages: Computer ScienceComputer Science (R0)