Abstract
Decision trees are some of the most popular and intuitive classification techniques. Based on the recursive division of the data, the goal is to ultimately identify regions in the space in which most instances belong to the same class. This paper proposes a game-theoretic decision tree using a two-player game to determine the splitting hyperplane at the node level based on the Nash equilibrium concept. The entropy on each sub-node is used as a payoff function that has to be minimized. The game’s equilibrium can be computed by minimizing an objective function constructed based on Nash equilibria properties. A new selection mechanism is proposed for the Covariance Matrix Adaptation - Evolution Strategy (CMA-ES) in order to approximate equilibria at each node level. Numerical experiments illustrate the behavior of the approach compared with other decision trees based methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Generated by using the function: make_classification(n_samples=50, n_features=2, n_redundant=0, n_informative=2, n_classes=2, random_state=50, class_sep=0.5, weights=[0.5]) from the Python module sklearn.datasets.
- 2.
UCI Machine Learning Repository https://archive.ics.uci.edu/ml/index.php, accessed October 2021.
References
Banos, R.C., Jaskowiak, P.A., Cerri, R., de Carvalho, A.C.P.L.F.: A framework for bottom-up induction of oblique decision trees. Neurocomputing 135(SI), 3–12 (2014). https://doi.org/10.1016/j.neucom.2013.01.067
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks, Monterey, CA (1984)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324
Cai, Y., Zhang, H., He, Q., Sun, S.: New classification technique: fuzzy oblique decision tree. Trans. Instit. Measur. Control 41(8, SI), 2185–2195 (2019). https://doi.org/10.1177/0142331218774614
ECNU: oblique decision tree in python. https://github.com/zhenlingcn/scikit-obliquetree (2021)
Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27(8), 861–874 (2006). https://doi.org/10.1016/j.patrec.2005.10.010. rOC Analysis in Pattern Recognition
Freedman, D.A.: Statistical Models: Theory and Practice. Cambridge University Press, 2 edn. (2009). https://doi.org/10.1017/CBO9780511815867
Hansen, N., Müller, S.D., Koumoutsakos, P.: Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evol. Comput. 11(1), 1–18 (2003). https://doi.org/10.1162/106365603321828970
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. SSS, Springer, New York (2009). https://doi.org/10.1007/978-0-387-84858-7
Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., Baesens, B.: An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decis. Support Syst. 51(1), 141–154 (2011). https://doi.org/10.1016/j.dss.2010.12.003
Leroux, A., Boussard, M., Des, R.: Inducing Readable Oblique Decision Trees. In: 2018 IEEE 30th International Conference on Tools With Artificial Intelligence (ICTAI), pp. 401–408 (2018). https://doi.org/10.1109/ICTAI.2018.00069. Volos, Greece, 05-07 Nov 2018
Li, Y., Dong, M., Kothari, R.: Classifiability-based omnivariate decision trees. IEEE Trans. Neural Netw. 16(6), 1547–1560 (2005)
McKelvey, R.D., McLennan, A.: Computation of equilibria in finite games. Handbook Comput. Econ. 1, 87–142 (1996)
Murthy, S.K., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. J. Artif. Intell. Res. 2, 1–32 (1994)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Rosset, S.: Model selection via the AUC. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 89. ICML 2004, Association for Computing Machinery, New York, NY, USA (2004). https://doi.org/10.1145/1015330.1015400
Scholz, M., Wimmer, T.: A comparison of classification methods across different data complexity scenarios and datasets. Expert Syst. Appl. 168, 114217 (2021). https://doi.org/10.1016/j.eswa.2020.114217
Setiono, R., Liu, H.: A connectionist approach to generating oblique decision trees. IEEE Trans. Syst. Man Cybern. Part B Cybern. 29(3), 440–444 (1999). https://doi.org/10.1109/3477.764880
yan Song, Y., Lu, Y.: Decision tree methods: applications for classification and prediction. Shanghai Archiv. Psychiatry 27, 130–135 (2015)
Wickramarachchi, D., Robertson, B., Reale, M., Price, C., Brown, J.: HHCART: an oblique decision tree. Comput. Statist. Data Anal. 96, 12–23 (2016). https://doi.org/10.1016/j.csda.2015.11.006
Wu, X., et al.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2008). https://doi.org/10.1007/s10115-007-0114-2
Zaki, M.J., Meira, W.: Data Mining and Machine Learning: Fundamental Concepts and Algorithms. Cambridge University Press, 2 edn. (2020). https://doi.org/10.1017/9781108564175
Acknowledgements
This work was supported by a grant of the Romanian Ministry of Education and Research, CNCS - UEFISCDI, project number PN-III-P4-ID-PCE-2020-2360, within PNCDI III.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lung, R.I., Suciu, MA. (2023). A Game Theoretic Decision Tree for Binary Classification. In: Legrand, P., et al. Artificial Evolution. EA 2022. Lecture Notes in Computer Science, vol 14091. Springer, Cham. https://doi.org/10.1007/978-3-031-42616-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-42616-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42615-5
Online ISBN: 978-3-031-42616-2
eBook Packages: Computer ScienceComputer Science (R0)