Abstract
Decision trees and their ensembles are very popular models of supervised machine learning. In this paper we merge the ideas underlying decision trees, their ensembles and FCA by proposing a new supervised machine learning model which can be constructed in polynomial time and is applicable for both classification and regression problems. Specifically, we first propose a polynomial-time algorithm for constructing a part of the concept lattice that is based on a decision tree. Second, we describe a prediction scheme based on a concept lattice for solving both classification and regression tasks with prediction quality comparable to that of state-of-the-art models.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Experiments source code. https://github.com/EgorDudyrev/FCApy/tree/main/notebooks/DecisionLattice_evaluation
Sci-kit learn description of decision trees. https://scikit-learn.org/stable/modules/tree.html
Sci-kit learn description of random forest. https://scikit-learn.org/stable/modules/ensemble.html#parameters
Belohlavek, R., De Baets, B., Outrata, J., Vychodil, V.: Inducing decision trees via concept lattices (2007)
Breiman, L., Friedman, J., Stone, C., Olshen, R.: Classification and Regression Trees. Taylor & Francis, New York (1984)
Breiman, L.: Random forests. Machine Learning (2001)
Buzmakov, A., Kuznetsov, S., Napoli, A.: Sofia: How to make FCA polynomial? In: FCA4AI@IJCAI (2015)
Buzmakov, A., Egho, E., Jay, N., Kuznetsov, S., Napoli, A., Rassi, C.: FCA and pattern structures for mining care trajectories. In: CEUR Workshop Proceedings, vol. 1058 (2013)
Drucker, H., Cortes, C.: Boosting decision trees, vol. 8, pp. 479–485 (1995)
Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations. Springer, Berlin Heidelberg (1999). https://doi.org/10.1007/978-3-642-59830-2
Kaytoue, M., Kuznetsov, S., Napoli, A., Duplessis, S.: Mining gene expression data with pattern structures in formal concept analysis. Inf. Sci. 181, 1989–2001 (2011)
Krause, T., Lumpe, L., Schmidt, S.: A link between pattern structures and random forests. In: CLA (2020)
Kuznetsov, S.: Pattern structures for analyzing complex data, vol. 5908, pp. 33–44 (2009)
Kuznetsov, S., Makhalova, T.: On interestingness measures of formal concepts. Inf. Sci. 442 (2016)
Kuznetsov, S., Obiedkov, S.: Comparing performance of algorithms for generating concept lattices. J. Exp. Theor. Artif. Intell. 14, 189–216 (2002)
Kuznetsov, S.O.: Machine learning and formal concept analysis. In: Eklund, P. (ed.) ICFCA 2004. LNCS (LNAI), vol. 2961, pp. 287–312. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24651-0_25
Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features (2019)
Wille, R.: Restructuring lattice theory: an approach based on hierarchies of concepts. In: Ferré, S., Rudolph, S. (eds.) ICFCA 2009. LNCS (LNAI), vol. 5548, pp. 314–339. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01815-2_23
Acknowledgments
The work of Sergei O. Kuznetsov on the paper was carried out at St. Petersburg Department of Steklov Mathematical Institute of Russian Academy of Science and supported by the Russian Science Foundation grant no. 17-11-01276
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Dudyrev, E., Kuznetsov, S.O. (2021). Decision Concept Lattice vs. Decision Trees and Random Forests. In: Braud, A., Buzmakov, A., Hanika, T., Le Ber, F. (eds) Formal Concept Analysis. ICFCA 2021. Lecture Notes in Computer Science(), vol 12733. Springer, Cham. https://doi.org/10.1007/978-3-030-77867-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-77867-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77866-8
Online ISBN: 978-3-030-77867-5
eBook Packages: Computer ScienceComputer Science (R0)