Abstract
Most dynamic ensemble selection (DES) techniques rely solely on local information to single out the most competent classifiers. However, data sparsity and class overlap may hinder the region definition step, yielding an unreliable local context for performing the selection task. Thus, we propose in this work a DES technique that uses both the local information and classifiers’ interactions to learn the ensemble combination rule. To that end, we encode the local information into a graph structure and the classifiers’ information into multiple meta-labels, and learn the DES technique end-to-end using a multi-label graph neural network (GNN). Experimental results over 35 high-dimensional problems show the proposed method outperforms most evaluated DES techniques as well as the static baseline, suggesting its suitability for dealing with sparse overlapped data.
The authors would like to thank the Canadian agencies FRQ (Fonds de Recherche du Québec) and NSERC (Natural Sciences and Engineering Research Council of Canada), and the Brazilian agencies CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior), CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico) and FACEPE (Fundação de Amparo à Ciência e Tecnologia de Pernambuco).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Armano, G., Tamponi, E.: Building forests of local trees. Pattern Recogn. 76, 380–390 (2018)
Cavalin, P.R., Sabourin, R., Suen, C.Y.: LoGID: an adaptive framework combining local and global incremental learning for dynamic selection of ensembles of HMMs. Pattern Recogn. 45(9), 3544–3556 (2012)
Cruz, R.M.O., Sabourin, R., Cavalcanti, G.D.C., Ren, T.I.: META-DES: a dynamic ensemble selection framework using meta-learning. Pattern Recogn. 48(5), 1925–1935 (2015)
Cruz, R.M.O., Hafemann, L.G., Sabourin, R., Cavalcanti, G.D.C.: DESlib: a dynamic ensemble selection library in python. J. Mach. Learn. Res. 21(8), 1–5 (2020)
Cruz, R.M., Sabourin, R., Cavalcanti, G.D.: Dynamic classifier selection: recent advances and perspectives. Inf. Fusion 41, 195–216 (2018)
Davtalab, R., Cruz, R.M., Sabourin, R.: Dynamic ensemble selection using fuzzy hyperboxes. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–9 (2022)
El-Sappagh, S., et al.: Alzheimer’s disease progression detection model based on an early fusion of cost-effective multimodal data. Futur. Gener. Comput. Syst. 115, 680–699 (2021)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, pp. 1024–1034 (2017)
Ho, T.K., Basu, M.: Complexity measures of supervised classification problems. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 289–300 (2002)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (ICLR) (2017)
Ko, A.H.-R., Sabourin, R., de Souza Britto Jr., A.: A new dynamic ensemble selection method for numeral recognition. In: Haindl, M., Kittler, J., Roli, F. (eds.) MCS 2007. LNCS, vol. 4472, pp. 431–439. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72523-7_43
Kuncheva, L.I.: A theoretical study on six classifier fusion strategies. IEEE Trans. Pattern Anal. Mach. Intell. 24(2), 281–286 (2002)
Li, D., Wen, G., Li, X., Cai, X.: Graph-based dynamic ensemble pruning for facial expression recognition. Appl. Intell. 49(9), 3188–3206 (2019)
Lorena, A.C., Costa, I.G., Spolaôr, N., De Souto, M.C.: Analysis of complexity indices for classification problems: cancer gene expression data. Neurocomputing 75(1), 33–42 (2012)
Narassiguin, A., Elghazel, H., Aussem, A.: Dynamic ensemble selection with probabilistic classifier chains. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10534, pp. 169–186. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71249-9_11
Oliveira, D.V., Cavalcanti, G.D., Porpino, T.N., Cruz, R.M., Sabourin, R.: K-nearest oracles borderline dynamic classifier ensemble selection. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2018)
Pereira, M., Britto, A., Oliveira, L., Sabourin, R.: Dynamic ensemble selection by K-nearest local Oracles with discrimination index. In: 2018 IEEE 30th International Conference on Tools with Artificial Intelligence, pp. 765–771. IEEE (2018)
Pinto, F., Soares, C., Mendes-Moreira, J.: CHADE: metalearning with classifier chains for dynamic combination of classifiers. In: Frasconi, P., Landwehr, N., Manco, G., Vreeken, J. (eds.) ECML PKDD 2016. LNCS (LNAI), vol. 9851, pp. 410–425. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46128-1_26
Salehi, A., Davulcu, H.: Graph attention auto-encoders. In: 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence, pp. 989–996 (2020)
Sánchez, J.S., Mollineda, R.A., Sotoca, J.M.: An analysis of how training data complexity affects the nearest neighbor classifiers. Pattern Anal. Appl. 10(3), 189–201 (2007)
Soares, R.G., Santana, A., Canuto, A.M., de Souto, M.C.P.: Using accuracy and diversity to select classifiers to build ensembles. In: The 2006 IEEE International Joint Conference on Neural Network (IJCNN) Proceedings, pp. 1310–1316 (2006)
Souza, M.A., Cavalcanti, G.D., Cruz, R.M., Sabourin, R.: Online local pool generation for dynamic classifier selection. Pattern Recogn. 85, 132–148 (2019)
Souza, M.A., Sabourin, R., Cavalcanti, G.D.C., Cruz, R.M.O.: Local overlap reduction procedure for dynamic ensemble selection. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–9 (2022)
Souza, M.A., Sabourin, R., Cavalcanti, G.D., Cruz, R.M.: OLP++: an online local classifier for high dimensional data. Inf. Fusion 90, 120–137 (2023)
Vandaele, R., Kang, B., De Bie, T., Saeys, Y.: The curse revisited: when are distances informative for the ground truth in noisy high-dimensional data? In: International Conference on Artificial Intelligence and Statistics, pp. 2158–2172. PMLR (2022)
Vanschoren, J., van Rijn, J.N., Bischl, B., Torgo, L.: OpenML: networked science in machine learning. SIGKDD Explor. 15(2), 49–60 (2013)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018)
Woloszynski, T., Kurzynski, M.: A probabilistic model of classifier competence for dynamic ensemble selection. Pattern Recogn. 44(10), 2656–2668 (2011)
Xia, F., et al.: Graph learning: a survey. IEEE Trans. Artif. Intell. 2(2), 109–127 (2021)
Zhang, S.: Challenges in KNN classification. IEEE Trans. Knowl. Data Eng. 34(10), 4663–4675 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
de Araujo Souza, M., Sabourin, R., da Cunha Cavalcanti, G.D., e Cruz, R.M.O. (2023). GNN-DES: A New End-to-End Dynamic Ensemble Selection Method Based on Multi-label Graph Neural Network. In: Vento, M., Foggia, P., Conte, D., Carletti, V. (eds) Graph-Based Representations in Pattern Recognition. GbRPR 2023. Lecture Notes in Computer Science, vol 14121. Springer, Cham. https://doi.org/10.1007/978-3-031-42795-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-42795-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42794-7
Online ISBN: 978-3-031-42795-4
eBook Packages: Computer ScienceComputer Science (R0)