Skip to main content

GNN-DES: A New End-to-End Dynamic Ensemble Selection Method Based on Multi-label Graph Neural Network

  • Conference paper
  • First Online:
Graph-Based Representations in Pattern Recognition (GbRPR 2023)

Abstract

Most dynamic ensemble selection (DES) techniques rely solely on local information to single out the most competent classifiers. However, data sparsity and class overlap may hinder the region definition step, yielding an unreliable local context for performing the selection task. Thus, we propose in this work a DES technique that uses both the local information and classifiers’ interactions to learn the ensemble combination rule. To that end, we encode the local information into a graph structure and the classifiers’ information into multiple meta-labels, and learn the DES technique end-to-end using a multi-label graph neural network (GNN). Experimental results over 35 high-dimensional problems show the proposed method outperforms most evaluated DES techniques as well as the static baseline, suggesting its suitability for dealing with sparse overlapped data.

The authors would like to thank the Canadian agencies FRQ (Fonds de Recherche du Québec) and NSERC (Natural Sciences and Engineering Research Council of Canada), and the Brazilian agencies CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior), CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico) and FACEPE (Fundação de Amparo à Ciência e Tecnologia de Pernambuco).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Armano, G., Tamponi, E.: Building forests of local trees. Pattern Recogn. 76, 380–390 (2018)

    Article  Google Scholar 

  2. Cavalin, P.R., Sabourin, R., Suen, C.Y.: LoGID: an adaptive framework combining local and global incremental learning for dynamic selection of ensembles of HMMs. Pattern Recogn. 45(9), 3544–3556 (2012)

    Article  Google Scholar 

  3. Cruz, R.M.O., Sabourin, R., Cavalcanti, G.D.C., Ren, T.I.: META-DES: a dynamic ensemble selection framework using meta-learning. Pattern Recogn. 48(5), 1925–1935 (2015)

    Article  Google Scholar 

  4. Cruz, R.M.O., Hafemann, L.G., Sabourin, R., Cavalcanti, G.D.C.: DESlib: a dynamic ensemble selection library in python. J. Mach. Learn. Res. 21(8), 1–5 (2020)

    Google Scholar 

  5. Cruz, R.M., Sabourin, R., Cavalcanti, G.D.: Dynamic classifier selection: recent advances and perspectives. Inf. Fusion 41, 195–216 (2018)

    Article  Google Scholar 

  6. Davtalab, R., Cruz, R.M., Sabourin, R.: Dynamic ensemble selection using fuzzy hyperboxes. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–9 (2022)

    Google Scholar 

  7. El-Sappagh, S., et al.: Alzheimer’s disease progression detection model based on an early fusion of cost-effective multimodal data. Futur. Gener. Comput. Syst. 115, 680–699 (2021)

    Article  Google Scholar 

  8. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  9. Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, pp. 1024–1034 (2017)

    Google Scholar 

  10. Ho, T.K., Basu, M.: Complexity measures of supervised classification problems. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 289–300 (2002)

    Article  Google Scholar 

  11. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (ICLR) (2017)

    Google Scholar 

  12. Ko, A.H.-R., Sabourin, R., de Souza Britto Jr., A.: A new dynamic ensemble selection method for numeral recognition. In: Haindl, M., Kittler, J., Roli, F. (eds.) MCS 2007. LNCS, vol. 4472, pp. 431–439. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72523-7_43

  13. Kuncheva, L.I.: A theoretical study on six classifier fusion strategies. IEEE Trans. Pattern Anal. Mach. Intell. 24(2), 281–286 (2002)

    Article  Google Scholar 

  14. Li, D., Wen, G., Li, X., Cai, X.: Graph-based dynamic ensemble pruning for facial expression recognition. Appl. Intell. 49(9), 3188–3206 (2019)

    Article  Google Scholar 

  15. Lorena, A.C., Costa, I.G., Spolaôr, N., De Souto, M.C.: Analysis of complexity indices for classification problems: cancer gene expression data. Neurocomputing 75(1), 33–42 (2012)

    Article  Google Scholar 

  16. Narassiguin, A., Elghazel, H., Aussem, A.: Dynamic ensemble selection with probabilistic classifier chains. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10534, pp. 169–186. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71249-9_11

    Chapter  Google Scholar 

  17. Oliveira, D.V., Cavalcanti, G.D., Porpino, T.N., Cruz, R.M., Sabourin, R.: K-nearest oracles borderline dynamic classifier ensemble selection. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2018)

    Google Scholar 

  18. Pereira, M., Britto, A., Oliveira, L., Sabourin, R.: Dynamic ensemble selection by K-nearest local Oracles with discrimination index. In: 2018 IEEE 30th International Conference on Tools with Artificial Intelligence, pp. 765–771. IEEE (2018)

    Google Scholar 

  19. Pinto, F., Soares, C., Mendes-Moreira, J.: CHADE: metalearning with classifier chains for dynamic combination of classifiers. In: Frasconi, P., Landwehr, N., Manco, G., Vreeken, J. (eds.) ECML PKDD 2016. LNCS (LNAI), vol. 9851, pp. 410–425. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46128-1_26

    Chapter  Google Scholar 

  20. Salehi, A., Davulcu, H.: Graph attention auto-encoders. In: 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence, pp. 989–996 (2020)

    Google Scholar 

  21. Sánchez, J.S., Mollineda, R.A., Sotoca, J.M.: An analysis of how training data complexity affects the nearest neighbor classifiers. Pattern Anal. Appl. 10(3), 189–201 (2007)

    Article  MathSciNet  Google Scholar 

  22. Soares, R.G., Santana, A., Canuto, A.M., de Souto, M.C.P.: Using accuracy and diversity to select classifiers to build ensembles. In: The 2006 IEEE International Joint Conference on Neural Network (IJCNN) Proceedings, pp. 1310–1316 (2006)

    Google Scholar 

  23. Souza, M.A., Cavalcanti, G.D., Cruz, R.M., Sabourin, R.: Online local pool generation for dynamic classifier selection. Pattern Recogn. 85, 132–148 (2019)

    Article  Google Scholar 

  24. Souza, M.A., Sabourin, R., Cavalcanti, G.D.C., Cruz, R.M.O.: Local overlap reduction procedure for dynamic ensemble selection. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–9 (2022)

    Google Scholar 

  25. Souza, M.A., Sabourin, R., Cavalcanti, G.D., Cruz, R.M.: OLP++: an online local classifier for high dimensional data. Inf. Fusion 90, 120–137 (2023)

    Article  Google Scholar 

  26. Vandaele, R., Kang, B., De Bie, T., Saeys, Y.: The curse revisited: when are distances informative for the ground truth in noisy high-dimensional data? In: International Conference on Artificial Intelligence and Statistics, pp. 2158–2172. PMLR (2022)

    Google Scholar 

  27. Vanschoren, J., van Rijn, J.N., Bischl, B., Torgo, L.: OpenML: networked science in machine learning. SIGKDD Explor. 15(2), 49–60 (2013)

    Article  Google Scholar 

  28. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018)

    Google Scholar 

  29. Woloszynski, T., Kurzynski, M.: A probabilistic model of classifier competence for dynamic ensemble selection. Pattern Recogn. 44(10), 2656–2668 (2011)

    Article  MATH  Google Scholar 

  30. Xia, F., et al.: Graph learning: a survey. IEEE Trans. Artif. Intell. 2(2), 109–127 (2021)

    Article  Google Scholar 

  31. Zhang, S.: Challenges in KNN classification. IEEE Trans. Knowl. Data Eng. 34(10), 4663–4675 (2022)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mariana de Araujo Souza .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 112 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

de Araujo Souza, M., Sabourin, R., da Cunha Cavalcanti, G.D., e Cruz, R.M.O. (2023). GNN-DES: A New End-to-End Dynamic Ensemble Selection Method Based on Multi-label Graph Neural Network. In: Vento, M., Foggia, P., Conte, D., Carletti, V. (eds) Graph-Based Representations in Pattern Recognition. GbRPR 2023. Lecture Notes in Computer Science, vol 14121. Springer, Cham. https://doi.org/10.1007/978-3-031-42795-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42795-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42794-7

  • Online ISBN: 978-3-031-42795-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics