Skip to main content
SpringerLink
Log in

A New Function for Ensemble Pruning

  • Conference paper
  • First Online:
Decision Support Systems VIII: Sustainable Data-Driven and Evidence-Based Decision Support (ICDSST 2018)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 313))

Included in the following conference series:

Abstract

We propose in this work a new function named Diversity and Accuracy for Pruning Ensembles (DAPE) which takes into account both accuracy and diversity to prune an ensemble of homogenous classifiers. A comparative study with a diversity based method and experimental results on several datasets show the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Comput. 9, 1545–1588 (1997)

    Article  Google Scholar 

  2. Asuncion, A., Newman, D.J.: UCI machine learning repository (2007). http://www.ics.uci.edu/mlearn/MLRepository.html

  3. Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Ensemble diversity measures and their application to thinning. Inf. Fusion 6(1), 49–62 (2005)

    Article  Google Scholar 

  4. Bhatnagar, V., Bhardwaj, M., Sharma, S., Haroon, S.: Accuracy-diversity based pruning of classifier ensembles. Prog. Artif. Intell. 2(2–3), 97–111 (2014)

    Article  Google Scholar 

  5. Biau, G., Cérou, F., Guyader, A.: On the rate of convergence of the bagged nearest neighbor estimate. J. Mach. Learn. Res. 11, 687–712 (2010)

    Google Scholar 

  6. Breiman, L.: Bagging predictors. Mach. Learn. 26(2), 123–140 (1996)

    Google Scholar 

  7. Breiman, L.: Randomizing outputs to increase prediction accuracy. Mach. Learn. 40, 229–242 (2000)

    Article  Google Scholar 

  8. Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)

    Article  Google Scholar 

  9. Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble selection from libraries of models. In: Proceedings of the 21st International Conference on Machine Learning (2004)

    Google Scholar 

  10. Cavalcanti, G.D.C., Oliveira, L.S., Moura, T.J.M., Carvalho, G.V.: Combining diversity measures for ensemble pruning. Pattern Recognit. Lett. 74, 38–45 (2016). ISSN 0167-8655

    Article  Google Scholar 

  11. Qun, D., Ye, R., Liu, Z.: Considering diversity and accuracy simultaneously for ensemble pruning. Appl. Soft Comput. 58, 75–91 (2017). ISSN 1568-4946

    Article  Google Scholar 

  12. Dietterich, T.G., Kong, E.B.: Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Technical report, Dept of Computer Science, Oregon State University, Covallis, Oregon (1995)

    Google Scholar 

  13. Fan, W., Chu, F., Wang, H., Yu, P.S.: Pruning and dynamic scheduling of cost-sensitive ensembles. In: Eighteenth National Conference on Artificial Intelligence, pp. 146–151. American Association for Artificial Intelligence (2002)

    Google Scholar 

  14. Fu, Q., Hu, S.X., Zhao, S.Y.: Clustering-based selective neural network ensemble. J. Zhejiang Univ. Sci. A 6(5), 387–392 (2005)

    Article  Google Scholar 

  15. Guo, H., Liu, H., Li, R., Wu, C., Guo, Y., Xu, M.: Margin & diversity based ordering ensemble pruning. Neurocomputing 275, 237–246 (2017). ISSN 0925-2312

    Article  Google Scholar 

  16. Hernández-Lobato, D., Martínez-Munoz, G.: A statistical instance-based pruning in ensembles of independent classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 364–369 (2009)

    Article  Google Scholar 

  17. Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)

    Article  Google Scholar 

  18. Margineantu, D., Dietterich, T.G.: Pruning adaptive boosting. In: Proceedings of the 14th International Conference on Machine Learning, pp. 211–218. Morgan Kaufmann, San Francisco (1997)

    Google Scholar 

  19. Markatopoulou, F., Tsoumakas, G., Vlahavas, I.: Instance-based ensemble pruning via multi-label classification. In: ICTAI 2010 (2010)

    Google Scholar 

  20. Martínez-Muñoz, G., Suárez, A.: Aggregation ordering in bagging. In: Proceedings of the IASTED International Conference on Artificial Intelligence and Applications, pp. 258–263. Acta Press (2004)

    Google Scholar 

  21. Martínez-Muñoz, G., Hernández-Lobato, D., Suárez, A.: Selection of decision stumps in bagging ensembles. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds.) ICANN 2007. LNCS, vol. 4668, pp. 319–328. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74690-4_33

    Chapter  Google Scholar 

  22. Li, N., Yu, Y., Zhou, Z.-H.: Diversity regularized ensemble pruning. In: Flach, P.A., De Bie, T., Cristianini, N. (eds.) ECML PKDD 2012. LNCS (LNAI), vol. 7523, pp. 330–345. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33460-3_27

    Chapter  Google Scholar 

  23. Partalas, I., Tsoumakas, G., Katakis, I., Vlahavas, I.: Ensemble pruning using reinforcement learning. In: Antoniou, G., Potamias, G., Spyropoulos, C., Plexousakis, D. (eds.) SETN 2006. LNCS (LNAI), vol. 3955, pp. 301–310. Springer, Heidelberg (2006). https://doi.org/10.1007/11752912_31

    Chapter  Google Scholar 

  24. Partalas, I., Tsoumakas, G., Vlahavas, I.: Focused ensemble selection: a diversity-based method for greedy ensemble selection. In: Ghallab, M., Spyropoulos, C.D., Fakotakis, N., Avouris, N.M. (eds.) ECAI 2008 - 18th European Conference on Artificial Intelligence. Proceedings of the Frontiers in Artificial Intelligence and Applications, Patras, Greece, 21–25 July 2008, vol. 178, pp. 117–121. IOS Press (2008)

    Google Scholar 

  25. Partalas, I., Tsoumakas, G., Vlahavas, I.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach. Learn. 81, 257–282 (2010)

    Article  Google Scholar 

  26. Quinlan, J.R.: C4.5 Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  27. Soto, V., Martínez-Muñoz, G., Hernández-Lobato, D., Suárez, A.: A double pruning algorithm for classification ensembles. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 104–113. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12127-2_11

    Chapter  Google Scholar 

  28. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, Los Altos (2005)

    Google Scholar 

  29. Zheng, Z., Webb, G.I.: Stochastic attribute selection committees. Technical report (TR C98/08), School of Computing and Mathematics, Deakin University, Australia (1998)

    Google Scholar 

  30. Zhou, H., Zhao, X., Wang, X.: An effective ensemble pruning algorithm based on frequent patterns. Knowl.-Based Syst. 56, 79–85 (2014). ISSN 0950-7051

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Economics, University of Oran 2, Oran, Algeria

    Souad Taleb Zouggar

  2. Department of Computer Science, University of Oran 1, Oran, Algeria

    Abdelkader Adla

Authors
  1. Souad Taleb Zouggar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelkader Adla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Souad Taleb Zouggar .

Editor information

Editors and Affiliations

  1. SimTech Simulation Technology, Graz, Austria

    Fatima Dargam

  2. Eastern Macedonia and Thrace Institute of Technology, Kavala, Greece

    Pavlos Delias

  3. University of Namur, Namur, Belgium

    Isabelle Linden

  4. ULB, Brussels, Belgium

    Bertrand Mareschal

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zouggar, S.T., Adla, A. (2018). A New Function for Ensemble Pruning. In: Dargam, F., Delias, P., Linden, I., Mareschal, B. (eds) Decision Support Systems VIII: Sustainable Data-Driven and Evidence-Based Decision Support. ICDSST 2018. Lecture Notes in Business Information Processing, vol 313. Springer, Cham. https://doi.org/10.1007/978-3-319-90315-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-90315-6_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-90314-9

  • Online ISBN: 978-3-319-90315-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation