Skip to main content
Springer Nature Link
Log in

Variable Consistency Bagging Ensembles

  • Chapter
Transactions on Rough Sets XI

Part of the book series: Lecture Notes in Computer Science ((TRS,volume 5946))

Abstract

In this paper we claim that the classification performance of bagging classifier can be improved by drawing to bootstrap samples objects being more consistent with their assignment to decision classes. We propose a variable consistency generalization of the bagging scheme where such sampling is controlled by two types of measures of consistency: rough membership and monotonic ε measure. The usefulness of this proposal is experimentally confirmed with various rule and tree base classifiers. The results of experiments show that variable consistency bagging improves classification accuracy on inconsistent data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Aha, D.W., Kibler, E., Albert, M.K.: Instance-based learning algorithms. Machine Learning Journal 6, 37–66 (1991)

    Google Scholar 

  2. Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine (2007), http://www.ics.uci.edu/~mlearn/MLRepositoru.html

  3. Błaszczyński, J., Greco, S., Słowiński, R., Szeląg, M.: On Variable Consistency Dominance-based Rough Set Approaches. In: Greco, S., Hata, Y., Hirano, S., Inuiguchi, M., Miyamoto, S., Nguyen, H.S., Słowiński, R. (eds.) RSCTC 2006. LNCS (LNAI), vol. 4259, pp. 191–202. Springer, Heidelberg (2006)

    Google Scholar 

  4. Błaszczyński, J., Greco, S., Słowiński, R., Szeląg, M.: Monotonic Variable Consistency Rough Set Approaches. In: Yao, J., Lingras, P., Wu, W.-Z., Szczuka, M.S., Cercone, N.J., Ślȩzak, D. (eds.) RSKT 2007. LNCS (LNAI), vol. 4481, pp. 126–133. Springer, Heidelberg (2007)

    Google Scholar 

  5. Breiman, L.: Bagging predictors. Machine Learning Journal 24(2), 123–140 (1996)

    MATH  MathSciNet  Google Scholar 

  6. Breiman, L.: Pasting small votes for classification in large databases and on-line. Machine Learning Journal 36, 85–103 (1999)

    Article  Google Scholar 

  7. Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  8. Cohen, W.W.: Fast effective rule induction. In: Proc. of the 12th Int. Conference on Machine Learning, pp. 115–123 (1995)

    Google Scholar 

  9. Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)

    MathSciNet  Google Scholar 

  10. Dietrich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  11. Greco, S., Matarazzo, B., Słowiński, R.: Rough sets theory for multicriteria decision analysis. European Journal of Operational Research 129, 1–47 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  12. Greco, S., Matarazzo, B., Słowiński, R., Stefanowski, J.: An algorithm for induction of decision rules consistent with dominance principle. In: Ziarko, W.P., Yao, Y. (eds.) RSCTC 2000. LNCS (LNAI), vol. 2005, pp. 304–313. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  13. Grzymala-Busse, J.W.: Managing uncertainty in machine learning from examples. In: Proc. 3rd Int. Symp. in Intelligent Systems, pp. 70–84 (1994)

    Google Scholar 

  14. Grzymala-Busse, J.W., Stefanowski, J.: Three approaches to numerical attribute discretization for rule induction. International Journal of Intelligent Systems 16(1), 29–38 (2001)

    Article  MATH  Google Scholar 

  15. Kuncheva, L.: Combining Pattern Classifiers. In: Methods and Algorithms, Wiley, Chichester (2004)

    Google Scholar 

  16. Latinne, P., Debeir, O., Decaestecker, Ch.: Different ways of weakening decision trees and their impact on classification accuracy of decision tree combination. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, p. 200. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  17. Nemenyi, P.B.: Distribution free multiple comparison. Ph.D. Thesis, Princenton Univeristy (1963)

    Google Scholar 

  18. Hoa, N.S., Nguyen, T.T., Son, N.H.: Rough sets approach to sunspot classification problem. In: Ślęzak, D., Yao, J., Peters, J.F., Ziarko, W.P., Hu, X. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3642, pp. 263–272. Springer, Heidelberg (2005)

    Google Scholar 

  19. Pawlak, Z.: Rough sets. International Journal of Information & Computer Sciences 11, 341–356 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  20. Quinlan, J.R.: Bagging, boosting and C4.5. In: Proc. of the 13th National Conference on Artificial Intelligence, pp. 725–730 (1996)

    Google Scholar 

  21. Słowiński, R., Greco, S., Matarazzo, B.: Rough set based decision support. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, pp. 475–527. Springer, Heidelberg (2005)

    Google Scholar 

  22. Slezak, D.: Approximate entropy reducts. Fundamenta Informaticae 53(3/4), 365–387 (2002)

    MathSciNet  Google Scholar 

  23. Stefanowski, J.: The rough set based rule induction technique for classification problems. In: Proc. of 6th European Conference on Intelligent Techniques and Soft Computing. EUFIT 1998, pp. 109–113 (1998)

    Google Scholar 

  24. Stefanowski, J.: Multiple and hybrid classifiers. In: Polkowski, L. (ed.) Formal Methods and Intelligent Techniques in Control, Decision Making, Multimedia and Robotics, Post-Proceedings of 2nd Int. Conference, Warszawa, pp. 174–188 (2001)

    Google Scholar 

  25. Stefanowski, J.: The bagging and n2-classifiers based on rules induced by MODLEM. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 488–497. Springer, Heidelberg (2004)

    Google Scholar 

  26. Stefanowski, J., Kaczmarek, M.: Integrating attribute selection to improve accuracy of bagging classifiers. In: Proc. of the AI-METH 2004 Conference - Recent Developments in Artificial Intelligence Methods, Gliwice, pp. 263–268 (2004)

    Google Scholar 

  27. Stefanowski, J., Nowaczyk, S.: An experimental study of using rule induction in combiner multiple classifier. International Journal of Computational Intelligence Research 3(4), 335–342 (2007)

    Article  Google Scholar 

  28. Stefanowski, J., Wilk, S.: Improving Rule Based Classifiers Induced by MODLEM by Selective Pre-processing of Imbalanced Data. In: Proc. of the RSKD Workshop at ECML/PKDD, Warsaw, pp. 54–65 (2007)

    Google Scholar 

  29. Suraj, Z., Gayar Neamat, E., Delimata, P.: A Rough Set Approach to Multiple Classifier Systems. Fundamenta Informaticae 72(1-3), 393–406 (2006)

    MATH  Google Scholar 

  30. Wilson, D.R., Martinez, T.: Reduction techniques for instance-based learning algorithms. Machine Learning Journal 38, 257–286 (2000)

    Article  MATH  Google Scholar 

  31. Valentini, G., Masuli, F.: Ensembles of Learning Machines. In: Marinaro, M., Tagliaferri, R. (eds.) WIRN 2002. LNCS, vol. 2486, pp. 3–19. Springer, Heidelberg (2002)

    Google Scholar 

  32. Wong, S.K.M., Ziarko, W.: Comparison of the probabilistic approximate classification and the fuzzy set model. Fuzzy Sets and Systems 21, 357–362 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  33. Ziarko, W.: Variable precision rough sets model. Journal of Computer and Systems Sciences 46(1), 39–59 (1993)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computing Science, Poznań University of Technology, 60-965, Poznań, Poland

    Jerzy Błaszczyński, Roman Słowiński & Jerzy Stefanowski

  2. Systems Research Institute, Polish Academy of Sciences, 01-447, Warsaw, Poland

    Roman Słowiński

Authors
  1. Jerzy Błaszczyński
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roman Słowiński
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jerzy Stefanowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Manitoba, Department of Electrical and Computer Engineering, , , ,, Winnipeg, R3T 5V6, Manitoba, Canada

    James F. Peters

  2. Institute of Mathematics, Warsaw University, Banacha 2, 02-097, Warsaw, Poland

    Andrzej Skowron

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Błaszczyński, J., Słowiński, R., Stefanowski, J. (2010). Variable Consistency Bagging Ensembles. In: Peters, J.F., Skowron, A. (eds) Transactions on Rough Sets XI. Lecture Notes in Computer Science, vol 5946. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11479-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-11479-3_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-11478-6

  • Online ISBN: 978-3-642-11479-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics