Skip to main content
SpringerLink
Log in

An ensemble method for fuzzy rule-based classification systems

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Fuzzy rule-based classification systems are very useful tools in the field of machine learning as they are able to build linguistic comprehensible models. However, these systems suffer from exponential rule explosion when the number of variables increases, degrading, therefore, the accuracy of these systems as well as their interpretability. In this article, we propose to improve the comprehensibility through a supervised learning method by automatic generation of fuzzy classification rules, designated SIFCO–PAF. Our method reduces the complexity by decreasing the number of rules and of antecedent conditions, making it thus adapted to the representation and the prediction of rather high-dimensional pattern classification problems. We perform, firstly, an ensemble methodology by combining a set of simple classification models. Subsequently, each model uses a subset of the initial attributes: In this case, we propose to regroup the attributes using linear correlation search among the training set elements. Secondly, we implement an optimal fuzzy partition thanks to supervised discretization followed by an automatic membership functions construction. The SIFCO–PAF method, analyzed experimentally on various data sets, guarantees an important reduction in the number of rules and of antecedents without deteriorating the classification rates, on the contrary accuracy is even improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. SIFCO is acronyme of systéme d’Inférence floue avec corrélation (fuzzy inference system with correlation).

  2. SIFCO–PAF is acronyme of système d’Inférence floue avec corrélation et partition floue supervisée (fuzzy inference system with correlation and fuzzy supervised partition).

  3. In SIFCO–PAF, the number of fuzzy sets associated with each attribute may be different. Here, one can take for K the largest number of fuzzy sets.

  4. http://www.ics.uci.edu/mlearn/MLSummary.html.

  5. http://www.cs.waikato.ac.nz/ml/weka/.

References

  1. Alcalá R, Gacto MJ, Herreraz F (2007) A multi objective genetic algorithm for tuning and rule selection to obtain accurate and compact linguistic fuzzy rule-based systems. Int J Uncertain Fuzziness Knowl Based Syst 15(5):539–557

    Article  MATH  Google Scholar 

  2. Alcalá-Fdez J, Alcalá R, Gacto MJ et al (2009) Learning the membership function contexts for mining fuzzy association rules by using genetic algorithms. Fuzzy Sets Syst 160(7):905–921

    Article  MATH  Google Scholar 

  3. Angelov P, Zhou X (2008a) Evolving fuzzy-rule-based classifiers from data streams. IEEE Trans Fuzzy Syst Special Issue Evol Fuzzy Syst 16(6):1462–1475

    Article  Google Scholar 

  4. Angelov P, Lughofer E, Zhou X (2008b) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159:3160–3182

    Article  MathSciNet  MATH  Google Scholar 

  5. Bellmann R (1961) Adaptive control processes: a guided tour. Princeton University Press, Princeton

    Google Scholar 

  6. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MathSciNet  MATH  Google Scholar 

  7. Borgi A, Bazin JM, Akdag H (1998) Two methods of linear correlation search for a knowledge based supervised classification. Lect Notes Comput Sci 1415:696–707

    Article  Google Scholar 

  8. Casillas J, Cordón O, del Jesus MJ et al (2005) Genetic tuning of fuzzy rule deep structures preserving interpretability for linguistic modeling. IEEE Trans Fuzzy Syst 13(1):13–29

    Article  Google Scholar 

  9. Cohen W (1995) Fast effective rule induction. In: 12th international conference on machine learning, Morgan Kaufmann, Los Altos, pp 115–123

  10. Cordon O, Gomide F, Herrera F et al (2004) Ten years of genetic fuzzy systems: current framework and new trends. Fuzzy Sets Syst 141(1):5–31

    Article  MathSciNet  MATH  Google Scholar 

  11. Dhir CS, Lee J, Lee S-Y (2011) Extraction of independent discriminant features for data with asymmetric distribution. Knowl Inf Syst. doi:10.1007/s10115-011-0381-9

  12. Dougherty J, Kohavi R, Saham M (1995) Supervised and unsupervised discretization of continuous features. In: Proceedings of Twelfth international conference on machine learning. Morgan Kaufman, Los Altos, pp 194–202

  13. Fayyad UM, Irani KB (1993) Multi-interval discretization of continuous valued attributes for classification learning. In: Proceedings of the 13th international joint conference on artificial intelligence, pp 1022–1027

  14. Frank E, Witten I (1998) Generating accurate rule sets without global optimization. In: 15th international conference on machine learning, Morgan Kaufmann, Los Altos, pp 144–151

  15. Freund Y (1990) Boosting a weak learning algorithm by majority. In: COLT ’90 proceedings of the third annual workshop on computational learning theory, Morgan Kaufmann, Los Altos

  16. Gomez JC, Boiy E, Moens M-F (2011) Highly discriminative statistical features for email classification. Knowl Inf Syst. doi:10.1007/s10115-011-0403-7

  17. Herrera F (2008) Genetic fuzzy systems: taxonomy, current research trends and prospects. Evol Intell 1(1):27–46

    Article  MathSciNet  Google Scholar 

  18. Holte R (1993) Very simple classification rules perform well on most commonly used datasets. Mach Learn 11:63–91

    Article  MATH  Google Scholar 

  19. Hussain F, Liu H, Tan CL et al (2002) Discretization: an enabling technique. Data Min Knowl Discov 6(4):393–423

    Article  MathSciNet  Google Scholar 

  20. Ishibuchi H, Nozaki K, Tanaka H (1992) Distributed representation of fuzzy rules and its application to pattern classification. Fuzzy Sets Syst 52:21–32

    Article  Google Scholar 

  21. Ishibuchi H, Nozaki K, Tanaka H (1993) Efficient fuzzy partition of pattern space for classification problems. Fuzzy Sets Syst 59:295–304

    Article  Google Scholar 

  22. Ishibuchi H, Nozaki K, Yamamoto N et al (1994) Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms. Fuzzy Sets Syst 65:253–273

    Article  MathSciNet  Google Scholar 

  23. Ishibuchi H, Murata T, Turksen IB (1997a) Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets Syst 89(2):135–150

    Article  Google Scholar 

  24. Ishibuchi H, Nakashima T, Morisawa T (1997b) Simple fuzzy rule-based classification systems performed well on commonly used real-world data sets. In: North American fuzzy information processing society meeting, pp 251–256

  25. Ishibuchi H, Nakashima T, Murata T (1999) Performance evaluation of fuzzy classifier systems for multi-dimensional pattern classification problems. IEEE Trans Syst Man Cybern Part B Cybern 29(5):601–618

    Article  Google Scholar 

  26. Ishibuchi H, Nakashima T, Murata T (2001a) Three-objective genetics-based machine learning for linguistic rule extraction. Inf Sci 136(1–4):109–133

    Google Scholar 

  27. Ishibuchi H, Yamamoto T, Nakashima T (2001b) Fuzzy data mining: effect of fuzzy discretization. In: Proceedings of 2001 IEEE international conference on data mining, San Jose, pp 241–248

  28. Ishibuchi H, Yamamoto N (2002) Effect of fuzzy discretization in fuzzy rule-based systems for classification problems with continuous attributes. Arch Control Sci 12(4):351–378

    MathSciNet  MATH  Google Scholar 

  29. Ishibuchi H, Yamamoto T (2004) Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining. Fuzzy Sets Syst 141(1):59–88

    Article  MathSciNet  MATH  Google Scholar 

  30. Ishibuchi H, Yamamoto T (2005) Rule weight specification in fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 13(4):428–435

    Article  Google Scholar 

  31. Ishibuchi H, Nojima Y (2007) Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning. Int J Approx Reason 44(1):4–31

    Article  MathSciNet  MATH  Google Scholar 

  32. Karr C (1991) Genetic algorithms for fuzzy controllers. AI Expert 6(2):26–33

    Google Scholar 

  33. Kerber R (1992) ChiMerge: discretization of numeric attributes. In: AAAI 92 10th national conference on artificial intelligence, pp 123–128

  34. Kohavi R (1995a) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th international joint conference on artificial intelligence (2), Canada

  35. Kohavi R (1995b) The power of decision tables. In: Eighth European conference on machine learning. Springer, Berlin, pp 174–189

  36. Kononenko I, Robnik-Sikija M, Pompe U (1996) ReliefF for estimation and discretization of attributes in classification, regression, and ILP problems. In: Ramsay AM (ed) Artificial intelligence: methodology, systems, applications, pp 31–40

  37. Lee HM, Chen C-M, Chen J-M et al (2001) An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans Syst Man Cybern Part B Cybern 31(3):426–432

    Google Scholar 

  38. Nakashima T, Morisawa T, Ishibuchi H, (1997) Input selection in fuzzy rule-based classification systems. In: Proceedings of 1997 IEEE international conference on fuzzy systems, Barcelona, pp 1457–1462

  39. Nozaki K, Ishibuchi H, Tanaka H (1994) Selecting fuzzy rules with forgetting in fuzzy classification systems. In: 3rd IEEE international conference on fuzzy systems (1), pp 618–623

  40. Prudhomme E, Lallich S (2008) Maps ensemble for semi-supervised learning of large high dimensional dataset. In: 19th international symposium ISMIS 2008, LNAI 4994, Springer, Berlin, pp 100–110

  41. Pulkkinen P, Koivisto H (2008) Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms. Int J Approx Reason 48(2):526–543

    Article  Google Scholar 

  42. Radovanovié M (2011) High-dimensional data representations and metrics for machine learning and data mining. Doctoral dissertation, Department of mathematics and informatics, faculty of science, University of Novi Sad, Serbia

  43. Rifkin R, Klautau A (2004) In defense of One-Vs-All classification. J Mach Learn Res, pp 101–141

  44. Rokach L (2009) Taxonomy for characterizing ensemble methods in classification tasks: a review and annotated bibliography. Comput Stat Data Anal 53(12):4046–4072

    Article  MathSciNet  MATH  Google Scholar 

  45. Roubos JA, Setnes M, Abonyi J (2003) Learning fuzzy classification rules from labeled data. Inf Sci 150:77–93

    Article  MathSciNet  Google Scholar 

  46. Saporta G (2006) Probabilités analyse de données et statistiques. Editions Technip Paris, deuxième édition

  47. Soua B, Borgi A, Tagina M (2009a) Fuzzy supervised partition in fuzzy rule based classification systems. In: 17th International conference on control systems and computer science (CSCS-17), 26–29 May 2009, Politehnica University of Bucharest, Bucharest. Ed. Politehnica Press. ISSN 2066 - 4451

  48. Soua B, Borgi A, Tagina M (2009b) Attributes regrouping in fuzzy rule based classification systems. In: International conference on signals, circuits and systems (SCS’09), IEEE, Djerba

  49. Valentini G, Masulli F (2002) Ensembles of learning machines. In: Marinaro M, Tagliaferri R (eds) Neural nets WIRN Vietri-02, LNCS 2486, Springer, Berlin, pp 3–19

  50. Verikas A, Guzaitis J, Gelzinis A et al (2011) A general framework for designing a fuzzy rule-based classifier. Knowl Inf Syst 29:203–221

    Article  Google Scholar 

  51. Verleysen M, François D, Simon G et al (2003) On the effects of dimensionality on data analysis with neural networks. In: Mira AJ (ed) International work-conference on artificial and natural neural networks, LNCS 2687. Springer, Berlin, pp 105–112

  52. Verleysen M (2003) Limitations and future trends in neural computation. In: Ablameyko S et al (eds) Chapter learning high-dimensional data. IOS Press, Amsterdam, pp 141–162

    Google Scholar 

  53. Vernazza G (1993) Image classification by extended certainty factors. In: Pattern recognition 26(11). Pergamon Press Ltd, Oxford, pp 1683–1694

  54. Witten I, Frank E (2005) Data mining: practical machine learning tools and techniques. Morgan Kaufmann, Los Altos

    Google Scholar 

  55. Zeidler J, Schlosser M (1995) Fuzzy handling of continuous-valued attributes in decision trees. In: Proceedings of the ECML-95 Mlnet familiarization workshop “statistics, machine learning and knowledge discovery in databases”, pp 41–46

Download references

Acknowledgments

The authors would like to thank Sarrah Ben Mbarek for revising the English of the present article.

Author information

Authors and Affiliations

  1. Centre de Développement des Télécommunications, Direction Systèmes d’Information, Tunisie Télecom, Parc Technologique de Raoued, Tunis, Tunisia

    Basma Soua

  2. URPAH/INSAT, Université de Carthage, Centre Urbain Nord de Tunis, 1080 , Tunis, Tunisia

    Amel Borgi

  3. National Computer Science School (ENSI), Université de la Manouba, 2010 , Manouba, Tunisia

    Moncef Tagina

Authors
  1. Basma Soua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amel Borgi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moncef Tagina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amel Borgi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Soua, B., Borgi, A. & Tagina, M. An ensemble method for fuzzy rule-based classification systems. Knowl Inf Syst 36, 385–410 (2013). https://doi.org/10.1007/s10115-012-0532-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-012-0532-7

Keywords

Search

Navigation