Skip to main content
SpringerLink
Log in

Supervised methods for regrouping attributes in fuzzy rule-based classification systems

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper focuses on ensemble methods for Fuzzy Rule-Based Classification Systems (FRBCS) where the decisions of different classifiers are combined in order to form the final classification model. The proposed methods reduce the FRBCS complexity and the generated rules number. We are interested in particular in ensemble methods which cluster the attributes into subgroups of attributes and treat each subgroup separately. Our work is an extension of a previous ensemble method called SIFRA. This method uses frequent itemsets mining concept in order to deduce the groups of related attributes by analyzing their simultaneous appearances in the databases. The drawback of this method is that it forms the groups of attributes by searching for dependencies between the attributes independently from the class information. Besides, since we deal with supervised learning problems, it would be very interesting to consider the class attribute when forming the attributes subgroups. In this paper, we proposed two new supervised attributes regrouping methods which take into account not only the dependencies between the attributes but also the information about the class labels. The results obtained with various benchmark datasets show a good accuracy of the built classification model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://archive.ics.uci.edu/ml/datasets.html

  2. https://www.cs.waikato.ac.nz/ml/weka/related.html

References

  1. Ishibuchi H, Nozaki K, Tanaka H (1992) Distributed representation of fuzzy rules and its application to pattern classification. Fuzzy Set Syst 52:21–32

    Article  Google Scholar 

  2. Cordon O, Del Jesus MJ, Herrera F (1999) A proposal on reasoning methods in fuzzy rule-based classification systems. Int J Approx Reason 20:21–45

    Article  Google Scholar 

  3. Dehzangi O, Zolghadri MJ, Taheri S, Fakhrahmad SM (2007) Efficient fuzzy rule generation: a new approach using data mining principles and rule weighting. Fuzzy Syst Knowl Discovery 2:134–139

    Article  Google Scholar 

  4. Elkano M, Galar M, Sanz J, Bustince H (2016) Fuzzy rule-based classification systems for multi-class problems using binary decomposition strategies: on the influence of n-dimensional overlap functions in the fuzzy reasoning method. Inf Sci 332:94–114

    Article  Google Scholar 

  5. Fakhrahmad SM, Zare A, Jahromi MZ (2007) Constructing accurate fuzzy rule-based classification systems using Apriori principles and rule-weighting. In: International conference on intelligent data engineering and automated learning. Springer, Berlin, pp 547–556

  6. Zadeh LA (1965) Fuzzy sets. Inform Control 8(3):338–353

    Article  Google Scholar 

  7. Lee HM, Chen CM, Chen JM, Jou YL (2001) An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans Syst Man Cybern, Part B: Cybern 31:426–432

    Article  Google Scholar 

  8. Chen YC, Pal NR, Chung IF (2012) An integrated mechanism for feature selection and fuzzy rule extraction for classification. IEEE Trans Fuzzy Syst 20(4):683–698

    Article  Google Scholar 

  9. Villar P, Fernandez A, Herrera F (2010) A genetic algorithm for feature selection and granularity learning in fuzzy rule-based classification systems for highly imbalanced data-sets. In: Information processing and management of uncertainty in knowledge-based systems. Theory and Methods, pp 741–750

  10. Phong PD (2015) An application of feature selection for the fuzzy rule-based classifier design based on an enlarger hedge algebras for high-dimensional datasets. J Sci Technol 53(5):583– 597

    Google Scholar 

  11. Gacto MJ, Alcala R, Herrera F (2009) Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems. Soft Comput 13:419–436

    Article  Google Scholar 

  12. Alcala R, Gacto MJ, Herrera F, Alcal-fdez J (2007) A multi-objective genetic algorithm for tuning and rule selection to obtain accurate and compact linguistic fuzzy rule-based systems. Int J Uncertainty, Fuzz Knowl-Based Syst 15:539–557

    Article  Google Scholar 

  13. Ishibuchi H, Yamamoto T (2004) Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining. Fuzzy Sets Syst 141:59–88

    Article  Google Scholar 

  14. Antonelli M, Ducange P, Marcelloni F (2014) A fast and efficient multi-objective evolutionary learning scheme for fuzzy rule-based classifiers. Inf Sci 283:36–54

    Article  MathSciNet  Google Scholar 

  15. Fazzolari M, Alcala R, Herrera F (2014) A multi-objective evolutionary method for learning granularities based on fuzzy discretization to improve the accuracy-complexity trade-off of fuzzy rule-based classification systems: d-MOFARC algorithm. Appl Soft Comput 24:470–481

    Article  Google Scholar 

  16. Rudzinski F (2016) A multi-objective genetic optimization of interpretability-oriented fuzzy rule-based classifiers. Appl Soft Comput 38:118–133

    Article  Google Scholar 

  17. Borgi A, Bazin JM, Akdag H (1998) Two methods of linear correlation search for a knowledge based supervised classification. In: 11th international conference on industrial and engineering applications of artificial intelligence and expert systems, IEA/AIE-98, Benicassim, Espagne, juin 1998. Lecture Notes in Computer Science, vol 1415/1998. Springer, Berlin , pp 696–707

    Google Scholar 

  18. Soua B, Borgi A, Tagina M (2013) An ensemble method for fuzzy rule-based classification systems. Knowl Inf Syst 36:385–410

    Article  Google Scholar 

  19. Ben Slima I, Borgi A (2015) Attributes regrouping by association rules in the fuzzy inference systems, “Regroupement d’attributs par règles d’association dans les systèmes d’inférence floue”, EGC 2015, vol RNTI-e-28, pp 317-328, Luxembourg

  20. Ishibuchi H, Nozaki K, Yamamoto N, Tanaka H (1994) Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms. Fuzzy Set Syst 65:237–253

    Article  MathSciNet  Google Scholar 

  21. Ishibuchi H, Nozaki K, Tanaka H (1993) Efficient fuzzy partition of pattern space for classification problems. Fuzzy Set Syst 59:295–304

    Article  Google Scholar 

  22. Wang XZ, Yeung DS, Tsang EC (2001) A comparative study on heuristic algorithms for generating fuzzy decision trees. IEEE Trans Syst Man Cybern Part B (Cybernetics) 31(2):215–226

    Article  Google Scholar 

  23. Pal NR, Chakraborty S (2001) Fuzzy rule extraction from ID3-type decision trees for real data. IEEE Trans Syst Man Cybern Part B (Cybernetics) 31(5):745–754

    Article  Google Scholar 

  24. Hefny HA, Ghiduk AS, Wahab AA, Elashiry M (2010) Effective method for extracting rules from fuzzy decision trees based on ambiguity and classifiability. Universal J Comput Sci Eng Technol 1(1):55–63

    Google Scholar 

  25. Mitra S, Hayashi Y (2000) Neuro-fuzzy rule generation: survey in soft computing framework. IEEE Trans Neural Netw 11(3):748–768

    Article  Google Scholar 

  26. Jang J-SR (1993) ANFIS: Adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23 (3):665–685

    Article  Google Scholar 

  27. Jang J-SR, Sun CT, Mizutani E (1997) Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence. Prentice Hall, pp 333–393

  28. Chen T, Shang C, Su P, Shen Q (2018) Induction of accurate and interpretable fuzzy rules from preliminary crisp representation. Knowl-Based Syst 146:152–166

    Article  Google Scholar 

  29. Pota M, Esposito M, De Pietro G (2017) Designing rule-based fuzzy systems for classification in medicine. Knowl-Based Syst 124:105–132

    Article  Google Scholar 

  30. Ishibuchi H, Nakashima T, Morisawa T (1999) Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets Syst 103(2):223–238

    Article  Google Scholar 

  31. Ishibuchi H, Yamamoto T, Nakashima T (2005) Hybridization of fuzzy GBML approaches for pattern classification problems. IEEE Trans Syst Man Cybern Part B 35(2):359–365

    Article  Google Scholar 

  32. Ishibuchi H, Yamamoto T (2004) Comparison of heuristic criteria for fuzzy rule selection in classification problems. Fuzzy Optim Decis Making 3:119–139

    Article  MathSciNet  Google Scholar 

  33. Mansoori EG, Zolghadri MJ, Katebi SD (2008) SGERD: A steady-state genetic algorithm for extracting fuzzy classification rules from data. IEEE Trans Fuzzy Syst 16(4):1061–1071

    Article  Google Scholar 

  34. Mansoori EG, Shafiee KS (2016) On fuzzy feature selection in designing fuzzy classifiers for high-dimensional data. Evolv Syst 7(4):255–265

    Article  Google Scholar 

  35. Hu YC, Chen RS, Tzeng GH (2003) Finding fuzzy classification rules using data mining techniques. Pattern Recogn Lett 24(1):509–519

    Article  Google Scholar 

  36. Chen Z, Chen G (2008) Building an associative classifier based on fuzzy association rules. Int J Comput Intel Syst 1(3):262–273

    Article  Google Scholar 

  37. Alcala-Fdez J, Alcala R, Herrera F (2011) A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning. IEEE Trans Fuzzy Syst 19(5):857–872

    Article  Google Scholar 

  38. Ishibuchi H, Yamamoto T (2005) Rule weight specification in fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 13(4):428–435

    Article  Google Scholar 

  39. Ren Y, Zhang L, Suganthan PN (2016) Ensemble classification and regression-recent developments, applications and future directions. IEEE Comput Intell Mag 11(1):41–53

    Article  Google Scholar 

  40. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MathSciNet  MATH  Google Scholar 

  41. Freund Y (1995) Boosting a weak learning algorithm by majority. Inf Comput 121:256–285

    Article  MathSciNet  Google Scholar 

  42. Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. In: Proceedings of international conference on machine learning, pp 148–156

  43. Cordon O, Quirin A, Sanchez L (2008) A first study on bagging fuzzy rule-based classiffication systems with multicriteria genetic selection of the component classiffiers. In: 3rd international workshop on genetic and evolving fuzzy systems. Witten-Bommerholz, Germany

  44. Skurichina M, Duin R (2002) Bagging, boosting and the random subspace method for linear classifiers. Pattern Anal Applic 5:121–135

    Article  MathSciNet  Google Scholar 

  45. Ho T (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20:832– 844

    Article  Google Scholar 

  46. Amouzadi A, Mirzaei A (2010) Hierarchical fuzzy rule-based classification system by evolutionary boosting algorithm. In: 5th international symposium on telecommunications (IST’2010). IEEE, pp 909–913

  47. Del Jesus MJ, Haffmann F, Navascués LJ, Snchez L (2004) Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms. IEEE Trans Fuzzy Syst 12:296–308

    Article  Google Scholar 

  48. Hoffmann F (2001) Boosting a genetic fuzzy classifier. In: IFSA world cogress and 20th NAFIPS international conference 2001, joint 9th. Vancouver, pp 1564–1569

  49. Cao J, Wang H, Kwong S, Li K (2011) Combining interpretable fuzzy rule-based classifiers via multi-objective hierarchical evolutionary algorithm. In: IEEE international conference on systems, man, and cybernetics (SMC), pp 1771–1776

  50. Nguyen TT, Liew AWC, To C, Pham XC, Nguyen MP (2014) Fuzzy if-then rules classifier on ensemble data. In: International conference on machine learning and cybernetics. Springer, Berlin, pp 362–370

    Google Scholar 

  51. Bharati K, Kanchan D (2016) Comparative study of frequent itemset mining algorithms: FP growth, FIN, prepost + and study of efficiency in terms of memory consumption, scalability and runtime. Int J Tech Res Appl 4:72–77

    Google Scholar 

  52. Agrawal R, Imielinski T, Swami A (1993) Mining association rules between sets of items in large databases. In: ACM SIGMOD conference. Washington DC, pp 207–216

  53. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. In: 20th VLDB Conference, Santiago, Chile, pp 487–499

  54. Gan W, Lin JCW, Fournier-Viger P, Chao HC, Fujita H (2018) Extracting non-redundant correlated purchase behaviors by utility measure. Knowl-Based Syst 143:30–41

    Article  Google Scholar 

  55. Lin JCW, Fournier-Viger P, Gan W (2016) FHN: an efficient algorithm for mining high-utility itemsets with negative unit profits. Knowl-Based Syst 111:283–298

    Article  Google Scholar 

  56. Yun U, Ryang H, Lee G, Fujita H (2017) An efficient algorithm for mining high utility patterns from incremental databases with one database scan. Knowl-Based Syst 124:188–206

    Article  Google Scholar 

  57. Yun U, Kim D, Yoon E, Fujita H (2018) Damped window based high average utility pattern mining over data streams. Knowl-Based Syst 144:188–205

    Article  Google Scholar 

  58. Hong TP, Lee CH, Wang SL (2009) Mining high average-utility itemsets. The Int Conf Syst Man Cybern, 2526–2530

  59. Grahne G, Zhu J (2005) Fast algorithms for frequent itemset mining using fp-trees. IEEE Trans Knowl Data Eng 17:1347– 1362

    Article  Google Scholar 

  60. Ansari E, Dastghaibifard GH, keshatkaran M (2008) Distributed Trie-based frequent itemset mining. In: Proceedings of international multiConference of engineers and computer scientists, vol 1, pp 978–988

  61. Han J, Pei J, Yin Y (2000) Mining frequent patterns without candidate generation. ACM SIGMOD Record 29:1–12

    Article  Google Scholar 

  62. Deng Z, Wang Z, Jiang J (2012) A new algorithm for fast mining frequent itemsets using N-lists. Sci China Inform Sci 55(9):2008–2030

    Article  MathSciNet  Google Scholar 

  63. Deng ZH, Lv SL (2014) Fast mining frequent itemsets using nodesets. Expert Syst Appl 41(10):4505–4512

    Article  Google Scholar 

  64. Agrawal R, Gehrke JE, Gunopulos D, Raghavan P (1999) Automatic subspace clustering of high dimensional data for data mining applications, U.S. Patent No. 6,003,029. washington, DC: U.S Patent and Trademark Office

  65. Shannon CE (1948) A mathematical theory of communication, Part i, Part II. Bell Syst Tech J 27:623–656

    Article  Google Scholar 

  66. Breiman L, Friedman J, Stone CJ, Olshen R (1984) Classification and regression trees. CRC Press, Boca Raton

    MATH  Google Scholar 

  67. Renyi A (1961) On measures of entropy and information. In: Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, vol 1: contributions to the theory of statistics. The Regents of the University of California

  68. Stephane T (2012) Data Mining et statistique décisionnelle: L’intelligence des données. Éditions Technip

  69. Dougherty J, Kohavi R, Sahami M (1995) Supervised and unsupervised discretization of continuous features. Mach Learn: Proc Twelfth Int Conf 12:194–202

    Google Scholar 

  70. Fayyad U, Irani K (1993) Multi-interval discretization of continuous valued attributes for classification learning. In: 13th international joint conference on artificial intelligence, Morgan Kaufmann. San Francisco, CA, pp 1022–1027

  71. Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: 14th international joint conference on artificial intelligence, Morgan Kaufmann. Canada, pp 1137–1143

  72. Marn-Blzquez JG, Shen Q (2002) From approximative to descriptive fuzzy classifiers. IEEE Trans Fuzzy Syst 10(4):484–497

    Article  Google Scholar 

  73. Hhn J, Hllermeier E (2009) Furia: an algorithm for unordered fuzzy rule induction. Data Mining Knowl Discovery 19(3):293–319

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculté des Sciences de Tunis, LR11ES14 LIPAH, University of Tunis El Manar, Campus El Manar, 2092, Tunis, Tunisia

    Ilef Ben Slima

  2. Institut Supérieur d’Informatique, LR11ES14 LIPAH, University of Tunis El Manar, Campus El Manar, 2092, Tunis, Tunisia

    Amel Borgi

Authors
  1. Ilef Ben Slima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amel Borgi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilef Ben Slima.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ben Slima, I., Borgi, A. Supervised methods for regrouping attributes in fuzzy rule-based classification systems. Appl Intell 48, 4577–4593 (2018). https://doi.org/10.1007/s10489-018-1224-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1224-0

Keywords

Search

Navigation