Skip to main content

Evolutionary Multiobjective Design of Fuzzy Rule-Based Classifiers

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 115))

The main goal in classifier design has been accuracy maximization on unseen patterns [23]. A number of learning algorithms have been proposed to minimize the classification errors on training patterns in various fields such as neural networks [84], fuzzy systems [76] and machine learning [81]. It is well-known that neural networks are universal approximators of nonlinear functions [34, 35]. Since fuzzy systems are also universal approximators [70, 73, 92], we can design fuzzy rule-based classifiers that can correctly classify all training patterns. Such a fuzzy rule-based classifier, however, does not usually have high accuracy on test patterns, as shown in Fig. 1, where a typical accuracy-complexity tradeoff relation is depicted. We can decrease the error rate of classifiers on training patterns by increasing their complexity (for example, by increasing the number of fuzzy rules in fuzzy rule-based classifiers), as shown by the dotted curve in Fig. 1. The classification accuracy on test patterns is, however, degraded by increasing the complexity too much, as shown by the solid curve in Fig. 1. Such an undesirable deterioration in the classification accuracy on test patterns is known as ‘overfitting to training patterns’ [23]. Finding the optimal complexity with the maximum accuracy on test patterns (that is, S * in Fig. 1) is one of the main research issues in machine learning, especially in the field of statistical learning theory [10].

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   389.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   499.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abe S 2001 Pattern Classification: Neuro-fuzzy Methods and Their Comparison. Springer-Verlag, Berlin.

    MATH  Google Scholar 

  2. Abe S, Lan M-S 1995 A method for fuzzy rules extraction directly from numerical data and its application to pattern classification. IEEE Trans. Fuzzy Systems, 3(1): 18-28.

    Article  MathSciNet  Google Scholar 

  3. Abe S, Thawonmas R 1997 A fuzzy classifier with ellipsoidal regions. IEEE Trans. Fuzzy Systems, 5(3): 358-368.

    Article  Google Scholar 

  4. Booker LB, Goldberg DE, Holland JH 1989 Classifier systems and genetic algorithms. Artificial Intelligence, 40(1-3): 235-282.

    Article  Google Scholar 

  5. Cano JR, Herrera F, Lozano M 2005 Stratification for scaling up evolutionary prototype selection. Pattern Recognition Letters, 26(7): 953-963.

    Article  Google Scholar 

  6. Cano JR, Herrera F, Lozano M 2006 On the combination of evolutionary algorithms and stratified strategies for training set selection in data mining. Applied Soft Computing, 6(3): 323-332.

    Article  Google Scholar 

  7. Casillas J, Cordon O, Herrera F, Magdalena L (eds.) 2003 Interpretability Issues in Fuzzy Modeling. Springer-Verlag, Berlin.

    MATH  Google Scholar 

  8. Casillas J, Cordon O, Herrera F, Magdalena L(eds.)(2003) Accuracy Improvements in Linguistic Fuzzy Modeling. Springer-Verlag, Berlin.

    MATH  Google Scholar 

  9. Castillo L, Gonzalez A, Perez R 2001 Including a simplicity criterion in the selection of the best rule in a genetic fuzzy learning algorithm. Fuzzy Sets and Systems, 120(2): 309-321.

    Article  MATH  MathSciNet  Google Scholar 

  10. Cherkassky V, Mulier F 1998 Learning from Data: Concepts, Theory, and Methods. Wiley, New York, NY.

    MATH  Google Scholar 

  11. Coello CAC, Lamont GB 2004 Applications of Multi-Objective Evolutionary Algorithms. World Scientific, Singapore.

    MATH  Google Scholar 

  12. Coello CAC, van Veldhuizen DA, Lamont GB 2002 Evolutionary Algorithms for Solving Multi-Objective Problems. Kluwer Academic Publishers, Boston, MA.

    MATH  Google Scholar 

  13. Cordon O, del Jesus MJ, Herrera F 1999 A Proposal on reasoning methods in fuzzy rule-based classification systems. Intl. J. Approximate Reasoning, 20(1): 21-45.

    Google Scholar 

  14. Cordon O, del Jesus MJ, Herrera F, Lozano M 1999 MOGUL: a methodology to obtain genetic fuzzy rule-based systems under the iterative rule learning approach. Intl. J Intelligent Systems, 14(11): 1123-1153.

    Article  MATH  Google Scholar 

  15. Cordon O, Gomide F, Herrera F, Hoffmann F, Magdalena L 2004 Ten years of genetic fuzzy systems: current framework and new trends. Fuzzy Sets and Systems, 141(1): 5-31.

    Article  MATH  MathSciNet  Google Scholar 

  16. Cordon O, Herrera F, Hoffman F, Magdalena L 2001 Genetic Fuzzy Systems. World Scientific, Singapore.

    MATH  Google Scholar 

  17. Curry R, Heywood MI 2004 Towards efficient training on large datasets for genetic programming. Lecture Notes in Artificial Intelligence 3060: Advances in Artificial Intelligence - Canadian AI 2004. Springer-Verlag, Berlin: 161-174.

    Google Scholar 

  18. Deb K 2001 Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, Chichester, UK.

    MATH  Google Scholar 

  19. Deb K, Pratap A, Agarwal S, Meyarivan T 2002 A fast and elitist multiobjec-tive genetic algorithm: NSGA-II. IEEE Trans. Evolutionary Computation, 6(2): 182-197.

    Article  Google Scholar 

  20. de la Iglesia B, Philpott MS, Bagnall AJ, Rayward-Smith VJ 2003 Data mining rules using multi-objective evolutionary algorithms. In: Sarker R, Reynolds R, Abbass H, Tan KC, McKay R, Essam D, Gedeon T (eds.) Proc. 2003 Congress Evolutionary Computation. 8-12 December, Canberra, Australia. IEEE Press, Piscataway, NJ: 1552-1559.

    Chapter  Google Scholar 

  21. de la Iglesia B, Reynolds A, Rayward-Smith VJ 2005 Developments on a multi-objective metaheuristic (MOMH) algorithm for finding interesting sets of classification rules. Lecture Notes in Computer Science 3410: Evolutionary Multi-Criterion Optimization - EMO 2005. Springer-Verlag, Berlin: 826-840.

    Google Scholar 

  22. de la Iglesia B, Richards G, Philpott MS, Rayward-Smith VJ 2006 The appli-cation and effectiveness of a multi-objective metaheuristic algorithm for partial classification. European J. Operational Research, 169(3): 898-917.

    Article  MATH  Google Scholar 

  23. Duda O, Hart PE, Stork DG 2001 Pattern Classification. Wiley, New York, NY.

    MATH  Google Scholar 

  24. Freitas AA 2002 Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer-Verlag, Berlin.

    MATH  Google Scholar 

  25. Ghosh A, Nath BT 2004 Multi-objective rule mining using genetic algorithms. Information Sciences, 163(1-3): 123-133.

    Article  MathSciNet  Google Scholar 

  26. Goldberg DE 1989 Genetic Algorithms in Search, Optimization, and Machine Learning. Addison Wesley, Reading, MA.

    MATH  Google Scholar 

  27. Gonzalez A, Perez R 1999 SLAVE: A genetic learning system based on an iterative approach. IEEE Trans. Fuzzy Systems, 7(2): 176-191.

    Article  Google Scholar 

  28. Guillaume S 2001 Designing fuzzy inference systems from data: an interpretability-oriented review. IEEE Trans. Fuzzy Systems, 9(3): 426-443.

    Article  MathSciNet  Google Scholar 

  29. Herrera F 2005 Genetic fuzzy systems: status, critical considerations and future directions. Intl. J. Computational Intelligence Research, 1(1): 59-67.

    Google Scholar 

  30. Herrera F, Lozano M, Verdegay JL 1998 A learning process for fuzzy control rules using genetic algorithms. Fuzzy Sets and Systems, 100(1-3): 143-158.

    Article  Google Scholar 

  31. Holland JH 1975 Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, MI.

    Google Scholar 

  32. Hong T-P, Kuo C-S, Chi S-C 2001 Trade-off between computation time and number of rules for fuzzy mining from quantitative data. Intl. J. Uncertainty, Fuzziness and Knowledge-Based Systems, 9(5): 587-604.

    MATH  Google Scholar 

  33. Horikawa S, Furuhashi T, Uchikawa Y 1992 On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm. IEEE Trans. Neural Networks, 3(5): 801-806.

    Article  Google Scholar 

  34. Hornik K 1991 Approximation capabilities of multilayer feedforward networks. Neural Networks, 4(2): 251-257.

    Article  Google Scholar 

  35. Hornik K, Stinchcombe M, White H 1989 Multilayer feedforward networks are universal approximators. Neural Networks, 2(5): 359-366.

    Article  Google Scholar 

  36. Hu YC, Chen RS, Tzeng GH 2002 Mining fuzzy association rules for classification problems. Computers & Industrial Engineering 43(4): 735-750.

    Article  Google Scholar 

  37. Hu YC, Chen RS, Tzeng GH 2003 Finding fuzzy classification rules using data mining techniques. Pattern Recognition Letters, 24(1-3): 509-519.

    Article  MATH  Google Scholar 

  38. Hughes EJ, 2005 Evolutionary many-objective optimization: many once or one many? In: Corne D (ed.) Proc. 2005 Congress Evolutionary Computation 2-5 September, Edinburgh, UK. IEEE Press, Piscataway, NJ: 222-227.

    Chapter  Google Scholar 

  39. Ishibuchi H, Doi T, Nojima Y 2006 Incorporation of scalarizing fitness func-tions into evolutionary multiobjective optimization algorithms. Lecture Notes in Computer Science 4193: PPSN IX. Springer-Verlag, Berlin: 493-502.

    Google Scholar 

  40. Ishibuchi H, Kuwajima I, Nojima Y 2007 Relation between Pareto-optimal fuzzy rules and Pareto-optimal fuzzy rule sets. In: Bonissone P, Coello CAC, Jin Y (eds.) Proc. 1st IEEE Symp. Computational Intelligence in Multicriteria Decision Making 1-5 April, 2007, Honolulu, HI. IEEE Press, Piscataway, NJ: 42-49.

    Chapter  Google Scholar 

  41. Ishibuchi H, Murata T (1998) A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Systems, Man, and Cybernetics - Part C: Applications and Reviews, 28(3): 392-403.

    Article  Google Scholar 

  42. Ishibuchi H, Murata T, Turksen IB 1997 Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets and Systems, 89(2): 135-150.

    Article  Google Scholar 

  43. Ishibuchi H, Nakashima T 1999 Improving the performance of fuzzy classifier systems for pattern classification problems with continuous attributes. IEEE Trans. Industrial Electronics 46(6): 157-168.

    Article  Google Scholar 

  44. Ishibuchi H, Nakashima T 2001 Effect of rule weights in fuzzy rule-based classification systems. IEEE Trans. Fuzzy Systems, 9(4): 506-515.

    Article  Google Scholar 

  45. Ishibuchi H, Nakashima T, Morisawa T 1999 Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets and Systems, 103(2): 223-238.

    Article  Google Scholar 

  46. Ishibuchi H, Nakashima T, Murata T 1999 Performance evaluation of fuzzy classifier systems for multi-dimensional pattern classification problems. IEEE Trans. Systems, Man, and Cybernetics - Part B: Cybernetics, 29(5): 601-618.

    Article  Google Scholar 

  47. Ishibuchi H, Nakashima T, Murata T 2001 Three-objective genetics-based machine learning for linguistic rule extraction. Information Sciences, 136(1-4): 109-133.

    Article  MATH  Google Scholar 

  48. Ishibuchi H, Nakashima T, Nii M 2004 Classification and Modeling with Linguistic Information Granules: Advanced Approaches to Linguistic Data Mining. Springer-Verlag, Berlin.

    Google Scholar 

  49. Ishibuchi H, Namba S 2004 Evolutionary Multiobjective Knowledge Extrac-tion for High-dimensional Pattern Classification Problems. Lecture Notes in Computer Science 3242: Parallel Problem Solving from Nature - PPSN VIII Springer, Berlin: 1123-1132.

    Google Scholar 

  50. Ishibuchi H, Nojima Y 2007 Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning. Intl. J. Approximate Reasoning, 44(1): 4-31.

    Article  MATH  MathSciNet  Google Scholar 

  51. Ishibuchi H, Nozaki K, Tanaka H 1992 Distributed representation of fuzzy rules and its application to pattern classification. Fuzzy Sets and Systems, 52(1): 21-32.

    Article  Google Scholar 

  52. Ishibuchi H, Nozaki K, Yamamoto N, Tanaka H 1995 Selecting fuzzy if-then rules for classification problems using genetic algorithms. IEEE Trans. Fuzzy Systems, 3(3): 260-270.

    Article  Google Scholar 

  53. Ishibuchi H, Shibata Y 2004 Mating scheme for controlling the diversity-convergence balance for multiobjective optimization. Lecture Notes in Com-puter Science 3102: Genetic and Evolutionary Computation - GECCO 2004. Springer-Verlag, Berlin: 1259-1271.

    Google Scholar 

  54. Ishibuchi H, Yamamoto T 2002 Effect of fuzzy discretization in fuzzy rule-based systems for classification problems with continuous attributes. Archives of Control Sciences, 12(4): 351-378.

    MATH  MathSciNet  Google Scholar 

  55. Ishibuchi H, Yamamoto T 2004 Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining. Fuzzy Sets and Systems, 141(1): 59-88.

    Article  MATH  MathSciNet  Google Scholar 

  56. Ishibuchi H, Yamamoto T 2005 Rule weight specification in fuzzy rule-based classification systems. IEEE Trans. Fuzzy Systems, 13(4): 428-435.

    Article  Google Scholar 

  57. Ishibuchi H, Yamamoto T, Nakashima T 2001 Fuzzy data mining: effect of fuzzy discretization. In: Cercone N, Lin TY, Wu X (eds.) Proc. 1st IEEE Intl. Conf. Data Mining 29 November - 2 December, San Jose, CA. IEEE Computer Society, Los Alamitos, CA: 241-248.

    Google Scholar 

  58. Ishibuchi H, Yamamoto T, Nakashima T 2005 Hybridization of fuzzy GBML approaches for pattern classification problems. IEEE Trans. Systems, Man, and Cybernetics - Part B: Cybernetics, 35(2): 359-365.

    Article  Google Scholar 

  59. Ishibuchi H, Yoshida T, Murata T 2003 Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. Evolutionary Computation, 7(2): 204-223.

    Article  Google Scholar 

  60. Jang J-S R 1993 ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans. Systems, Man, and Cybernetics, 23(3): 665-685.

    Article  MathSciNet  Google Scholar 

  61. Jaszkiewicz A 2002 Genetic local search for multi-objective combinatorial optimization. European J. Operational Research, 137(1): 50-71.

    Article  MATH  MathSciNet  Google Scholar 

  62. Jaszkiewicz A 2004 On the computational efficiency of multiple objective metaheuristics: the knapsack problem case study. European J. Operational Research, 158(2): 418-433.

    Article  MATH  MathSciNet  Google Scholar 

  63. Jimenez F, Gomez-Skarmeta AF, Sanchez G, Roubos H, Babuska R (2001) Accurate, transparent and compact fuzzy models for function approxima-tion and dynamic modeling through multi-objective evolutionary optimiza-tion. Lecture Notes in Computer Science 1993: Evolutionary Multi-Criterion Optimization - EMO 2001. Springer-Verlag, Berlin: 653-667.

    Google Scholar 

  64. Jin Y 2000 Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement. IEEE Trans. Fuzzy Systems, 8(2): 212-221.

    Article  Google Scholar 

  65. Jin Y (ed.) 2006 Multi-Objective Machine Learning. Springer-Verlag, Berlin.

    MATH  Google Scholar 

  66. Jin Y, von Seelen W, Sendhoff B 1999 On generating FC3 fuzzy rule sys-tems from data using evolution strategies. IEEE Trans. Systems, Man, and Cybernetics - Part B: Cybernetics, 29(6): 829-845.

    Article  Google Scholar 

  67. Karr CL, Gentry EJ 1993 Fuzzy control of pH using genetic algorithms. IEEE Trans. Fuzzy Systems, 1(1): 46-53.

    Article  Google Scholar 

  68. Kaya M 2006 Multi-objective genetic algorithm based approaches for mining optimized fuzzy association rules. Soft Computing, 10(7): 578-586.

    Article  MATH  MathSciNet  Google Scholar 

  69. Knowles JD, Corne DW 2000 Approximating the nondominated front using Pareto archived evolution strategy Evolutionary Computation, 8(2): 149-172.

    Google Scholar 

  70. Kosko B 1992 Fuzzy systems as universal approximators. In: Bezdek J (ed.) Proc. IEEE Intl. Conf. Fuzzy Systems. 8-12 March, San Diego, CA. IEEE Press, Piscataway, NJ: 1153-1162.

    Chapter  Google Scholar 

  71. Kuncheva LI 2000 Fuzzy Classifier Design. Physica-Verlag, Heidelberg.

    MATH  Google Scholar 

  72. Kuncheva LI 2000 How good are fuzzy if-then classifiers? IEEE Trans. Systems, Man, and Cybernetics - Part B: Cybernetics, 30(4): 501-509.

    Article  Google Scholar 

  73. Mendel JM 1995 Fuzzy-logic systems for engineering - a tutorial. Proc. IEEE, 83(3): 345-377.

    Article  Google Scholar 

  74. Miettinen K 1998 Nonlinear Multiobjective Optimization. Kluwer, Boston, MA.

    Google Scholar 

  75. Mikut R, Jakel J, Groll L 2005 Interpretability issues in data-based learning of fuzzy systems. Fuzzy Sets and Systems, 150(2): 179-197.

    Article  MATH  MathSciNet  Google Scholar 

  76. Nauck D, Klawonn F, Kruse R 1997 Foundations of Neuro-Fuzzy Systems. Wiley, Chichester, UK.

    Google Scholar 

  77. Nauck D, Kruse R 1997 A neuro-fuzzy method to learn fuzzy classification rules from data. Fuzzy Sets and Systems, 89(3): 277-288.

    Article  MathSciNet  Google Scholar 

  78. Nauck D, Kruse R 1998 How the learning of rule weights affects the inter-pretability of fuzzy systems. In: Keller J (ed.) Proc. 7th IEEE Intl. Conf. Fuzzy Systems 4-9 May, Anchorage, AK. IEEE Press, Piscataway, NJ: 1235-1240.

    Google Scholar 

  79. Nauck D, Kruse R 1999 Obtaining interpretable fuzzy classification rules from medical data. Artificial Intelligence in Medicine, 16(2): 149-169.

    Article  MathSciNet  Google Scholar 

  80. Nozaki K, Ishibuchi H, Tanaka H 1996 Adaptive fuzzy rule-based classification systems. IEEE Trans. Fuzzy Systems, 4(3): 238-250.

    Article  Google Scholar 

  81. Quinlan JR 1993 C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  82. Reynolds A, de la Iglesia B 2006 Rule induction using multi-objective meta-heuristics: encouraging rule diversity. In: Wang L (ed.) Proc. 2006 Intl. Joint Conf. Neural Networks 16-21 July, British Columbia, Canada. IEEE Press, Piscataway, NJ: 6375-6382.

    Google Scholar 

  83. Roubos H, Setnes M 2001 Compact and transparent fuzzy models and classi-fiers through iterative complexity reduction. IEEE Trans. Fuzzy Systems, 9(4): 516-524.

    Article  Google Scholar 

  84. Rumelhart DE, McClelland JL, the PDP Research Group (1986) Parallel Distributed Processing. MIT Press, Cambridge, MA.

    Google Scholar 

  85. Schaffer JD 1985 Multiple objective optimization with vector evaluated genetic algorithms. In: Grefenstette J (ed.) Proc. 1st Intl. Conf. Genetic Algorithms and Their Applications. 24-26 July, Pittsburgh, PA. Lawrence Erlbaum Associates, Hillsdale, NJ: 93-100.

    Google Scholar 

  86. Setnes M, Babuska R, Verbruggen B 1998 Rule-based modeling: precision and transparency. IEEE Trans. Systems, Man, and Cybernetics - Part C: Applications and Reviews, 28(1): 165-169.

    Article  Google Scholar 

  87. Setnes M, Roubos H 2000 GA-based modeling and classification: complexity and performance. IEEE Trans. Fuzzy Systems, 8(5): 509-522.

    Article  Google Scholar 

  88. Smith SF 1980 A Learning System based on Genetic Algorithms. PhD Dissertation Department of Computer Science, University of Pittsburgh, PA.

    Google Scholar 

  89. van den Berg J, Kaymak U, van den Bergh WM 2002 Fuzzy classification using probability based rule weighting. In: Fogel DB (ed.) Proc. 11th IEEE Intl. Conf. Fuzzy Systems 12-17 May, Honolulu, HI. IEEE Press, Piscataway, NJ: 991-996.

    Google Scholar 

  90. Wang H, Kwong S, Jin Y, Wei W, Man KF 2005 Agent-based evolution-ary approach for interpretable rule-based knowledge extraction. IEEE Trans. Systems, Man, and Cybernetics - Part C: Applications and Reviews, 35(2): 143-155.

    Article  Google Scholar 

  91. Wang H, Kwong S, Jin Y, Wei W, Man KF 2005 Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction. Fuzzy Sets and Systems, 149(1): 149-186.

    Article  MATH  MathSciNet  Google Scholar 

  92. Wang LX, Mendel JM 1992 Fuzzy basis functions, universal approximation, and orthogonal least-squares learning. IEEE Trans. Neural Networks,3(5): 807-814.

    Article  Google Scholar 

  93. Wang LX, Mendel JM 1992 Generating fuzzy rules by learning from examples. IEEE Trans. Systems, Man, and Cybernetics 22(6): 1414-1427.

    Article  MathSciNet  Google Scholar 

  94. Zitzler E, Thiele L 1999 Multiobjective evolutionary algorithms: a compara-tive case study and the strength Pareto approach. IEEE Trans. Evolutionary Computation, 3(4): 257-271.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ishibuchi, H., Nojima, Y., Kuwajima, I. (2008). Evolutionary Multiobjective Design of Fuzzy Rule-Based Classifiers. In: Fulcher, J., Jain, L.C. (eds) Computational Intelligence: A Compendium. Studies in Computational Intelligence, vol 115. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78293-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78293-3_15

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78292-6

  • Online ISBN: 978-3-540-78293-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics