Abstract
One advantage of evolutionary multiobjective optimization (EMO) algorithms over classical approaches is that many non-dominated solutions can be simultaneously obtained by their single run. This paper shows how this advantage can be utilized in genetic rule selection for the design of fuzzy rulebased classification systems. Our genetic rule selection is a two-stage approach. In the first stage, a pre-specified number of candidate rules are extracted from numerical data using a data mining technique. In the second stage, an EMO algorithm is used for finding non-dominated rule sets with respect to three objectives: to maximize the number of correctly classified training patterns, to minimize the number of rules, and to minimize the total rule length. Since the first objective is measured on training patterns, the evolution of rule sets tends to overfit to training patterns. The question is whether the other two objectives work as a safeguard against the overfitting. In this paper, we examine the effect of the three-objective formulation on the generalization ability (i.e., classification rates on test patterns) of obtained rule sets through computer simulations where many non-dominated rule sets are generated using an EMO algorithm for a number of high-dimensional pattern classification problems.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., and Verkamo, A. I.: Fast Discovery of Association Rules, in Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P., and Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, AAAI Press, Metro Park (1996) 307–328.
Casillas, J., Cordon, O., Herrera, F., and Magdalena, L. (Eds.): Accuracy Improvement in Linguistic Fuzzy Modelling, Physica-Verlag (2003 in press).
Castillo, L., Gonzalez, A., and Perez, R.: Including a Simplicity Criterion in the Selection of the Best Rule in a Genetic Fuzzy Learning Algorithm, Fuzzy Sets and Systems 120 (2001) 309–321.
Cordon, O., Del Jesus, M. J., and Herrera, F.: A Proposal on Reasoning Methods in Fuzzy Rule-Based Classification Systems, International Journal of Approximate Reasoning 20 (1999) 21–45.
Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T.: A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II, IEEE Trans. on Evolutionary Computation 6 (2002) 182–197.
Elomaa, T., and Rousu, J.: General and Efficient Multisplitting of Numerical Attributes, Machine Learning 36 (1999) 201–244.
Gonzalez, A., and Perez, R.: SLAVE: A Genetic Learning System Based on an Iterative Approach, IEEE Trans. on Fuzzy Systems 7 (1999) 176–191.
Hong, T.-P., Kuo, C.-S., and Chi, S.-C.: Trade-off between Computation Time and Number of Rules for Fuzzy Mining from Quantitative Data, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 9 (2001) 587–604.
Ishibuchi, H., Murata, T., and Turksen, I. B.: Single-Objective and Two-Objective Genetic Algorithms for Selecting Linguistic Rules for Pattern Classification Problems, Fuzzy Sets and Systems 89 (1997) 135–149.
Ishibuchi, H., and Nakashima, T.: Effect of Rule Weights in Fuzzy Rule-Based Classification Systems, IEEE Trans. on Fuzzy Systems 9 (2001) 506–515.
Ishibuchi, H., Nakashima, T., and Morisawa, T.: Voting in Fuzzy Rule-Based Systems for Pattern Classification Problems, Fuzzy Sets and Systems 103 (1999) 223–238.
Ishibuchi, H., Nakashima, T., and Murata, T.: Three-Objective Genetics-Based Machine Learning for Linguistic Rule Extraction, Information Sciences 136 (2001) 109–133.
Ishibuchi, H., Nozaki, K., and Tanaka, H.: Distributed Representation of Fuzzy Rules and Its Application to Pattern Classification, Fuzzy Sets and Systems 52 (1992) 21–32.
Ishibuchi, H., Nozaki, K., Yamamoto, N., and Tanaka, H.: Construction of Fuzzy Classification Systems with Rectangular Fuzzy Rules Using Genetic Algorithms, Fuzzy Sets and Systems 65 (1994) 237–253.
Ishibuchi, H., Nozaki, K., Yamamoto, N., and Tanaka, H.: Selecting Fuzzy If-Then Rules for Classification Problems Using Genetic Algorithms, IEEE Trans. on Fuzzy Systems 3 (1995) 260–270.
Ishibuchi, H., and Yamamoto, T.: Fuzzy Rule Selection by Data Mining Criteria and Genetic Algorithms, Proc. of Genetic and Evolutionary Computation Conference (2002) 399–406.
Ishibuchi, H., and Yamamoto, T.: Comparison of Heuristic Rule Weight Specification Methods, Proc. of 11th IEEE International Conference on Fuzzy Systems (2002) 908–913.
Ishibuchi, H., and Yamamoto, T.: Comparison of Fuzzy Rule Selection Criteria for Classification Problems, in A. Abraham et al. (eds.): Soft Computing Systems: Design, Management and Applications (Frontiers in Artificial Intelligence and Applications, Volume 87), pp. 132–141, IOS Press, 2002.
Ishibuchi, H., Yamamoto, T., and Nakashima, T.: Fuzzy Data Mining: Effect of Fuzzy Discretization, Proc. of 1st IEEE International Conference on Data Mining (2001) 241–248.
Jin, Y.: Fuzzy Modeling of High-dimensional Systems: Complexity Reduction and Interpretability Improvement, IEEE Trans. on Fuzzy Systems 8 (2000) 212–221.
Jin, Y., Von Seelen, W., and Sendhoff, B.: On Generating FC3 Fuzzy Rule Systems from Data Using Evolution Strategies, IEEE Trans. on Systems, Man and Cybernetics — Part B: Cybernetics 29 (1999) 829–845.
Knowles, J. D., and Corne, D. W.: Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy, Evolutionary Computation 8 (2000) 149–172.
Kuncheva, L. I.: Fuzzy Classifier Design, Physica-Verlag, Heidelberg (2000).
Murata, T., and Ishibuchi, H.: MOGA: Multi-Objective Genetic Algorithms, Proc. of 2nd IEEE International Conference on Evolutionary Computation (1995) 289–294.
Quinlan, J. R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993).
Roubos, H., and Setnes, M.: Compact and Transparent Fuzzy Models and Classifiers Through Iterative Complexity Reduction, IEEE Trans. on Fuzzy Systems 9 (2001) 516–524.
Sanchez, L., Couso, I., and Corrales, J. A.: Combining GA Operators with SA Search to Evolve Fuzzy Rule Base Classifiers, Information Sciences 136 (2001) 175–191.
Setnes, M., Babuska, R., Kaymak, U., and Van Nauta Lemke, H. R.: Similarity Measures in Fuzzy Rule Base Simplification, IEEE Trans. on Systems, Man, and Cybernetics — Part B: Cybernetics 28 (1998) 376–386.
Setnes, M., Babuska, R., and Verbruggen, B.: Rule-based Modeling: Precision and Transparency, IEEE Trans. on Systems, Man, and Cybernetics — Part C: Applications and Reviews 28 (1998) 165–169.
Van den Berg, J., Kaymak, U., and Van den Bergh, W.-M.: Fuzzy Classification Using Probability Based Rule Weighting, Proc. of 11th IEEE International Conference on Fuzzy Systems (2002) 991–996.
Weiss, S. M., and Kulikowski, C. A.: Computer Systems That Learn, Morgan Kaufmann Publishers, San Mateo (1991).
Zitzler, E., and Thiele, L.: Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach, IEEE Trans. on Evolutionary Computation 3 (1999) 257–271.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ishibuchi, H., Yamamoto, T. (2003). Effects of Three-Objective Genetic Rule Selection on the Generalization Ability of Fuzzy Rule-Based Systems. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Thiele, L., Deb, K. (eds) Evolutionary Multi-Criterion Optimization. EMO 2003. Lecture Notes in Computer Science, vol 2632. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36970-8_43
Download citation
DOI: https://doi.org/10.1007/3-540-36970-8_43
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-01869-8
Online ISBN: 978-3-540-36970-7
eBook Packages: Springer Book Archive