Skip to main content

Advertisement

SpringerLink
Log in

Surrogate-Assisted Multi-objective Genetic Algorithms for Fuzzy Rule-Based Classification

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

In this paper, we present a surrogate-assisted multi-objective genetic algorithm to mine a small number of linguistically interpretable fuzzy rules for high-dimensional classification task in the realm of data mining. However, the difficulties like (1) handling of high-dimensional problems by fuzzy rule-based systems (i.e., the exponential increase in the number of fuzzy rules with the number of input variables), (2) the deterioration in the comprehensibility of fuzzy rules when they involve many antecedent conditions, and (3) the optimization of multiple objectives in fuzzy rule-based system may stand as pertinent issues. Hence, to combat with the aforesaid issues, we design the problem as a combinatorial optimization problem with three objectives: to maximize the number of correctly classified training patterns, minimize the number of fuzzy rules, and minimize the total number of antecedent conditions. We optimize these objectives of the fuzzy rule-based system by using a multi-objective genetic algorithm. Further to reduce the number of required evaluations to approximate the Pareto front of computationally expensive multi-objective genetic algorithm for fuzzy rule-based system, a radial basis neural network surrogate model is adapted. This approach searches for non-dominated rule sets with respect to these three objectives. The performance of the surrogate-assisted model is evaluated through a few benchmark datasets obtained from knowledge extraction based on evolutionary learning data repository. The experimental outcome confirm that this model is competitive compared to the non-surrogate-assisted model. However, the performance of the model has drawn a clear edge over rule mining approaches like Decision Table, JRip, OneR, PART, and ZeroR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Witten, I.H., Frank, E., Hall, M.A., Pal, C.: Data Mining: Practical Machine Learning Tools and Techniques, 4th edn. Morgan Kaufmann, Burlington (2016)

    Google Scholar 

  2. Weiss, S.M., Kapouleas, I.: An empirical comparison of pattern recognition, neural nets and machine learning classification methods. In: Readings in Machine Learning, pp. 177–183. Morgan Kaufmann Publishers Inc., Burlington (1990)

  3. Lim, T.S., Loh, W.Y., Shih, Y.S.: A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms. Mach. Learn. 40(3), 203–228 (2000)

    Article  MATH  Google Scholar 

  4. Freitas, A.A.: A survey of evolutionary algorithms for data mining and knowledge discovery. In: Ghosh, A., Tsutsui, S. (eds.) Advances in Evolutionary Computation. Springer, Berlin (2001)

    Google Scholar 

  5. Smith, E., Dike, B.A., Stegmann, S.A.: Fitness inheritance in genetic algorithms. In: Proceedings of the 1995 ACM Symposium on Applied Computing, pp. 345–350. ACM Press, New York (1995)

  6. Ishibuchi, H., Nakasima, T.: Linguistic rule extraction by genetic-based machine learning. In: Proceedings of Genetic and Evolutionary Computation Conference, pp. 195–202. Morgan Kaufmann (2000)

  7. Ishibuchi, H., Murata, T., Turksen, I.B.: Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets Syst. 89(2), 135–150 (1997)

    Article  Google Scholar 

  8. Zimmermann, H.J.: Fuzzy Set Theory and Its Applications. Springer, Berlin (2011)

    Google Scholar 

  9. Kramer, O.: Genetic Algorithm Essentials. Studies in Computational Intelligence, vol. 679. Springer, Cham (2017)

    MATH  Google Scholar 

  10. Fernandez, A., Garcia, S., Luengo, J., Bernado-Mansilla, E., Herrera, F.: Genetics-based machine learning for rule induction: state of art, taxonomy and comparative study. IEEE Trans. Evolut. Comput. 14(6), 913–943 (2010)

    Article  Google Scholar 

  11. Herrera, F.: Genetic fuzzy systems: taxonomy, current research trends and prospects. Evolut. Intell. 1(1), 27–46 (2008)

    Article  Google Scholar 

  12. Smith, S.F.: A learning system based on genetic algorithms. Ph.D. dissertation, University of Pittsburg, Pittsburg PA (1980)

  13. Ong, Y.S., Nair, P.B., Keane, A.J.: Evolutionary optimization of computationally expensive problems via surrogate modelling. Am. Inst. Aeronaut. Astronaut. J. 41(4), 687–696 (2003)

    Article  Google Scholar 

  14. Jin, Y., Olhofer, M., Sendhoff, B.: A framework for evolutionary optimization with approximate fitness functions. IEEE Trans. Evolut. Comput. 6(5), 481–494 (2002)

    Article  Google Scholar 

  15. Shi, L., Rasheed, K.: A survey of fitness approximation methods applied in evolutionary algorithms. In: Goh, C.K., Tenne, Y. (eds.) Computational Intelligence in Expensive Optimization Problem, pp. 3–28. Springer, Berlin (2010)

    Chapter  Google Scholar 

  16. Eby, D., Averill, R., Panch, W., Goohman, E .: Evaluation of injection island model GA performance on flywheel design optimization. In: Third Conference on Adaptive Computing in Design and Manufacturing, pp. 121–136. Springer (1998)

  17. Sefrioui, M., Periaux, J.: A hierarchical genetic algorithm using multiple models for optimization. In: Parallel Problem Solving From Nature, PPSN VI, pp. 879-888. Springer, Berlin (2000)

  18. Abbond, K., Schoenauer, M.: Surrogate Deterministic Mutation. Artificial Evolution, vol. 2310, pp. 103–115. Springer, Berlin (2002)

    Google Scholar 

  19. Rasheed, K.: Informed operators: speeding up genetic-algorithm-based design optimization using reduced models. In: Proceedings of Genetic and Evolutionary Computation Conference, pp. 628–635. Morgan Kaufmann, Las Vegas (2000)

  20. Papadrakakis, M., Lagaros, N., Tsompanakis, Y.: Optimization of large-scale 3D trusses using evolutionary strategies and neural networks. Int. J. Space Struct. 14(3), 211–223 (1997)

    Article  Google Scholar 

  21. Emmerich, M., Giotis, A., Ozdenir, M., Back, T., Giannakoglou, K.: Meta-model-assisted evolution strategies. In: International Conference on parallel problem solving from nature, pp. 361–370. Springer, Berlin (2002)

  22. Jin, Y.: Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement. IEEE Trans. Fuzzy Syst. 8, 212–222 (2000)

    Article  Google Scholar 

  23. Ishibuchi, H., Nakashima, T., Morisawa, T.: Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets Syst. 103, 223–238 (1999)

    Article  Google Scholar 

  24. Ishibuchi, H., Nozaki, K., Yamamoto, N., Tanaka, H.: Selecting fuzzy if-then rules for classification problem using genetic algorithms. IEEE Trans. Fuzzy Syzt. 3(3), 260–270 (1995)

    Article  Google Scholar 

  25. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. Evolut. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  26. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: improving the performance of the strength of Pareto evolutionary algorithm. Technical Report 103, Computer Engineering and Communication Networks Lab, Swiss Federal Institute of Technology, Zurich (2001)

  27. Watanabe, S., Hiroyassu, T., Miki, M.: Multi-objective rectangular packing problem and its applications. In: Proceedings of 2nd International Conference on Multi-criterion Optimization, pp. 565–577 (2003)

  28. Ghosh, A., Das, M.K.: Non-dominated rank based sorting genetic algorithms. Fundam. Inf. 83(3), 231–252 (2008)

    MathSciNet  MATH  Google Scholar 

  29. Sanchez, L., Otero, J., Couso, I.: Obtaining linguistic fuzzy rule-based regression models from imprecise data with multiobjective genetic algorithms. Soft Comput. 13(5), 467–479 (2009)

    Article  MATH  Google Scholar 

  30. Ishibuchi, H., Kuwajima, I., Nojima, Y.: Use of Pareto-optimal and near Pareto-optimal candidate rules in genetic fuzzy rule selection. In: Melin, P., et al. (eds.) Analysis and Design of Intelligent Systems Using Soft Computing Techniques, pp. 387–396. Springer, Berlin (2007)

    Chapter  Google Scholar 

  31. Ji-lin, C., Yuan-long, H., Zong-yi, X., Li-min, J., Zhong-zhi, T.: A multi-objective genetic-based method for design fuzzy classification systems. Int. J. Comput. Sci. Netw. Secur. 6(8A), 110–117 (2006)

    Google Scholar 

  32. Zhang, Y., Wu, X.B., Xing, Z.Y., Hu, W.L.: On generating interpretable and precise fuzzy systems based on Pareto multi-objective cooperative co-evolutionary algorithm. Appl. Soft Comput. 11(1), 1284–1294 (2011)

    Article  Google Scholar 

  33. Nojima, Y., Kaisho, Y., Ishibuchi, H.: Accuracy improvement of genetic fuzzy rule selection with candidate rule addition and membership tuning. In: Proceedings of IEEE International Conference on Fuzzy Systems, Barcelona, pp. 1–8 (2010)

  34. Alcalá, R., Nojima, Y., Herrera, F., Ishibuchi, H.: Generating single granularity-based fuzzy classification rules for multi-objective genetic fuzzy rule selection. In: Proceedings of IEEE International Conference on Fuzzy Systems, pp. 1718–1723 (2009)

  35. Nojima, Y., Ishibuchi, H.: Incorporation of user preference into multi-objective genetic fuzzy rule selection for pattern classification problems. Artif. Life Robot. 14(3), 418–421 (2009)

    Article  Google Scholar 

  36. Ishibuchi, H., Nakashima, Y., Nojima, Y.: Performance evaluation of evolutionary multiobjective optimization algorithms for multi-objective fuzzy genetics-based machine learning. Soft Comput. 15(12), 2415–2434 (2011)

    Article  Google Scholar 

  37. Alcalá, R., Nojima, Y., Herrera, F., Ishibuchi, H.: Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions. Soft Comput. 15(12), 2303–2318 (2011)

    Article  Google Scholar 

  38. Trawiski, K., Cordon, O., Quirin, A.: A study on the use of multiobjective genetic algorithms for classifier selection in FURIA-based fuzzy multiclassifiers. Int. J. Comput. Intell. Syst. 5(2), 231–253 (2012)

    Article  Google Scholar 

  39. Fazzolari, M., Giglio, B., Alcala, R., Marcelloni, F., Herrera, F.: A study on the application of instance selection techniques in genetic fuzzy rule-based classification systems: accuracy-complexity trade-off. Knowl. Based Syst. 54, 32–41 (2013)

    Article  Google Scholar 

  40. Fazzolari, M., Alcala, R., Herrera, F.: A multi-objective evolutionary method for learning granularities based on fuzzy discretization to improve the accuracy-complexity trade-off of fuzzy rule-based classification systems: D-MOFARC algorithm. Appl. Soft Comput. 24, 470–481 (2014)

    Article  Google Scholar 

  41. Antonelli, M., Ducange, P., Marcelloni, F.: A fast and efficient multi-objective evolutionary learning scheme for fuzzy rule-based classifiers. Inf. Sci. 283(1), 36–54 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Gorzaczany, M.B., Filip, R.: A multi-objective genetic optimization for fast, fuzzy rule-based credit classification with balanced accuracy and interpretability. Appl. Soft Comput. 40, 206–220 (2016)

    Article  Google Scholar 

  43. Antonelli, M., Ducange, P., Lazzerini, B., Marcelloni, F.: Multi-objective evolutionary design of granular rule-based classifiers. Granul. Comput. 1(1), 37–58 (2016)

    Article  Google Scholar 

  44. Rudziski, F.: A multi-objective genetic optimization of interpretability-oriented fuzzy rule-based classifiers. Appl. Soft Comput. 38, 118–133 (2016)

    Article  Google Scholar 

  45. Antonelli, M., Ducange, P., Lazzerini, B., Marcelloni, F.: Learning knowledge bases of multi-objective evolutionary fuzzy systems by simultaneously optimizing accuracy, complexity and partition integrity. Soft Comput. 15(12), 2335–2354 (2011)

    Article  Google Scholar 

  46. Gacto, M.J., Alcala, R., Herrera, H.: A multi-objective evolutionary algorithm for tuning fuzzy rule-based systems for preserving interpretability. In: Proceedings of the Joint International Fuzzy Systems Association World Congress and the European Society for Fuzzy Logic and Technology Conference (IFSA/EUSFLAT 2009), pp. 1146–1151 (2009)

Download references

Acknowledgements

This work was partly supported by Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (R0124-16-0002), Emotional Intelligence Technology to Infer Human Emotion and Carry on Dialogue Accordingly.

Author information

Authors and Affiliations

  1. Department of Computer Science, Seemanta Engineering College, Mayurbhanj, Odisha, 757086, India

    Harihar Kalia

  2. Department of Information and Communication Technology, Fakir Mohan University, Balasore, Odisha, 756019, India

    Satchidananda Dehuri

  3. Machine Intelligence Unit and Center for Soft Computing Research, Indian Statistical Institute, 203 B.T. Road, Kolkata, 700108, India

    Ashish Ghosh

  4. Department of Computer Science, Yonsei University, Soft Computing Laboratory, 134 Shinchon-dong, Sudaemoon-gu, Seoul, 120-749, South Korea

    Sung-Bae Cho

Authors
  1. Harihar Kalia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satchidananda Dehuri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ashish Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sung-Bae Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sung-Bae Cho.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kalia, H., Dehuri, S., Ghosh, A. et al. Surrogate-Assisted Multi-objective Genetic Algorithms for Fuzzy Rule-Based Classification. Int. J. Fuzzy Syst. 20, 1938–1955 (2018). https://doi.org/10.1007/s40815-018-0478-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-018-0478-3

Keywords

Search

Navigation