Rule extraction from neural networks is a powerful tool for knowledge discovery from data. In order to facilitate rule extraction, trained neural networks are often pruned so that the extracted rules are understandable to human users. This chapter presents a method for extracting interpretable rules from neural networks that are generated using an evolutionary multi-objective algorithm. In the algorithm, the accuracy on the training data and the complexity of the neural networks are minimized simultaneously. Since there is a tradeoff between accuracy and complexity, a number of Pareto-optimal neural networks, instead of one single optimal neural network, are obtained. We show that the Pareto-optimal networks with a minimal degree of complexity are often interpretable in that understandable logic rules can be extracted from them straightforwardly. The proposed approach is verified on two benchmark problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Andrew R, Diederich J, Tickle A (1995) A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge Based Systems 8(6):373–389
Jin Y, Sendhoff B (2003) Extracting interpretable fuzzy rules from RBF networks. Neural Processing Letters 17(2):149–164
Jin Y (2003) Advanced fuzzy systems design and applications. Springer, Heidelberg
Duch W, Setiono R, Zurada J (2004) Computational intelligence methods for rule-based data understanding. Proceedings of the IEEE 92(5):771–805
Kolman E, Margaliot M (2005) Are artificial neural networks white boxes? IEEE Transactions on Neural Networks 16(4):844–852
Etchell T A, Lisboa P J G (2006) Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach. IEEE Transactions on Neural Networks 17(2):374–38
Mitra S, Pal S K, Mitra P (2002) Data mining in soft computing framework: A survey. IEEE Transactions on Neural Networks 13(1):3–14
Jin Y, Seelen W V, Sendhoff B (1999) Generating FC3 fuzzy rule systems from data using evolution strategies. IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics 29(6):829–845
Jin Y (2000) Fuzzy modeling of high-dimensional systems: Complexity reduction and interpretability improvement. IEEE Transactions on Fuzzy Systems 8(2):212–221
Jin Y, Sendhoff B, Körner E (2005) Evolutionary multi-objective optimization for simultaneous generation of signal-type and symbol-type representations. The Third International Conference on Evolutionary Multi-Objective Optimization, 752–766
Jin Y, Sendhoff B, Körner E (2006) Simultaneous generation of accurate and interpretable neural network classifiers. In Y. Jin (eds) Multi-objective Machine Learning, 291–312
Jin Y (2006) (eds) Multi-objective machine learning. Springer, Berlin.
Bishop C M (1995) Neural networks for pattern recognition. Oxford University Press, Oxford, UK
Burnham K P, Anderson D R (2002) Model selection and multimodel inference. Springer, New York
Coello Coello C A, Veldhuizen D, Lamont G (2002) Evolutionary algorithms for solving multi-objective problems. Kluwer Academic, New York
Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley, Chichester
Deb K, Agrawal S, Pratap A, Meyarivan T (2000) A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In Parallel Problem Solving from Nature 6:849–858
Duch W, Adamczak R, Grabczewski K (1998) Extraction of logical rules from backpropagation networks. Neural Processing Letters 7:1–9
Fodor J A, Pylyshyn Z W (1998) Connectionism and cognitive architecture: A critical analysis. Cognition 28(3):3–71
Gabrieli J, Poldrack R, Desmond J (1998) The role of left prefrontal cortex in langrange and memory. Proceedings of the National Academy of Sciences 95:906–913
Hüsken M, Gayko J E, Sendhoff B (2000) Optimization for problem classes–Neural networks that learn to learn. In Yao X, Fogel D B (eds) IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (ECNN 2000), IEEE Press, 98–109
Igel C, Hüsken M (2000) Improving the Rprop learning algorithm. In Proceedings of the 2nd ICSC International Symposium on Neural Computation, 115–121
Ishibuchi H, Yamamoto T (2003) Evolutionary multiobjective optimization for generating an ensemble of fuzzy rule-based classifiers. In Proceedings of the Genetic and Evolutionary Computation Conference, 1077–188
Ishikawa M (2000) Rule extraction by successive regularization. Neural Networks 13:1171–1183
Jin Y, Okabe T, Sendhoff B (2004) Neural network regularization and ensembling using multi-objective evolutionary algorithms. In Congress on Evolutionary Computation, IEEE 1–8
Miller D A, Zurada J M (1998) A dynamical system perspective of structural learning with forgetting. IEEE Transactions on Neural Networks 9(3):508–515
Prechelt L (1994) PROBEN1 - a set of neural network benchmark problems and benchmarking rules. Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe
Reed R D, Marks II R J (1999) Neural smithing. The MIT Press
Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In IEEE International Conference on Neural Networks, 586–591
Setiono R (2000) Generating concise and accurate classification rules for breast cancer diagnosis. Artificial Intelligence in Medicine 18:205–219
Setiono R, Liu H (1996) Symbolic representation of neural networks. IEEE Computer 29(3):71–77
Taha I, Ghosh J (1999) Symbolic interpretation of artificial neural networks. IEEE Transactions on Knowledge and Data Engineering 11(3):448–463
Anderson E (1935) The rises of the gaspe peninsula. Bulletin of the American Iris Society 59:2–5
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Jin, Y., Sendhoff, B., Körner, E. (2008). Rule Extraction from Compact Pareto-optimal Neural Networks. In: Ghosh, A., Dehuri, S., Ghosh, S. (eds) Multi-Objective Evolutionary Algorithms for Knowledge Discovery from Databases. Studies in Computational Intelligence, vol 98. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77467-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-77467-9_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77466-2
Online ISBN: 978-3-540-77467-9
eBook Packages: EngineeringEngineering (R0)