Abstract
The interpretation of neural network responses as symbolic rules is actually a difficult task. Our first approach consists in characterising the discriminant hyper-plane frontiers. More particularly, we point out that the shape of a discriminant frontier built by a standard multi-layer perceptron is related to an equation with two terms. The first one is linear, and the second is logarithmic.
This equation is not sufficient to easily generate symbolic rules. So, we introduce the Discretized Interpretable Multi Layer Perceptron (DIMLP) model that is a more constrained multi-layer architecture. From this special network, rules are extracted in polynomial time and continuous attributes do not need to be binary transformed.
We apply DIMLP to three applications of the public domain in which it gives better average predictive accuracy than C4.5 algorithm and discuss rule quality.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Andrews, R., Diederich, J., Tickle, A.B.: Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks. Knowledge-Based Systems 8(6), 373–389 (1995)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Quinlan, J.R.: Comparing Connectionist and Symbolic Learning Methods. In: Rivest, R. (ed.) Computational Learning Theory and Natural Learning, pp. 445–456 (1994)
Breiman, L., Friedmann, J.H., Olshen, R.A., Stone, J.: Classification and Regression Trees. Wadsworth and Brooks, Monterey, California (1984)
Maire, F.: Rule Extraction by Backpropagation of Polyhedra. Neural Networks 12(4–5), 717–725 (1999)
Bologna, G.: Rule Extraction from the IMLP Neural Network: a Comparative Study. In: Proceedings of the Workshop of Rule Extraction from Trained Articial Neural Networks (after the Neural Information Processing Conference), pp. 13–19 (1996)
Rudel, R., Sangiovanni-Vincentelli, A.: Espresso-MV: Algorithms for Multiple- Valued Logic Minimisation. In: Proceedings of the Custom International Circuit Conference (CICC 1985), Portland, pp. 230–234 (1985)
Corwin, E., Logar, A., Oldham, W.: An Iterative Method for Training Multi-layer Networks with Threshold Functions. IEEE Transactions on Neural Networks Journal 5(3), 507–508 (1994)
Aarts, E.H.L., Laarhoven, P.J.M.: Simulated Annealing: Theory and Applications. Kluwer Academic Publishers, Dordrecht (1987)
Thrun, S.B.: The Monk’s Problems: a Performance Comparison of Different Learning Algorithms. Technical Report, Carnegie Mellon University, CMU-CS-91-197 (1991)
Gorman, R.P., Sejnowski, T.J.: Analysis of Hidden Units in a Layered Network Trained to Classify Sonar Targets. Neural Networks 1(1), 75–88 (1988)
Alimoglu, F., Alpaydin, E.: Combining Multiple Representations and Classifiers for Pen-based Handwritten Digit Recognition. In: Proceedings of the Fourth International Conference on Document Analysis and Recognition (ICDAR 1997), Ulm, Germany (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bologna, G. (2000). Symbolic Rule Extraction from the DIMLP Neural Network. In: Wermter, S., Sun, R. (eds) Hybrid Neural Systems. Hybrid Neural Systems 1998. Lecture Notes in Computer Science(), vol 1778. Springer, Berlin, Heidelberg. https://doi.org/10.1007/10719871_17
Download citation
DOI: https://doi.org/10.1007/10719871_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67305-7
Online ISBN: 978-3-540-46417-4
eBook Packages: Springer Book Archive