Olcay Boz
ABSTRACT
Neural Networks are successful in acquiring hidden knowledge in datasets. Their biggest weakness is that the knowledge they acquire is represented in a form not understandable to humans. Researchers tried to address this problem by extracting rules from trained Neural Networks. Most of the proposed rule extraction methods required specialized type of Neural Networks; some required binary inputs and some were computationally expensive. Craven proposed extracting MofN type Decision Trees from Neural Networks. We believe MofN type Decision Trees are only good for MofN type problems and trees created for regular high dimensional real world problems may be very complex. In this paper, we introduced a new method for extracting regular C4.5 like Decision Trees from trained Neural Networks. We showed that the new method (DecText) is effective in extracting high fidelity trees from trained networks. We also introduced a new discretization technique to make DecText be able to handle continuous features and a new pruning technique for finding simplest tree with the highest fidelity.
- L. Atlas, R. Cole, J. Connor, M. El-Sharkawi, R. J. Marks, V. Muthusamy, and E. Barnard. Performance comparisons between backpropagation networks and classification trees on three real-world applications. In Advances in Neural Information Processing Systems, volume 2, pages 622--629, 1990. Google Scholar
Digital Library
- O. Boz. Converting A Trained Neural Network To A Decision Tree DecText - Decision Tree Etxractor. PhD thesis, Computer Science and Engineering, Lehigh University, 2000. Google ScholarDigital Library
- W. Buntine and T. Niblett. A further comparison of splitting rules for decision-tree induction. Machine Learning, 8:75, 1992. Google ScholarDigital Library
- M. W. Craven. Extracting Comprehensible Models from Trained Neural Networks. PhD thesis, Department of Computer Sciences, University of Wisconsin-Madison, 1996. (Also appears as UW Technical Report CS-TR-96-1326). Google ScholarDigital Library
- M. W. Craven and J. W. Shavlik. Extracting tree-structured representations of trained networks. In Advances in Neural Information Processing Systems, volume 8, pages 24--30, Denver, CO, 1996. MIT Press.Google Scholar
- D. Fisher and K. McKusick. An emprical comparison of id3 and back propagation. In Proceedings of the eleventh international Joint Conference on Artificial Intelligence, pages 788--793, Detroit, 1989. Morgan Kaufmann.Google Scholar
- P. Murphy and M. Pazzani. Id2-of-3: Constructive induction of n-of-m concepts for discriminators in decision trees. In Proceedings of the Eighth International Machine learning Workshop, pages 183--187, Evanston, IL, 1991. Morgan Kaufmann.Google ScholarCross Ref
- J. W. Shavlik, R. J. Mooney, and G. G. Towell. Symbolic and neural learning algorithms: An experimental comparison. Machine Learning, 6:111--143, 1991. Google ScholarDigital Library
- B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman & Hall, London, 1986.Google ScholarCross Ref
- S. M. Weiss and I. Kapouleas. An empirical comparison of pattern recognition, neural nets, and machine learning classification methods. In J. W. Shavlik and T. G. Dietterich, editors, Readings in Machine Learning. Morgan Kaufman, San Mateo, CA, 1990.Google Scholar
Index Terms
- Extracting decision trees from trained neural networks
Recommendations
Extracting symbolic rules from trained neural network ensembles
Artificial Intelligence Advances in ChinaNeural network ensemble can significantly improve the generalization ability of neural network based systems. However, its comprehensibility is even worse than that of a single neural network because it comprises a collection of individual neural ...
Extracting symbolic rules from trained neural network ensembles
Special issue on Artificial intelligence advances in ChinaNeural network ensemble can significantly improve the generalization ability of neural network based systems. However, its comprehensibility is even worse than that of a single neural network because it comprises a collection of individual neural ...
Extracting rules from trained neural networks
Presents an algorithm for extracting rules from trained neural networks. The algorithm is a decompositional approach which can be applied to any neural network whose output function is monotone such as a sigmoid function. Therefore, the algorithm can be ...
Comments