Skip to main content
SpringerLink
Log in

Rule extraction: Using neural networks or for neural networks?

  • Knowledge and Data Processing
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In the research of rule extraction from neural networks,fidelity describes how well the rules mimic the behavior of a neural network whileaccuracy describes how well the rules can be generalized. This paper identifies thefidelity-accuracy dilemma. It argues to distinguishrule extraction using neural networks andrule extraction for neural networks according to their different goals, where fidelity and accuracy should be excluded from the rule quality evaluation framework, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andrews R, Diederich J, Tickle A B. Survey and critique of techniques for extracting rules from trained artificial neural networks.Knowledge-Based Systems, 1995, 8(6): 373–389.

    Article  Google Scholar 

  2. Tickle A B, Andrews R, Golea Met al. The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks.IEEE Trans. Neural Networks, 1998, 9(6): 1057–1067.

    Article  Google Scholar 

  3. Gallant S I. Connectionist expert systems.Communications of the ACM, 1988, 31(2): 152–169.

    Article  Google Scholar 

  4. Fu L. Rule learning by searching on adapted nets. InProc. the 9th National Conference on Artificial Intelligence, Anaheim, CA, 1991, pp.590–595.

  5. Thrun S. Extracting rules from artificial neural networks with distributed representations. InAdvances in Neural Information Processing Systems 7, Tesauro G, Touretzky D, Leen T (Eds.), Cambridge, MA, MIT Press, 1995, pp.505–512.

    Google Scholar 

  6. Craven M W, Shavlik J W. Extracting tree-structured representations of trained networks. InAdvances in Neural Information Processing Systems 8, Touretzky D, Mozer M, Hasselmo M (Eds.), Cambridge, MA, MIT Press, 1996, pp.24–30.

    Google Scholar 

  7. Krishnan R. A systematic method for decompositional rule extraction from neural networks. InProc. the NIPS'96 Workshop on Rule Extraction from Trained Artificial Neural Networks, Queensland, Australia, 1997, pp.38–45.

  8. Setiono R. Extracting rules from neural networks by pruning and hidden-unit splitting.Neural Computation, 1997, 9(1): 205–225.

    Article  MATH  Google Scholar 

  9. Zhou Z H, Chen S F, Chen Z Q. A statistics based approach for extracting priority rules from trained neural networks. InProc. the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Como, Italy, 2000, 3: 401–406.

  10. Masuoka R, Watanabe N, Kawamura Aet al. Neurofuzzy systems_— Fuzzy inference using a structured neural network. InProc. the International Conference on Fuzzy Logic and Neural Networks, Iizuka, Japan, 1990, pp.173–177.

  11. Mitra S. Fuzzy MLP based expert system for medical diagnosis.Fuzzy Sets and Systems, 1994, 65(2–3): 285–296.

    Article  Google Scholar 

  12. Castro J L, Mantas C J, Benitez J M. Interpretation of artificial neural networks by means of fuzzy rules.IEEE Trans. Neural Networks, 2002, 13(1): 101–116.

    Article  Google Scholar 

  13. Giles C L, Miller C B, Chen Det al. Learning and extracting finite state automata with second-order recurrent neural networks.Neural Computation, 1992, 4(3): 393–405.

    Article  Google Scholar 

  14. Omlin C W, Giles C L, Miller C B. Heuristics for the extraction of rules from discrete time recurrent neural networks. InProc. the International Joint Conference on Neural Networks, Baltimore, MD, 1992, Vol. 1, pp.33–38.

  15. Giles C L, Omlin C W. Extraction, insertion, and refinement of symbolic rules in dynamically driven recurrent networks.Connection Science, 1993, 5(3–4): 307–328.

    Article  Google Scholar 

  16. Saito K, Nakano R. Extracting regression rules from neural networks.Neural Networks, 2002, 15(10): 1279–1288.

    Article  Google Scholar 

  17. Setiono R, Leow W K, Zurada J M. Extraction of rules from artificial neural networks for nonlinear regression.IEEE Trans. Neural Networks, 2002, 13(3): 564–577.

    Article  Google Scholar 

  18. Tickle A B, Orlowski M, Diederich J. DEDEC: A methodology for extracting rule from trained artificial neural networks. InProc. the AISB'96 Workshop on Rule Extraction from Trained Neural Networks, Brighton, UK, 1996, pp.90–102.

  19. Craven M W, Shavlik J W. Using sampling and queries to extract rules from trained neural networks. InProc. the 11th Int. Conf. Machine Learning, New Brunswick, NJ, 1994, pp.37–45.

  20. Golea M. On the complexity of rule extraction from neural networks and network querying. InProc. the AISB'96 Workshop on Rule Extraction from Trained Neural Networks, Brighton, UK, 1996, pp.51–59.

  21. Roy A. On connectionism, rule extraction, and brainlike learning.IEEE Trans. Fuzzy Systems, 2000, 8(2): 222–227.

    Article  Google Scholar 

  22. Duch W, Adamczak R, Grabczewski K. A new methodology of extraction, optimization and application of crisp and fuzzy logical rules.IEEE Trans. Neural Networks, 2001, 12(2): 277–306.

    Article  Google Scholar 

  23. Zhou Z H, Jiang Y, Chen S F. Extracting symbolic rules from trained neural network ensembles.AI Communications, 2003, 16(1): 3–15.

    Google Scholar 

  24. Towell G, Shavlik J. The extraction of refined rules from knowledge based neural networks.Machine Learning, 1993, 13(1): 71–101.

    Google Scholar 

  25. Fu L. Rule generation from neural networks.IEEE Trans. Systems, Man and Cybernetics, 1994, 24(8): 1114–1124.

    Article  Google Scholar 

  26. Craven M W, Shavlik J W. Extracting comprehensible concept representations from trained neural networks. InWorking Notes on the IJCAI'95 Workshop on Comprehensibility in Machine Learning, Montreal, Canada, 1995, pp.61–75.

  27. Taha I A, Ghosh J. Symbolic interpretation of artificial neural networks.IEEE Trans. Knowledge and Data Engineering, 1999, 11(3): 448–463.

    Article  Google Scholar 

  28. Setiono R. Extracting M-of-N rules from trained neural networks.IEEE Trans. Neural Networks, 2000, 11(2): 512–519.

    Article  Google Scholar 

  29. Wolpert D H, Macready W G. No free lunch theorems for optimization.IEEE Trans. Evolutionary Computation, 1997, 1(1): 67–82.

    Article  Google Scholar 

  30. Quinlan J R. Comparing connectionist and symbolic learning methods. InComputational Learning Theory and Natural Learning Systems, Rivest R L (Ed.), Vol. 1, Cambridge, MA, MIT Press, 1994, pp.445–456.

    Google Scholar 

  31. Chalup S, Hayward R, Diederich J. Rule extraction from artificial neural networks trained on elementary number classification tasks. InProc. the 9th Australian Conference on Neural Networks, Brisbane, Australia, 1998, pp.265–270.

  32. Maire F. Rule-extraction by backpropagation of polyhedra.Neural Networks, 1999, 12(4–5): 717–725.

    Article  Google Scholar 

  33. Bologna G. Rule extraction from a multi layer perceptron with staircase activation functions. InProc. the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Como, Italy, 2000, 3: 419–424.

  34. Vahed A, Omlin C W. Rule extraction from recurrent neural networks using a symbolic machine learning algorithm. InProc. the 6th International Conference on Neural Information Processing, Dunedin, New Zealand, 1999, pp.712–717.

Download references

Author information

Authors and Affiliations

  1. National Laboratory for Novel Software Technology, Nanjing University, 210093, Nanjing, P.R. China

    Zhi-Hua Zhou

Authors
  1. Zhi-Hua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi-Hua Zhou.

Additional information

This work was supported by the National Outstanding Youth Foundation of China under Grant No.60325237 and the National Natural Science Foundation of China under Grant No.60273033.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhou, ZH. Rule extraction: Using neural networks or for neural networks?. J. Comput. Sci. & Technol. 19, 249–253 (2004). https://doi.org/10.1007/BF02944803

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02944803

Keywords

Search

Navigation