Skip to main content
SpringerLink
Log in

Designing a decompositional rule extraction algorithm for neural networks with bound decomposition tree

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The neural networks are successfully applied to many applications in different domains. However, due to the results made by the neural networks are difficult to explain the decision process of neural networks is supposed as a black box. The explanation of reasoning is important to some applications such like credit approval application and medical diagnosing software. Therefore, the rule extraction algorithm is becoming more and more important in explaining the extracted rules from the neural networks. In this paper, a decompositional algorithm is analyzed and designed to extract rules from neural networks. The algorithm is simple but efficient; can reduce the extracted rules but improve the efficiency of the algorithm at the same time. Moreover, the algorithm is compared to the other two algorithms, M-of-N and Garcez, by solving the MONK’s problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Tickle AB, Andrews R, Golea M, Diederich J (1968) The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks. IEEE Trans Neural Netw 9(6):1057–1068

    Article  Google Scholar 

  2. d’Avila Garcez AS, Broda K, Gabbay DM (2001) Symbolic knowledge extraction from trained neural networks: a sound approach. Artif Intell 125:155–207

    Article  MATH  MathSciNet  Google Scholar 

  3. Pop E, Hayward R, Diedrich A (1994) RULENEG: extracting rules from a trained neural network using stepwise negation, Technical Report, (Neurocomputing Research Centre, Queensland University of Technology, 1994)

  4. Towell GG, Shavlik JW (1993) The extraction of refined rules from knowledge-based neural networks. Mach Learn 13:71–101

    Google Scholar 

  5. Towell GG, Shavlik JW (1994) Knowledge-based artificial neural networks. Artif Intell 70:119–165

    Article  MATH  Google Scholar 

  6. Tsukimoto H (2000) Extracting rules from trained neural networks. IEEE Trans Neural Netw 11(2):377–389

    Article  Google Scholar 

  7. Freeman JA, Skapura DM (1992) Neural networks. Addison-Wesley, Reading

  8. Fu LM (1994) Rule generation from neural networks. IEEE Trans Syst Man Cybern 28(8):1114–1124

    Google Scholar 

  9. Pratt LY, Mostow J, Kamm CA (1991) Direct transfer of learned information among neural networks. In: Proceedings of the 9th National Conference on Artificial Intelligence. Anaheim, pp 584–589

  10. Craven MW, Shavlik JW (1994) Using sampling and queries to extract rules from trained neural networks. In: Proceedings of the 11th International Conference on Machine Learning. New Brunswick, pp 37–45

  11. Jackson P (1999) Introduction to expert systems. Addison-Wesley, Reading

  12. Andrews R, Diederich J, Tickle AB (1995) Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl Base Syst 8(6):373–389

    Article  Google Scholar 

  13. Andrews R, Geva S (1995) Inserting and extracting knowledge from constrained error back propagation networks. In: Proceedings of the 6th Australian Conference on Neural Networks. Sydney

  14. Andrews R, Geva S (2002) Rule extraction from local cluster neural nets. Neuro Comput 47:1–20

    MATH  Google Scholar 

  15. Setiono R (1997) A penalty function approach for pruning feedforward neural networks. Neural Comput 9(1):185–204

    Article  MATH  Google Scholar 

  16. Setiono R (1997) Extracting rules from neural networks by pruning and hidden-unit splitting. Neural Comput 9(1):205–225

    Article  MATH  Google Scholar 

  17. Thrun SB, et al. (1991) The MONK’s problems: a performance comparison of different learning algorithms, Technical Report CMU-CS-91-197. Carnegie Mellon University

  18. Thrun SB (1994) Extracting provably correct rules from artificial neural networks, Technical Report IAI-TR-93-5. Institut für Informatik III, University of Bonn, Germany

    Google Scholar 

  19. Naturopathic association (1999) Naturopathy of coronary artery disease. Chinese edn. Ye-He Publishing

  20. Chen JC, Liu TS, Weng CS, Heh JS (2005) An expert system of coronary artery disease in Chinese and western medicine. In: Proceedings of the 6th Asian–Pacific conference on medical and biological engineering. Tsukuba

  21. Pham DT, Salem Z (2004) A new technique for rule pruning in machine learning, In: International conference on information and communication technologies: from theory to applications. Damascus, pp 437–438

  22. Chang M, Chen JC, Chang JW, Heh JS (2006) Advanced process control expert system of CVD membrane thickness based on neural network. Material Science Forum, pp 505–507, 313–318

  23. Krishnan R, Sivakumar G, Bhattacharya P (1999) A search technique for rule extraction from trained neural networks. Pattern Recognit Lett 20:273–280

    Article  MATH  Google Scholar 

  24. Taha IA, Ghosh J (1999) Symbolic interpretation of artificial neural networks. IEEE Trans Knowl Data Eng 11(3):448–463

    Article  Google Scholar 

  25. Chen JC, Heh JS, Chang M (2006) Designing A Decompositional Rule Extraction Algorithm for Neural Networks. Lect Notes Comput Sci 3971:1305–1311

    Google Scholar 

  26. Chang M, Chen JC, Heh J (2006) The control of membrane thickness in Pecvd process utilizing a rule extraction technique of neural networks. Lect Notes Comput Sci 3973:1091–1098

    Article  Google Scholar 

  27. Chen JC, Chang M, Chu KK (2006) Representing the abnormal Web access for advertisement counting based on the rule-extraction mechanism of neural networks. In: Proceedings of the 8th International conference on artificial intelligence and soft computing. Zakopane

  28. Takagi T, Sugeno M (1985) Fuzzy identification of system and its applications to modeling and control. IEEE Trans Syst Man Cybern 15(0):116–132

    MATH  Google Scholar 

  29. Roger Jang JS, Sun CT (1993) Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE Trans Neural Netw 4:156–158

    Article  Google Scholar 

  30. Leng G, McGinnity TM, Prasad G (2005) An approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network. Fuzzy Sets Syst 150:211–243

    Article  MATH  MathSciNet  Google Scholar 

  31. McGarry KJ, Wermter S, MacIntyre J (2001) Knowledge extraction from local function networks. In: Proceedings of the 17th international joint conference on artificial intelligence. Seattle

  32. Ultsch A, Halmans G, Mantyk R (1991) CONKAT: a connectionist knowledge acquisition tool. In: Proceedings of the 24th Hawaii IEEE International Conference on System Sciences, Hawaii

  33. Castellano G, Fanelli AM, Mencar C (2005) A neuro-fuzzy network to generate human-understandable knowledge from data. Cogn Syst Res 3:125–144

    Article  Google Scholar 

  34. Malone J, McGarry KJ, Bowerman C, Wermter S (2006) Rule extraction from Kohonen neural networks. Neural Comput Appl J 15(1):9–17

    Article  Google Scholar 

  35. Jeff Schlimmer (1984) United States congressional voting records database, Congressional Quarterly Almanac, 98th Congress, 2nd session 1984, vol XL. Congressional Quarterly Inc., Washington, DC

  36. Jeff Schlimmer (1987) Concept acquisition through representational adjustment, Doctoral dissertation. Department of Information and Computer Science, University of California, Irvine

  37. Jeff Schlimmer (1981) Mushroom records drawn from The Audubon Society field guide to north American mushrooms, In: Alfred A. Knopf (ed). G. H. Lincoff (Pres), New York

Download references

Author information

Authors and Affiliations

  1. Department of Information and Computer Engineering, Chung Yuan Christian University, 32099, Chung Li, Taiwan

    Jia Sheng Heh

  2. Department of Electronic Engineering, Chung Yuan Christian University, 32099, Chung Li, Taiwan

    Jen Cheng Chen

  3. National Science and Technology Program for e-Learning, P.O. Box 9-187, 32099, Chung-Li, Taiwan

    Maiga Chang

Authors
  1. Jia Sheng Heh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jen Cheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maiga Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jen Cheng Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Heh, J.S., Chen, J.C. & Chang, M. Designing a decompositional rule extraction algorithm for neural networks with bound decomposition tree. Neural Comput & Applic 17, 297–309 (2008). https://doi.org/10.1007/s00521-007-0115-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-007-0115-9

Keywords

Search

Navigation