Skip to main content
Springer Nature Link
Log in

Rule extraction from neural network trained using deep belief network and back propagation

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Representing the knowledge learned by neural networks in the form of interpretable rules is a prudent technique to justify the decisions made by neural networks. Heretofore many algorithms exist to extract symbolic rules from neural networks, but among them, a few extract rules from deep neural networks trained using deep learning techniques. So, this paper proposes an algorithm to extract rules from a multi-hidden layer neural network, pre-trained using deep belief network and fine-tuned using back propagation. The algorithm analyzes each node of a layer and extracts knowledge from each layer separately. The process of knowledge extraction from the first hidden layer is different from the other layers. Consecutively, the algorithm combines all the knowledge extracted and refines them to construct a final ruleset consisting of symbolic rules. The algorithm further subdivides the subspace of a rule in the ruleset if it satisfies certain conditions. Results show that the algorithm extracted rules with higher accuracy compared to some existing rule extraction algorithms. Other than accuracy, the efficacy of the extracted rules is also validated with fidelity and various other performance measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Cohen S, Rokach L, Maimon O (2007) Decision-tree instance-space decomposition with grouped gain ratio. Inf Sci 177(17):3592–3612

    Google Scholar 

  2. Yager R (2006) An extension of the nave Bayesian classifier. Inf Sci 176(5):577–588

    Google Scholar 

  3. Kaikhah K, Doddmeti S (2006) Discovering trends in large datasets using neural network. Appl Intell 24(1):51–60

    Google Scholar 

  4. Dam H, Abbass A, Lokan C, Yao X (2008) Neural based learning classifier systems. IEEE Trans Knowl Data Eng 20(1):26–39

    MATH  Google Scholar 

  5. Andrews R, Diederich J, Tickle AB (1995) Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl Based Syst 8(6):373–389

    MATH  Google Scholar 

  6. Setiono R (1997) Extracting rules from neural networks by pruning and hidden-unit splitting. Neural Comput 9(1):205–225

    MATH  Google Scholar 

  7. Hua Y, Guo J and Zhao H (2015) Deep belief networks and deep learning. In: International conference on intelligent computing and internet of things (ICIT).IEEE

  8. Larochelle H, Bengio Y, Louradour J, Lamblin P (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10(1):1–40

    MATH  Google Scholar 

  9. Hinton G, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    MathSciNet  MATH  Google Scholar 

  10. Towel G, Shavli J (1993) The extraction of refined rules from knowledge based neural networks. Mach Learn 13(1):71–101

    Google Scholar 

  11. Setiono R, Liu H (1996) Symbolic representation of neural networks. IEEE Comput 29(3):71–77

    Google Scholar 

  12. Odajimaa K, Hayashi Y, Tianxia G, Setiono R (2008) Greedy rule generation from discrete data and its use in neural network rule extraction. Neural Networks 21(7):1020–1028

    MATH  Google Scholar 

  13. Setiono R, Liu H (1997) NeuroLinear: From neural networks to oblique decision rules. Neurocomputing 17(1):1–24

    Google Scholar 

  14. Sestito S, Dillon T(1992) Automated knowledge acquisition of rules with continuously valued attributes. In: Proceedings of 12th international conference on expert systems and their applications, pp 645–656

  15. Taha IA, Ghosh J (1999) Symbolic interpretation of artificial neural networks. IEEE Trans Knowl Data Eng 11(3):448–463

    Google Scholar 

  16. Etchells TA, Lisboa PJG (2006) Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach. IEEE Trans Neural Netw 17(2):374–384

    Google Scholar 

  17. Augusta MG, Kathirvalavakumar T (2012) Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems. Neural Process Lett 35(2):131–150

    Google Scholar 

  18. Biswas SK, Chakraborty M, Purkayastha B, Thounaojam DM, Roy P (2017) Rule extraction from training data using neural network. Int J Artif Intell Tool 26(3):1750006

    Google Scholar 

  19. Craven M, Shavlik J (1996) Extracting tree-structured representations of trained network. In: Advances in neural information processing systems (NIPS), vol 8. MIT Press, Cambridge, pp 24–30

  20. Anbananthen SK, Sainarayanan G, Chekima A, Teo J (2006) Data mining using pruned artificial neural network tree (ANNT). Inf Commun Technol 1:1350–1356

    Google Scholar 

  21. Setiono R, Baesens B, Mues C (2008) Recursive neural network rule extraction for data with mixed attributes. IEEE Trans Neural Networks 19(2):299–307

    Google Scholar 

  22. Hara A, Hayashi Y (2012) Ensemble neural network rule extraction using Re-RX algorithm. In: The 2012 international joint conference on neural networks (IJCNN), pp 1–6

  23. Chakraborty M, Biswas SK, Purkayastha B (2018) Recursive rule extraction from NN using reverse engineering technique. New Gener Comput 36(2):119–142

    Google Scholar 

  24. Hayashi Y, Nakano S, Fujisawa S (2015) Use of the recursive-rule extraction algorithm with continuous attributes to improve diagnostic accuracy in thyroid disease. Inform Med Unlocked 1:1–8

    Google Scholar 

  25. Hayashi Y (2016) Application of a rule extraction algorithm family based on the Re-RX algorithm to financial credit risk assessment from a Pareto optimal perspective. Oper Res Perspect 3:32–42

    MathSciNet  Google Scholar 

  26. Hayashi Y, Yukita S (2016) Rule extraction using Recursive-Rule extraction algorithm with J48graft combined with sampling selection techniques for the diagnosis of type 2 diabetes mellitus in the Pima Indian dataset. Inform Med Unlocked 2:92–104

    Google Scholar 

  27. Rudy S, Kheng W (2000) FERNN: an algorithm for fast extraction of rules from neural networks. Appl Intell 12(1):15–25

    Google Scholar 

  28. Iqbal RA (2012) Eclectic rule extraction from Neural Networks using aggregated Decision Trees. In: 7th international conference on electrical and computer engineering (ICECE). IEEE, pp 129–132

  29. Zilke JR, Loza ME, Janssen F (2016) DeepRED—rule extraction from deep neural networks. In: Calders T, Ceci M, Malerba D (eds) Discovery science. DS 2016. Lecture notes in computer science, vol 9956. Springer, Cham

  30. Bondarenko A, Aleksejeva L, Jumutc V, Borisov A (2017) Classification tree extraction from trained artificial neural networks. Procedia Comput Sci 104:556–563

    Google Scholar 

  31. Fortuny EJD, Martens D (2015) Active learning-based pedagogical rule extraction. IEEE Trans Neural Netw Learn Syst 26(11):2664–2677

    MathSciNet  Google Scholar 

  32. Hailesilassie T (2016) Rule Extraction Algorithm for deep neural networks: a review. IJCSIS Int J Comput Sci Inf Secur 14(7):371–381

    Google Scholar 

  33. Tran SN, Garcez A (2013) Knowledge extraction from deep belief networks for images. In: Workshop on neural-symbolic learning and reasoning

  34. Bologna G, Hayashi Y (2016) A rule extraction study on a neural network trained by deep learning. In: International joint conference on neural networks (IJCNN), Vancouver, BC, Canada, pp 668–675

  35. Tran SN, Garcez A (2016) Deep logic networks: inserting and extracting knowledge from deep belief networks. IEEE Trans Neural Netw Learn Syst 1(99):1–13

    Google Scholar 

  36. Biswas SK, Chakraborty M, Singh HR, Devi D, Purkayastha B, Das AK (2016) Hybrid case-based reasoning system by cost-sensitive neural network for classification. Soft Comput 21:7579–7596

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, NIT Silchar, Silchar, Assam, 788010, India

    Manomita Chakraborty, Saroj Kumar Biswas & Biswajit Purkayastha

Authors
  1. Manomita Chakraborty
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Saroj Kumar Biswas
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Biswajit Purkayastha
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Manomita Chakraborty.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborty, M., Biswas, S.K. & Purkayastha, B. Rule extraction from neural network trained using deep belief network and back propagation. Knowl Inf Syst 62, 3753–3781 (2020). https://doi.org/10.1007/s10115-020-01473-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-020-01473-0

Keywords