Skip to main content
SpringerLink
Log in

Extraction of Logical Rules from Data by Means of Piecewise-Linear Neural Networks

  • Conference paper
  • First Online:
Book cover Discovery Science (DS 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2534))

Included in the following conference series:

  • 953 Accesses

Abstract

The extraction of logical rules from data by means of artificial neural networks is receiving increasingly much attention. The meaning the extracted rules may convey is primarily determined by the set of their possible truth values, according to which two basic kinds of rules can be differentiated - Boolean and fuzzy. Though a wide spectrum of theoretical principles has been proposed for ANN-based rule extraction, most of the existing methods still rely mainly on heuristics. Moreover, so far apparently no particular principles have been employed for the extraction of both kinds of rules, what can be a serious drawback when switching between Boolean and fuzzy rules. This paper presents a mathematically well founded approach based on piecewise-linear activation functions, which is suitable for the extraction of both kinds of rules. Basic properties of piecewise-linear neural networks are reviewed, most importantly, the replaceability of suboptimal computable mappings, and the preservation of polyhedra. Based on those results, a complete algorithm for the extraction of Boolean rules with that approach is given. In addition, two modifications of the algorithm are described, relying on different assumptions about the way how the properties of a polyhedron determine the decision to replace the polyhedron with a hyperrectangle. Finally, a biological application in which the presented approach has been successfully employed is briefly sketched.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. S. Aguzzoli and D. Mundici. Weierstrass approximations by Lukasiewicz formulas with one quantified variable. In 31st IEEE International Symposium on Multiple-Valued Logic, 2001.

    Google Scholar 

  2. R. Andrews, J. Diederich, and A.B. Tickle. Survey and critique of techniques for extracting rules from trained artificical neural networks. Knowledge Based Systems, 8:378–389, 1995.

    Article  Google Scholar 

  3. L.O. Cignoli, I.M.L. D'Ottaviano, and D. Mundici. Algebraic Foundations of Many-valued Reasoning. Kluwer Academic Publishers, Dordrecht, 2000.

    MATH  Google Scholar 

  4. J.E. Dennis and R.B. Schnabel. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice Hall, Englewood Cliffs, 1983.

    MATH  Google Scholar 

  5. W. Duch, R. Adamczak, and K. Grabczewski. Extraction of logical rules from neural networks. Neural Processing Letters, 7:211–219, 1998.

    Article  Google Scholar 

  6. W. Duch, R. Adamczak, and K. Grabczewski. A new methodology of extraction, optimization and application of crisp and fuzzy logical rules. IEEE Transactions on Neural Networks, 11:277–306, 2000.

    Google Scholar 

  7. G.D. Finn. Learning fuzzy rules from data. Neural Computing & Applications, 8:9–24, 1999.

    Article  Google Scholar 

  8. M.T. Hagan, H.B. Demuth, and M.H. Beale. Neural Network Design. PWS Publishing, Boston, 1996.

    Google Scholar 

  9. M.T. Hagan and M. Menhaj. Training feedforward networks with the Marquadt algorithm. IEEE Transactions on Neural Networks, 5:989–993, 1994.

    Article  Google Scholar 

  10. P. Hájek. Metamathematics of Fuzzy Logic. Kluwer Academic Publishers, Dordrecht, 1998.

    MATH  Google Scholar 

  11. P. Hájek and T. Havránek. Mechanizing Hypothesis Formation. SpringerVerlag, Berlin, 1978.

    MATH  Google Scholar 

  12. M. Holeňa. Ordering of neural network architectures. Neural Network World, 3:131–160, 1993.

    Google Scholar 

  13. M. Holeňa. Lattices of neural network architectures. Neural Network World, 4:435–464, 1994.

    Google Scholar 

  14. M. Holeňa. Observational logic integrates data mining based on statistics and neural networks. In D.A. Zighed, J. Komorowski, and J.M. Żytkov, editors, Principles of Data Mining and Knowledge Discovery, pages 440–445. Springer Verlag, Berlin, 2000

    Chapter  Google Scholar 

  15. M. Holeňa. Mining rules from empirical data with an ecological application. Technical report, Brandenburg University of Technology, Cottbus, 2002. ISBN 3-934934-07-2, 62 pages.

    Google Scholar 

  16. K. Hornik, M. Stinchcombe, H. White, and P. Auer. Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives. Neural Computation, 6:1262–1275, 1994.

    Article  MATH  Google Scholar 

  17. P. Howes and N. Crook. Using input parameter influences to support the decisions of feedforward neural networks. Neurocomputing, 24:191–206, 1999.

    Article  MATH  Google Scholar 

  18. M. Ishikawa. Rule extraction by successive regularization. Neural Networks, 13:1171–1183, 2000.

    Article  Google Scholar 

  19. V. Kůrková. Kolmogorov’s theorem and multilayer neural networks. Neural Networks, 5:501–506, 1992.

    Article  Google Scholar 

  20. M. Leshno, V.Y. Lin, A. Pinkus, and S. Schocken. Multilayer feedforward networks with a non-polynomial activation can approximate any function. Neural Networks, 6:861–867, 1993.

    Article  Google Scholar 

  21. W. Maass. Bounds for the computational power and learning complexity of analog neural nets. SIAM Journal on Computing, 26:708–732, 1997.

    Article  MATH  MathSciNet  Google Scholar 

  22. F. Maire. Rule-extraction by backpropagation of polyhedra. Neural Networks, 12:717–725, 1999.

    Article  Google Scholar 

  23. E.N. Mayoraz. On the complexity of recognizing regions computable by two-layered perceptrons. Annals of Mathematics and Artificial Intelligence, 24:129–153, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  24. S. Mitra, R.K. De, and S.K. Pal. Knowledge-based fuzzy MLP for classification and rule generation. IEEE Transactions on Neural Networks, 8:1338–1350, 1997.

    Article  Google Scholar 

  25. S. Mitra and Y. Hayashi. Neuro-fuzzy rule generation: Survey in soft computing framework. IEEE Transactions on Neural Networks, 11:748–768, 2000.

    Article  Google Scholar 

  26. D. Nauck, U. Nauck, and R. Kruse. Generating classification rules with the neuro-fuzzy system NEFCLASS. In Proceedings of the Biennial Conference of the North American Fuzzy Information Processing Society NAFIPS’96, pages 466–470, 1996.

    Google Scholar 

  27. D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations by error backpropagation. In D.E. Rumelhart and J.L. McClelland, editors, Parallel Distributed Processing: Experiments in the Microstructure of Cognition, pages 318–362, 1986.

    Google Scholar 

  28. A.B. Tickle, R. Andrews, M. Golea, and J. Diederich. The truth will come to light: Directions and challenges in extracting rules from trained artificial neural networks. IEEE Transactions on Neural Networks, 9:1057–1068, 1998.

    Article  Google Scholar 

  29. H. Tsukimoto. Extracting rules from trained neural networks. IEEE Transactions on Neural Networks, 11:333–389, 2000.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod vodárenskou věží 2, CZ-182 07, Praha 8

    Martin Holeňa

Authors
  1. Martin Holeňa
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Deutsches Forschungszentrum für Künstliche Intelligenz, Stuhlsatzenhausweg 3, 66123, Saarbrücken, Germany

    Steffen Lange

  2. National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, 101-8430, Tokyo, Japan

    Ken Satoh

  3. Department of Computer Science, University of Maryland, College Park, 20742, Maryland, MD, USA

    Carl H. Smith

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Holeňa, M. (2002). Extraction of Logical Rules from Data by Means of Piecewise-Linear Neural Networks. In: Lange, S., Satoh, K., Smith, C.H. (eds) Discovery Science. DS 2002. Lecture Notes in Computer Science, vol 2534. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36182-0_18

Download citation

  • DOI: https://doi.org/10.1007/3-540-36182-0_18

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00188-1

  • Online ISBN: 978-3-540-36182-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation