Skip to main content
SpringerLink
Log in

Class-Entropy minimisation networks for domain analysis and rule extraction

  • Articles
  • Published:
Neural Computing & Applications Aims and scope Submit manuscript

Abstract

When applied to supervised classification problems, neural rule extraction aims at making classification mechanisms explicit by reversing the knowledge embedded in a network's connections. To this end, the present research has borrowed notions from information theory; using Conditional Class Entropy as a network cost function improves representation efficiency by forcing the network to emphasize only task-essential information. We present a library of methods to analyse, simplify and rearrange the knowledge embedded in CCE-trained networks; the final result is a hierarchy of if-then rules modelling the classification process in symbolic form. Experimental results on a clinical testbed (diagnosis of Lyme disease) confirmed the effectiveness and the generalisation power of the methodologies described. In addition, satisfactory results obtained on this still unsolved clinical problem endowed this research with interdisciplinary value.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Towell G, Shavlik J. Interpretation of artificial neural networks: mapping knowledge-based neural networks into rules. Advances in NIPS 4. pp. 977–984, Morgan-Kaufman, 1992

  2. Abe S, Kayama M, Takenaga H, Kitamura T. Extracting algorithms from pattern classification neural networks. Neural Networks 1993; 6(5): 729–735

    Google Scholar 

  3. Oja E. Principal components, minor components, and linear neural networks. Neural Networks November–December 1992; 5(6): 927–936

    Google Scholar 

  4. Fukunaga K. Introduction to Statistical Pattern Recognition, 2nd Ed. Academic Press, 1990

  5. Kramer MA. Nonlinear principal component analysis using autoassociative neural networks. AIChE J February 1991; 37(2): 233–243

    Google Scholar 

  6. Kohonen T. Self-organization and associative memory, 3rd Ed. Springer-Verlag, 1989

  7. Linsker R. Self-organization in a perceptual neural network. Computer 1988; 21: 105–117

    Google Scholar 

  8. Bichsel M, Seitz P. Minimum class entropy: a maximum information approach to layered networks. Neural Networks 1989; 2: 133–141

    Google Scholar 

  9. Burgdorfer W, Barbour AG, Hayes SF, Benach JL, Grunwaldt E, Davis JP. Lyme disease — A tick-borne spirochetosis? Science 1982; 216(1317)

  10. Bernelot Moens HJ, van der Kost J. Computer-assisted diagnosis of rheumatic disorders. Seminars in Arthritis and Rheumatism December 1991; 21(3): 156–169

    Google Scholar 

  11. Moneta C, Parodi GC, Rovetta S, Zunino R. Automated diagnosis and disease characterization by using neural network analysis. In Proc IEEE Int Conf Sys Man and Cybern Chicago, IL, October 1992, pp. 123–128

  12. Barinaga M. Furor at Lyme disease conference. Science 1992; 256(1384)

  13. Fahlman SE, Lebiere C. The cascade-correlation learning architecture. Advances in NIPS 2. pp. 524–532, Morgan-Kaufman, 1990

  14. Kirkpatrick S, Gelatt CD Jr., Vecchi MP. Optimization by simulated annealing. Science 220: 1983; 671

    Google Scholar 

  15. Corana A, Marchesi M, Martini C, Ridella S. Minimizing multimodal functions of continuous variables with the ‘simulated annealing’ algorithm. ACM Trans Math Soft 1987; (51): 262–280

    Google Scholar 

  16. Baba N. A new approach for finding the global minimum of error function of neural networks. Neural Networks 1989; 2: 367–373

    Google Scholar 

  17. Widrow B, Lehr MA. 30 years of adaptive neural networks: perceptron, madaline, and backpropagation. Proc. IEEE September 1990; 78(9): 1415–1442

    Google Scholar 

  18. Weigend R, Rumelhart DE, Huberman BA. Generalization by weight-elimination with application to forecasting. Advances in NIPS 3, Morgan-Kaufman, 1991

  19. Nowlan SJ, Hinton GE. Simplifying neural networks by soft weight sharing. Neural Computation 1992; 4(4): 473–493

    Google Scholar 

  20. Bartee TC, Lebow IL, Reed IS. Theory and Design of Digital Machines. McGraw-Hill, 1962

  21. Michalski R, Carbonell J, Mitchell T. Machine Learning: An Artificial Intelligence Approach, Vol. I. Morgan-Kaufman, 1983

  22. Saito K, Nakano R. Medical diagnostic expert system based on PDP model. In: Proc IEEE Int Conf on Neural Networks 1988; pp. I 255–I 262

  23. Rovetta G, et al. Epidemiology of Lyme borreliosis in Italy. In: Proc V Int Conf on Lyme Borreliosis, A340, Arlington, VA, June 1992

Download references

Author information

Authors and Affiliations

  1. Department of Biophysical and Electronic Engineering, University of Genova, Genova, Italy

    Sandro Ridella, Gian Luca Speroni, Paolo Trebino & Rodolfo Zunino

Authors
  1. Sandro Ridella
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gian Luca Speroni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paolo Trebino
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rodolfo Zunino
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ridella, S., Speroni, G.L., Trebino, P. et al. Class-Entropy minimisation networks for domain analysis and rule extraction. Neural Comput & Applic 2, 40–52 (1994). https://doi.org/10.1007/BF01423097

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01423097

Keywords

Search

Navigation