Skip to main content
SpringerLink
Log in

Informational properties of neural nets performing algorithmic and logical tasks

  • Original Papers
  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

It is argued that the genetic information necessary to encode an algorithmic neural processor tutoring an otherwise randomly connected biological neural net is represented by the entropy of the analogous minimal Turing machine. Such a near-minimal machine is constructed performing the whole range of bivalent propositional logic in n variables. Neural nets computing the same task are presented; their informational entropy can be gauged with reference to the analogous Turing machine. It is also shown that nets with one hidden layer can be trained to perform algorithms solving propositional logic by error back-propagation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arbib MA (1987) Brains, machines, and mathematics. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Ash RB (1965) Information theory. Dover, New York

    Google Scholar 

  • Bergmann E, Noll H (1977) Mathematische Logik mit Informatik-Anwendungen. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Börger E (1986) Berechenbarkeit, Komplexität, Logik. Vieweg, Braunschweig

    Google Scholar 

  • Cover TM, Thomas JA (1991) Elements of information theory. Wiley. New York

    Google Scholar 

  • Denker JS, Schwartz D, Wittner B, Solla S, Howard R, Jackel L, Hopfield J (1987) Large automatic learning, rule extraction, and generalization. Complex Syst 1:877

    Google Scholar 

  • Gallagher R (1968) Information theory and reliable communication. Wiley New York

    Google Scholar 

  • Hampson SE, Volper DJ (1987) Disjunctive models of Boolean category learning. Biol Cybern 56:121–137

    Google Scholar 

  • Hermes H (1961) Aufzählbarkeit, Entscheidbarkeit, Berechenbarkeit. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Hertz J, Krogh A, Palmer R (1991) Introduction to the theory of neural computation. Addison-Wesley, Reading, Mass

    Google Scholar 

  • Hofacker GL, Borstnik B, Schöniger M (1988) Evolutionary adaptation to a real and an artificial world. In: Clementi E, Chin S (eds) Biological and artificial intelligence systems. ESCOM, Leiden

    Google Scholar 

  • Hopcroft JE, Ullmann JD (1988) Einführung in die Automatentheorie, Formale Sprachen und Komplexitätstheorie. Addison-Wesley, Reading, Mass

    Google Scholar 

  • Martin AP, Palumbi SR (1993) Body size, metabolic rate, generation time, and the molecular clock. Proc Natl Acad Sci USA 90:4087–4091

    Google Scholar 

  • Minsky ML (1967) Computation: Finite and infinite machines. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  • Minsky ML, Papert S (1969) Perceptrons: an introduction to computational geometry. MIT Press, Cambridge, Mass

    Google Scholar 

  • Pinkas G (1991) Symmetric neural networks and propositional logic satisfiability. Neural Comput 3:282–291

    Google Scholar 

  • Ritz B, Hofacker GL (1995) The information contents of neural nets performing object-adapted algorithmic and logical tasks. In: Elsner N, Menzel R (eds) Goettingen neurobiology report 1995. Thieme, Stuttgart, p 883

    Google Scholar 

  • Rumelhart DE, Hinton GE, Williams RJ (1986a) Learning representations by back-propagating errors. Nature 323:533–536

    Google Scholar 

  • Rumelhart DE, Hinton GE, Williams RJ (1986b) Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing: exploration in the microstructure of cognition, vol. 1. MIT Press, Mass

    Google Scholar 

  • Shannon CE, Weaver W (1963) The mathematical theory of communication. University of Illinois Press, Urbana

    Google Scholar 

  • Siegelmann HT (1995) Computation beyond the turing limit. Science 268:545–548

    Google Scholar 

  • Turing AM (1936) On computable numbers, with an application to the Entscheidungsproblem. Proc Lond Math Soc, Ser 2, 42:230–265

    Google Scholar 

  • Turing AM (1969) Intelligent machinery. In: Meltzer B, Michie D (eds) Machine intelligence 5. Edinburgh

  • Valiant LG (1984) A theory of the learnable. Communications of the ACM, Vol 27, no. 11

  • Zell A et al. (1993) SNNS: Stuttgart Neural Network Simulator, user manual (v3.1 report no. 3/93 (revised)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of California, San Diego, 92093-0114, La Jolla, CA, USA

    Barbara M. Ritz

  2. Institut für Physikalische und Theoretische Chemie, Technische Universität München, D-85747, Garching bei München, Germany

    G. Ludwig Hofacker

Authors
  1. Barbara M. Ritz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Ludwig Hofacker
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This article was processed by the author using the LaTEX style file pljour2 from Springer-Verlag.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ritz, B.M., Hofacker, G.L. Informational properties of neural nets performing algorithmic and logical tasks. Biol. Cybern. 74, 549–555 (1996). https://doi.org/10.1007/BF00209426

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00209426

Keywords

Search

Navigation