Introduction

There has recently been much interest in “artificial neural networks,” machines (or models of computation) based loosely on the ways in which the brain is believed to work. Neurobiologists are interested in using these machines as a means of modeling biological brains, but much of the impetus comes from their applications. For example, engineers wish to create machines that can perform “cognitive” tasks, such as speech recognition, and economists are interested in financial time series prediction using such machines.

In this chapter we focus on individual “artificial neurons” and feed-forward artificial neural networks. We are particularly interested in cases where the neurons are linear threshold neurons, sigmoid neurons, polynomial threshold neurons, and spiking neurons. We investigate the relationships between types of artificial neural network and classes of Boolean function. In particular, we ask questions about the type of Boolean functions a given type of network can compute, and about how extensive or expressive the set of functions so computable is.

Artificial Neural Networks

Introduction

It appears that one reason why the human brain is so powerful is the sheer complexity of connections between neurons. In computer science parlance, the brain exhibits huge parallelism, with each neuron connected to many other neurons. This has been reflected in the design of artificial neural networks. One advantage of such parallelism is that the resulting network is robust: in a serial computer, a single fault can make computation impossible, whereas in a system with a high degree of parallelism and many computation paths, a small number of faults may be tolerated with little or no upset to the computation.