Neuron

Synonyms

Node; Unit

Definition

Neurons carry out the computational operations of a network; together with connections (see Topology of a Neural Network, Weights), they constitute the neural network. Computational neurons are highly abstracted from their biological counterparts. In most cases, the neuron forms a weighted sum of a large number of inputs (activations of other neurons), applies a nonlinear transfer function to that sum, and broadcasts the resulting output activation to a large number of other neurons. Such activation models the firing rate of the biological neuron, and the nonlinearity is used to limit it to a certain range (e.g., 0/1 with a threshold, (0 . . 1) with a sigmoid, ( − 1 . . 1) with a hyperbolic tangent, or (0 . . ∞) with an exponential function). Each neuron may also have a bias weight, i.e., a weight from a virtual neuron that is always maximally activated, which the learning algorithm can use to adjust the input sum quickly into the most effective range of...