Information theoretic implications of embodiment for neural network learning

Abstract

The traditional view of neural networks is algorithmic. The general learning problems are typically hard and powerful networks and learning algorithms such as MLPs with BP must be used. It is a well-known fact that the power of the learning algorithm required to solve a problem depends on the statistical distribution of the input data. If the distribution is known, a “taylored” network can be used. We argue that in the real-world the- distributions are not given, but can be generated in the process of sensory-motor coordination as the embodied autonomous agent interacts with its environment. It is shown that sensory-motor coordination can lead to dramatic reduction of learning comlexity in the information theoretic sense. The ideas discussed in this paper tie in with a set of design principles for autonomous agents that we have established over the last few years.