Learning Sensory-Motor Coordination by Experimentation and Reinforcement Learning

Abstract

This work shows how a neural network can learn a motor control task by trial and error using a reinforcement learning scheme, exemplified by a system that learns to focus an “eye” on moving objects or salient parts of pictures. No explicit knowledge about the details of the “oculomotor system” is used during training. The system described is embedded in an environment in which it acts. It can perceive the changes it causes in its environment and evaluates them with respect to some goal implicit in its architecture. The solutions the network arrives at are achieved by correlation of visual input with random gestures (experimentation) by a reinforcement learning scheme that makes use of “heterosynaptic modulation,” as proposed by Reeke & Edelman (1989). Through learning, the performance of the system gradually improves so that random move generation becomes obsolete. Simulations have shown that the system is able to learn to track moving objects, as well as to trace the contours of stationary pictures.