Building Faster Connectionist Systems With Bumptrees

Abstract

This paper describes “bumptrees”, a new approach to improving the computational efficiency of a wide variety of connectionist algorithms. We describe the use of these structures for representing, learning, and evaluating smooth mappings, smooth constraints, classification regions, and probability densities. We present an empirical comparison of a bumptree approach to more traditional connectionist approaches for learning the mapping between the kinematic and visual representations of the state of a 3 joint robot arm. Simple networks based on backpropagation with sigmoidal units are unable to perform the task at all. Radial basis function networks perform the task but by using bumptrees, the learning rate is hundreds of times faster at reasonable error levels and the retrieval time is over fifty times faster with 10,000 samples. Bumptrees are a natural generalization of oct-trees, k-d trees, balltrees and boxtrees and are useful in a variety of circumstances. We describe both the underlying ideas and extensions to constraint and classification learning that are under current investigation.