Evolution in the Orange Box — A New Approach to the Sphere-Packing Problem in CMAC-Based Neural Networks

Abstract

The sphere-packing problem is the task of finding an arrangement to achieve the maximum density of identical spheres in a given space. This problem arises in the placement of kernel functions for uniform input space quantisation in machine learning algorithms. One example is the Cerebellar Model Articulation Controller (CMAC), where the problem arises as the placement of overlapping grids. In such situations, it is desirable to achieve a uniform placement of grid vertices in input space. This is akin to the sphere-packing problem, where the grid vertices are the centres of spheres. The nature of space quantisation inherent in such algorithms imposes constraints on the solution and usually requires a regular tessellation of spheres. The sphere-packing problem is difficult to solve analytically, especially with these constraints. The current approach in the case of CMAC-based methods is to rely on published tables of grid spacings, but this has two shortcomings. First, no analytical solution has been published for the calculation of such tables - they were arrived at by exhaustive search. Second, the tables include input spaces of only ten dimensions or less. Many data mining problems now rely upon machine learning techniques to solve problems in higher dimensional spaces. A new approach to obtaining suitable grid spacings, based on a Genetic Algorithm, is described, which is potentially faster than exhaustive search. The resulting grid spacings are very similar to the published tables, and empirical trials show that where they differ, the performance on an automated classifier problem is unchanged. The new approach is also feasible for more than ten dimensions, and tables are presented for grid spacings in higher dimensional spaces. The results are applicable to any application where a regular division of input space is required. They allow the investigation of space quantising algorithms for solving problems in high dimensional spaces.