A Tiny Flat-island in a Huge Lake — How can we search for it if completely flatland elsewhere?

Abstract

In the background of this paper, lies a simulation of an associative memory model with spiking neurons. We want, however, to put the issue aside for a while, since we came across a problem, very simple but extremely difficult one, when we explored a fitness landscape — a weight configuration space of high dimensionality where weight solutions are supposed to look like peaks. In the landscape, the location of one of those peaks is already known. This is called the Hebbian peak — a weight configuration in which two neurons are wired when they both fire. We guess many other peaks exist though we have not found any yet so far. During we searched for such solutions, we observed that the fitness landscape was almost everywhere completely flatland of altitude zero except for the Hebbian peak which shows a peculiar shape like a-tiny-flat-island-in-a-huge-lake. In such circumstances how could we search for other peaks? This paper is a call for challenges to the problem.