Learning to memorize input--output mapping as bifurcation in neural dynamics: relevance of multiple timescales for synapse changes

Abstract

When a certain input--output mapping is memorized, the neural dynamics provide a prescribed neural activity output that depends on the external input. Without such an input, neural states do not provide memorized output. Only upon input, memory is recalled as an attractor, while neural activity without an input need not fall on such attractor but can fall on another attractor distinct from the evoked one. With this background, we propose that memory recall occurs as a bifurcation from the spontaneous attractor to the corresponding attractor matching the requested target output, as the strength of the external input is increased. We introduce a neural network model that enables the learning of such memories as bifurcations. After the learning process is complete, the neural dynamics are shaped to generate a prescribed target in the presence of each input. We find that the capacity of such memory depends on the timescales for the neural activity and synaptic plasticity. The maximal memory capacity is achieved at a certain relationship between the timescales, where the residence time at previous learned targets during the learning process is minimized.