Towards a Scalable Spiking Neural Network

Abstract

Spiking Neural Network (SNN) is the biological-brain model that has the capability of solving complex problems related to pattern recognition and character classification. Each neuron fires an output based on the discrete spikes that are collected from its predecessors which results in changing the ionic level or the membrane potential in the neuron. Once the level reaches a threshold, the neuron transmits signal to its successors. Transmitting data among neurons is a data-dependency problem because each neuron depends on receiving signals from its surrounding neurons at the current time and transmitting it into next neurons. Due to this high level of dependency, scaling up spiking neural network (represented by increasing number of neurons and hidden layers) is a real challenge. In hardware, the number of neurons is limited by the hardware (device) capacity and the number of internal wires available to connect between neurons. In this paper, we examine the main factors that significantly impact the scalability of spiking neural networks in hardware through implementing a spiking neural network model using the hardware description language SystemVerilog. Through evaluating the design on Alveo U55 high performance compute FPGA card, we found that the highest number of neurons that we can map on the hardware is 128 neurons per hidden layer and the higher number of synapses is 384,000 per hidden layer.