Abstract
Spiking neural networks (SNNs) are a kind of data-driven and event-driven hierarchical networks, and they are closer to the biological mechanism than other traditional neural networks. In SNNs, signals are transmitted as spikes between neurons, and spike transmission is easily implemented on hardware platform for large-scale real-time deep network computing. However, the unsupervised learning methods for spike neurons, such as the STDP learning methods, generally are ineffective in training deep spiking neural networks for image classification application. In this paper, the network parameters (weights and bias) obtained from training a convolution neural network (CNN), are converted and utilized in a deep spiking neural network with the similar structure as the CNN, which make the deep SNN be capable of classifying images. Since the CNN is composed of analog neurons, there will be some transfer losses in the process of conversion. After the main sources of transfer losses are analyzed, some reasonable optimization strategies are proposed to reduce the losses while retain a higher accuracy, such as max-pooling, softmax and weight normalization. The deep spiking neural network proposed in this paper is closer to the biological mechanism in the design of neurons and our work is helpful for understanding the spike activity of the brain. The proposed deep SNN is evaluated on CIFAR and MNIST benchmarks and the experimental results have shown that the proposed deep SNN outperforms the state-of-the-art spiking network models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Neftci, E.O., Pedroni, B.U., Joshi, S., et al.: Stochastic synapses enable efficient brain-inspired learning machines. Front. Neurosci. 10, 241 (2016)
MFolowosele, F., Vogelstein, R.J., Etienne-Cummings, R.: Real-time silicon implementation of V1 in hierarchical visual information processing. In: Biomedical Circuits and Systems Conference, pp. 181–184. IEEE Press (2008)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE Press 86, 2278–2324 (1999). Morgan Kaufmann
Brader, J.M., Senn, W., Fusi, S.: Grid Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881–2912 (2007)
Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vis. 113, 54–66 (2015)
Diehl, P.U., Neil, D., Binas, J., et al.: Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: International Joint Conference on Neural Networks (IJCNN) 2015, IEEE, pp. 1–8. IEEE Press (2015)
Hunsberger, E., Eliasmith, C.: Spiking deep networks with LIF neurons. arXiv:1510.08829 (2015)
Maass, W., Markram, H.: On the computational power of circuits of spiking neurons. J. Comput. Syst. Sci. 69, 593–616 (2004)
Camunas-Mesa, L., Zamarreno-Ramos, C., Linares-Barranco, A., et al.: An event-driven multi-kernel convolution processor module for event-driven vision sensors. IEEE J. Solid-State Circ. 47, 504–517 (2012)
O’Connor, P., Neil, D., Liu, S.C., et al.: Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7 (2013)
Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The Physiology of the Grid: an Open Grid Services Architecture for Distributed Systems Integration. Technical report, Global Grid Forum (2002)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)
Vincent, P., Larochelle, H., Bengio, Y., et al.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning. ACM, pp. 1096–1103 (2008)
Angela, J.Y., Giese, M.A., Poggio, T.A.: Biophysiologically plausible implementations of the maximum operation. Neural Comput. 14, 2857–2881 (2002)
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)
Krizhevsky, A., Hinton, G.: Learning Multiple Layers of Features from Tiny Images (2009)
Acknowledgments
This study was partly supported by the National Natural Science Foundation of China (No. 41571402), the Science Fund for Creative Research Groups of the National Natural Science Foundation of China (No. 61221003), and Shanghai Jiao Tong University Agri-X Fund (No. Agri-X2015004).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Li, J., Hu, W., Yuan, Y., Huo, H., Fang, T. (2017). Bio-Inspired Deep Spiking Neural Network for Image Classification. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10635. Springer, Cham. https://doi.org/10.1007/978-3-319-70096-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-70096-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70095-3
Online ISBN: 978-3-319-70096-0
eBook Packages: Computer ScienceComputer Science (R0)