Abstract
Deep Belief Networks (DBNs) has made a good effect in machine learning and object classification. However, the current question is how to reduce the computational cost without detrimental to accuracy. To solve this problem, this paper is undertaken to convert the Siegert neuron into LIF neuron in DBNs and analyze the effects of changing the value of parameters for spiking neurons such as thresholds and firing rates. Besides, we also add intrinsic plasticity (IP) into the network to render better adaptive capability. Besides, the most exciting results is the spiking DBN with intrinsic plasticity submits its first correct guess of the output label within an average of 2.5 ms after the onset of the simulated Poisson spike train input with the initial firing rates beyond 200 Hz, and the recognition accuracy is still more than 94 percent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mohamed, A.R., Yu, D., Deng, L.: Investigation of full-sequence training of deep belief networks for speech recognition. In: Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September, pp. 2846–2849. INTERSPEECH (2010)
Li, C., Li, Y.: A review on synergistic learning. IEEE Access 4, 119–134 (2016)
Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Deep, big, simple neural nets for handwritten digit recognition. Neural Comput. 22(12), 3207–3220 (2010)
Erhan, D., Bengio, Y., Courville, A., Manzagol, P.A., Vincent, P., Bengio, S.: Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11(3), 625–660 (2010)
Battenberg, E., Wessel, D.: Analyzing drum patterns using conditional deep belief networks. In: Ismir (2012)
Jug, F., Lengler, J., Krautz, C., Steger, A., Lengler, J., Krautz, C.: Spiking networks and their rate-based equivalents: does it make sense to use siegert neurons? Cadmo.ethz.ch
Daoudal, G., Debanne, D.: Long-term plasticity of intrinsic excitability: learning rules and mechanisms. Learn. Mem. 10(6), 456–465 (2003)
Hinton, G., Deng, L., Yu, D., Dahl, G.E., Mohamed, A., Jaitly, N., et al.: Deep neural networks for acoustic modeling in speech recognition. IEEE Sig. Process. Mag. 29(6), 82–97 (2012)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Dahl, G.E., Dong, Y., Li, D., Acero, A.: Large vocabulary continuous speech recognition with context-dependent DBN-HMMS. IEEE Int. Conf. Acoust. Speech Sig. Process. 125, 4688–4691 (2011)
Arel, I., Rose, D.C., Karnowski, T.P.: Deep machine learning - a new frontier in artificial intelligence research [research frontier]. IEEE Comput. Intell. Mag. 5(4), 13–18 (2010)
Triesch, J.: A gradient rule for the plasticity of a neurons intrinsic excitability. Int. Conf. Artif. Neural Netw.: Biol. Inspirations 3696, 65–70 (2005)
Desai, N.S., Rutherford, L.C., Turrigiano, G.G.: Plasticity in the intrinsic excitability of cortical pyramidal neurons. Nature Neurosci. 2(6), 515–520 (1999)
Wallisch, P., Lusignan, M.E., Benayoun, M.D., Baker, T.I., Dickey, A.S., Hatsopoulos, N.G.: MATLAB for neuroscientists: an introduction to scientific computing in MATLAB (2014)
O’Connor, P., Neil, D., Liu, S.C., Delbruck, T., Pfeiffer, M.: Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7, 178 (2013)
Ponulak, F., Kasiński, A.: Supervised learning in spiking neural networks with resume: sequence learning, classification, and spike shifting. Neural Comput. 22(2), 467–510 (2010)
Kourrich, S., Calu, D.J., Bonci, A.: Intrinsic plasticity: an emerging player in addiction. Nat. Rev. Neurosci. 16(3), 173–184 (2015)
Siegert, A.J.F.: On the first passage time probability problem. Phys. Rev. 81(4), 617–623 (1951)
Andrew, A.M.: Spiking neuron models: single neurons, populations, plasticity. Kybernetes 4(7/8), 277C–280 (2003)
Zhang, W., Linden, D.J.: The other side of the engram: experience-driven changes in neuronal intrinsic excitability. Nat. Rev. Neurosci. 4(11), 885–900 (2003)
Bengio, Y.: Learning deep architectures for ai. Found. Trends Mach. Learn. 2(1), 1–127 (2009)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Nos. 61304165 and 61473051) and Natural Science Foundation of Chongqing (No. cstc2016jcyjA0015).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Xue, F., Chen, X., Li, X. (2017). Real-Time Classification Through a Spiking Deep Belief Network with Intrinsic Plasticity. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10261. Springer, Cham. https://doi.org/10.1007/978-3-319-59072-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-59072-1_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59071-4
Online ISBN: 978-3-319-59072-1
eBook Packages: Computer ScienceComputer Science (R0)