Skip to main content
SpringerLink
Log in

Analytical performance assessment and high-throughput low-latency spike routing algorithm for spiking neural network systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Large-scale artificial neural networks (ANNs) have been used to mimic the information processing function of the brain. Spiking neural networks (SNNs) are a kind of ANN, which mimic real biological neural networks, conveying information through the communication of short pulses between neurons. Since each neuron in these networks is connected to thousands of others, high bandwidth is required. Moreover, since the spike times are used to encode information in SNN, very low communication latency is also required. The 2D-NoC was used as a solution to provide a scalable interconnection fabric in large-scale parallel SNN systems. The 3D-ICs have also attracted a lot of attention as a potential solution to resolve the interconnect bottleneck. The combination of these two emerging technologies provides a new horizon for IC designs to satisfy the high requirements of low-power and small footprint in emerging AI applications. This paper first presents an analytical model to analyze the performance of different neural network topologies and compare it with a system-level simulation. Second, we present an architecture and a low-latency routing algorithm for spike traffic routing in 3D-NoC of spiking neurons (3DNoC-SNN). The 3DNoC-SNN is validated based on an RTL-level implementation, while area/power analysis is performed using 45-nm CMOS technology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Akbari N, Modarressi M (2017) A high-performance network-on-chip topology for neuromorphic architectures. In: 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), vol 2, pp 9–16. https://doi.org/10.1109/CSE-EUC.2017.188

  2. Akopyan F, Sawada J, Cassidy A, Alvarez-Icaza R, Arthur J, Merolla P, Imam N, Nakamura Y, Datta P, Nam GJ, Taba B, Beakes M, Brezzo B, Kuang JB, Manohar R, Risk WP, Jackson B, Modha DS (2015) Truenorth: design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE Trans Comput Aid Des Integr Circuits Syst 34(10):1537–1557. https://doi.org/10.1109/TCAD.2015.2474396

    Article  Google Scholar 

  3. Akram BA, Abderazek BA (2016) Adaptive fault-tolerant architecture and routing algorithm for reliable many-core 3d-noc systems. J Parallel Distrib Comput 93(C):30–43. https://doi.org/10.1016/j.jpdc.2016.03.014

    Google Scholar 

  4. Belhadj B, Valentian A, Vivet P, Duranton M, He L, Temam O (2014) The improbable but highly appropriate marriage of 3d stacking and neuromorphic accelerators. In: 2014 International Conference on Compilers, Architecture and Synthesis for Embedded Systems (CASES), pp 1–9. https://doi.org/10.1145/2656106.2656130

  5. Ben Abdallah A (2017) Advanced multicore systems-on-chip architecture, on-chip network, design. Springer, Berlin

    Book  Google Scholar 

  6. Benjamin BV, Gao P, McQuinn E, Choudhary S, Chandrasekaran AR, Bussat JM, Alvarez-Icaza R, Arthur JV, Merolla PA, Boahen K (2014) Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations. Proc IEEE 102(5):699–716. https://doi.org/10.1109/JPROC.2014.2313565

    Article  Google Scholar 

  7. Boahen KA (2000) Point-to-point connectivity between neuromorphic chips using address events. IEEE Trans Circuits Syst II Analog Digit Signal Process 47(5):416–434. https://doi.org/10.1109/82.842110

    Article  MATH  Google Scholar 

  8. Cao Y, Chen Y, Khosla D (2015) Spiking deep convolutional neural networks for energy-efficient object recognition. Int J Comput Vis 113(1):54–66. https://doi.org/10.1007/s11263-014-0788-3

    Article  MathSciNet  Google Scholar 

  9. Carrillo S, Harkin J, McDaid L, Pande S, Cawley S, McGinley B, Morgan F (2012) Advancing interconnect density for spiking neural network hardware implementations using traffic-aware adaptive network-on-chip routers. Neural Netw 33:42–57

    Article  Google Scholar 

  10. Carrillo S, Harkin J, McDaid LJ, Morgan F, Pande S, Cawley S, McGinley B (2013) Scalable hierarchical network-on-chip architecture for spiking neural network hardware implementations. IEEE Trans Parallel Distrib Syst 24(12):2451–2461. https://doi.org/10.1109/TPDS.2012.289

    Article  Google Scholar 

  11. Carrillo S, Harkin J, McDaid LJ, Pande S, Cawley S, McGinley B, Morgan F (2012) Hierarchical network-on-chip and traffic compression for spiking neural network implementations. In: 2012 IEEE/ACM Sixth International Symposium on Networks-on-Chip, pp 83–90

  12. Cawley S, Morgan F, Mcginley B, Pande S, Mcdaid L, Carrillo S, Harkin J (2011) Hardware spiking neural network prototyping and application. Genet Progr Evolv Mach 12(3):257–280

    Article  Google Scholar 

  13. Dang KN, Ahmed AB, Okuyama Y, Abderazek BA (2017) Scalable design methodology and online algorithm for TSV-cluster defects recovery in highly reliable 3d-noc systems. IEEE Trans Emerg Top Comput. https://doi.org/10.1109/TETC.2017.2762407

  14. Dang KN, Meyer M, Okuyama Y, Abdallah AB (2016) Reliability assessment and quantitative evaluation of soft-error resilient 3d network-on-chip systems. In: 2016 IEEE 25th Asian Test Symposium (ATS), pp 161–166

  15. Dong Y, Li C, Lin Z, Watanabe T (2010) Multiple network-on-chip model for high performance neural network. J Semicond Technol Sci 10:28–36

    Article  Google Scholar 

  16. Ebrahimi M, Daneshtalab M, Liljeberg P, Plosila J, Flich J, Tenhunen H (2014) Path-based partitioning methods for 3d networks-on-chip with minimal adaptive routing. IEEE Trans Comput 63(3):718–733. https://doi.org/10.1109/TC.2012.255

    Article  MathSciNet  MATH  Google Scholar 

  17. Ehsan MA, Zhou Z, Yi Y (2017) Modeling and analysis of neuronal membrane electrical activities in 3d neuromorphic computing system. In: 2017 IEEE International Symposium on Electromagnetic Compatibility Signal/power Integrity (EMCSI), pp 745–750. https://doi.org/10.1109/ISEMC.2017.8077966

  18. Furber S (2016) Large-scale neuromorphic computing systems. J Neural Eng 13(5):051,001

    Article  Google Scholar 

  19. Furber S, Temple S (2006) Neural systems engineering. J R Soc Interface 4(13):193–206

    Article  Google Scholar 

  20. Furber SB, Galluppi F, Temple S, Plana LA (2014) The spinnaker project. Proc IEEE 102(5):652–665. https://doi.org/10.1109/JPROC.2014.2304638

    Article  Google Scholar 

  21. Goldwyn H, Imennov N, Famulare M, Shea-Brown E (2011) Stochastic differential equation models for ion channel noise in Hodgkin–Huxley neurons. Phys Rev E 83:4190–4208

    Google Scholar 

  22. Henry M (2006) The blue brain project. Nat Rev Neurosci 2(7):153–159. https://doi.org/10.1038/nrn1848

    MathSciNet  Google Scholar 

  23. Hojabr R, Modarressi M, Daneshtalab M, Yasoubi A, Khonsari A (2017) Customizing clos network-on-chip for neural networks. IEEE Trans Comput 66(11):1865–1877. https://doi.org/10.1109/TC.2017.2715158

    Article  MathSciNet  MATH  Google Scholar 

  24. Lazzaro J, Wawrzynek J (1995) A multi-sender asynchronous extension to the AER protocol. In: Proceedings Sixteenth Conference on Advanced Research in VLSI, pp 158–169

  25. Legenstein R, Maass W (2007) Edge of chaos and prediction of computational performance for neural circuit models. Neural Netw 20(3):323–334. https://doi.org/10.1016/j.neunet.2007.04.017

    Article  MATH  Google Scholar 

  26. Levin JA, Rangan V, MALONE EC (2014) Efficient hardware implementation of spiking networks. Patent No. US 2014/0351190 A1, Filed May 1, 2014, Pub. Date Nov. 27, 2014

  27. Lin X, Ni LM (1993) Multicast communication in multicomputer networks. IEEE Trans Parallel Distrib Syst 4(10):1105–1117. https://doi.org/10.1109/71.246072

    Article  Google Scholar 

  28. Liu J, Harkin J, Maguire LP, McDaid LJ, Wade JJ, Martin G (2016) Scalable networks-on-chip interconnected architecture for astrocyte-neuron networks. IEEE Trans Circuits Syst I Regul Pap 63(12):2290–2303. https://doi.org/10.1109/TCSI.2016.2615051

    Article  Google Scholar 

  29. Liu S, Kramer J, Indiveri G, Delbrück T, Burg T, Douglas RJ (2001) Orientation-selective avlsi spiking neurons. Neural Netw 14(6–7):629–643. https://doi.org/10.1016/S0893-6080(01)00054-5

    Article  Google Scholar 

  30. Maas W (1997) Networks of spiking neurons: The third generation of neural network models. Trans Soc Comput Simul Int 14(4):1659–1671. http://dl.acm.org/citation.cfm?id=281543.281637

  31. Merolla PA, Arthur JV, Alvarez-Icaza R, Cassidy AS, Sawada J, Akopyan F, Jackson BL, Imam N, Guo C, Nakamura Y, Brezzo B, Vo I, Esser SK, Appuswamy R, Taba B, Amir A, Flickner MD, Risk WP, Manohar R, Modha DS (2014) A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197):668–673. https://doi.org/10.1126/science.1254642

    Article  Google Scholar 

  32. Monroe TB, e, PAB, Bruehl SP, Dietrich MS, Rogers BP, Gore JC, Atalla SW, Cowan RL (2017) The impact of alzheimer’s disease on the resting state functional connectivity of brain regions modulating pain: a cross sectional study. J Alzheimer’s Dis. https://doi.org/10.3233/JAD-161187

  33. Mortara A, Vittoz EA, Venier P (1995) A communication scheme for analog vlsi perceptive systems. IEEE J Solid State Circuits 30(6):660–669. https://doi.org/10.1109/4.387069

    Article  Google Scholar 

  34. Naveros F, Luque NR, Garrido JA, Carrillo RR, Anguita M, Ros E (2015) A spiking neural simulator integrating event-driven and time-driven computation schemes using parallel cpu-gpu co-processing: a case study. IEEE Trans Neural Netw Learn Syst 26(7):1567–1574. https://doi.org/10.1109/TNNLS.2014.2345844

    Article  MathSciNet  Google Scholar 

  35. Pande S, Morgan F, Cawley S, McGinley B, Carrillo S, Harkin J, McDaid L (2010) Embrace-sysc for analysis of noc-based spiking neural network architectures. In: 2010 International Symposium on System on Chip, pp 139–145. https://doi.org/10.1109/ISSOC.2010.5625566

  36. Parhami B (2013) Exact formulas for the average internode distance in mesh and binary tree networks. Comput Sci Inf Technol 1:165–168. https://doi.org/10.13189/csit.2013.010214

    Google Scholar 

  37. Pasero E, Perri M (2004) Hw-sw codesign of a flexible neural controller through a FPGA-based neural network programmed in vhdl. In: 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), vol 4, pp 3161–3165. https://doi.org/10.1109/IJCNN.2004.1381180

  38. Rojas R (1996) Neural networks: a systematic introduction. Springer, Berlin

    Book  MATH  Google Scholar 

  39. Samman FA, Hollstein T, Glesner M (2010) Adaptive and deadlock-free tree-based multicast routing for networks-on-chip. IEEE Trans Very Large Scale Integr (VLSI) Syst 18(7):1067–1080. https://doi.org/10.1109/TVLSI.2009.2019758

    Article  Google Scholar 

  40. Schemmel J, Fieres J, Meier K (2008) Wafer-scale integration of analog neural networks. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp 431–438

  41. Strogatz SH (2001) Exploring complex networks. Lit Rev Nat 410:268–276. https://doi.org/10.1038/35065725

    MATH  Google Scholar 

  42. Suzuki K, Okuyama Y, Abdallah AB (2018) Hardware design of a leaky integrate and fire neuron core towards the design of a low-power neuro-inspired spike-based multicore soc. In: Information Processing Society Tohoku Branch Conference

  43. Vainbrand D, Ginosar R (2010) Network-on-chip architectures for neural networks. In: 2010 Fourth ACM/IEEE International Symposium on Networks-on-Chip, pp 135–144. https://doi.org/10.1109/NOCS.2010.23

  44. Vainbrand D, Ginosar R (2011) Scalable network-on-chip architecture for configurable neural networks. Microprocess Microsyst 35(2):152–166. https://doi.org/10.1016/j.micpro.2010.08.005

    Article  Google Scholar 

  45. Vu TH, Murakami R, Okuyama Y, Abdallah AB (2018) Efficient optimization and hardware acceleration of CNNS towards the design of a scalable neuro inspired architecture in hardware. In: 2018 IEEE International Conference on Big Data and Smart Computing (BigComp), pp 326–332

  46. Xiang D, Shen K (2016) A new unicast-based multicast scheme for network-on-chip router and interconnect testing. ACM Trans Des Autom Electron Syst 21(2):24:1–24:23. https://doi.org/10.1145/2821506

    Article  MathSciNet  Google Scholar 

  47. Yang S, Wang J, Deng B, Liu C, Li H, Fietkiewicz C, Loparo KA (2018) Real-time neuromorphic system for large-scale conductance-based spiking neural networks. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2018.2823730

Download references

Acknowledgements

This work is supported by the Competitive Research Funding (CRF), the University of Aizu, Ref. P-2-2018. The first and the third authors are the main contributors of this work. This work is also partially supported by the VLSI Design and Education Center (VDEC), the University of Tokyo, Japan, in Collaboration with Synopsys, Inc., and Cadence Design Systems, Inc.

Author information

Authors and Affiliations

  1. Adaptive Systems Laboratory, Graduate School of Computer Science and Engineering, The University of Aizu, Aizu-Wakamatsu City, Fukushima, 965-8580, Japan

    Huy-The Vu, Yuichi Okuyama & Abderazek Ben Abdallah

  2. Hung Yen University of Technology and Education, Hai Duong, Vietnam

    Huy-The Vu

Authors
  1. Huy-The Vu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuichi Okuyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abderazek Ben Abdallah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huy-The Vu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vu, HT., Okuyama, Y. & Abdallah, A.B. Analytical performance assessment and high-throughput low-latency spike routing algorithm for spiking neural network systems. J Supercomput 75, 5367–5397 (2019). https://doi.org/10.1007/s11227-019-02792-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-019-02792-y

Keywords

Search

Navigation