Skip to main content
SpringerLink
Log in

Hardware spiking neural network prototyping and application

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

EMBRACE has been proposed as a scalable, reconfigurable, mixed signal, embedded hardware Spiking Neural Network (SNN) device. EMBRACE, which is yet to be realised, targets the issues of area, power and scalability through the use of a low area, low power analogue neuron/synapse cell, and a digital packet-based Network on Chip (NoC) communication architecture. The paper describes the implementation and testing of EMBRACE-FPGA, an FPGA-based hardware SNN prototype. The operation of the NoC inter-neuron communication approach and its ability to support large scale, reconfigurable, highly interconnected SNNs is illustrated. The paper describes an integrated training and configuration platform and an on-chip fitness function, which supports GA-based evolution of SNN parameters. The practicalities of using the SNN development platform and SNN configuration toolset are described. The paper considers the impact of latency jitter noise introduced by the NoC router and the EMBRACE-FPGA processor-based neuron/synapse model on SNN accuracy and evolution time. Benchmark SNN applications are described and results demonstrate the evolution of high quality and robust solutions in the presence of noise. The reconfigurable EMBRACE architecture enables future investigation of adaptive hardware applications and self repair in evolvable hardware.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. S. Grossberg, W. Maass, H. Markram, Introduction: spiking Neurons in Neuroscience and Technology. Neural Netw. 14, 587 (2001)

    Article  Google Scholar 

  2. W. Maass, Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10, 1659–1671 (1997)

    Article  Google Scholar 

  3. W. Gerstner, W.M. Kistler, Spiking neuron models, Cambridge University Press, (2002)

  4. A. Upegui, C.A. Peña-Reyes, E. Sanchez, An FPGA platform for on-line topology exploration of spiking neural networks. Microprocess. Microsyst. 29, 211–223 (2005)

    Article  Google Scholar 

  5. M. Pearson, A. Pipe, B. Mitchinson, K. Gurney, C. Melhuish, I. Gilhespy, M. Nibouche, Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach. Neural Netw. IEEE Trans. 18, 1472–1487 (2007)

    Article  Google Scholar 

  6. E. Ros, E. Ortigosa, R. Agis, R. Carrillo, M. Arnold, Real-time computing platform for spiking neurons (RT-spike). Neural Netw. IEEE Trans. 17, 1050–1063 (2006)

    Article  Google Scholar 

  7. R. Vogelstein, U. Mallik, J. Vogelstein, G. Cauwenberghs, Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses. Neural Netw. IEEE Trans. 18, 253–265 (2007)

    Article  Google Scholar 

  8. D.B. Thomas, W. Luk, FPGA accelerated simulation of biologically plausible spiking neural networks. FCCM (2009)

  9. J. Harkin, F. Morgan, L. McDaid, S. Hall, B. McGinley, S. Cawley, A reconfigurable and biologically inspired paradigm for computation using network-on-chip and spiking neural networks. Int. J. Reconfig. Comput. 2009, 1–13 (2009)

    Article  Google Scholar 

  10. M. Khan, D. Lester, L. Plana, A. Rast, X. Jin, E. Painkras, S. Furber, SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor, Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, pp. 2849–2856 (2008)

  11. Yajie Chen, L. McDaid, S. Hall, P. Kelly, A programmable facilitating synapse device, Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, pp. 1615–1620 (2008)

  12. H. Markram, The Blue Brain Project. Nat. Rev. Neurosci. 7, 153–160 (2006)

    Article  Google Scholar 

  13. J. Harkin, F. Morgan, S. Hall, P. Dudek, T. Dowrick, L. McDaid, Reconfigurable platforms and the challenges for large-scale implementations of spiking neural networks, Field Programmable Logic and Applications, 2008. FPL 2008. International Conference on, pp. 483–486 (2008)

  14. J. Navaridas, M. Luján, J. Miguel-Alonso, L.A. Plana, S. Furber, Understanding the interconnection network of SpiNNaker, Proceedings of the 23rd international conference on Supercomputing, pp. 286–295 (2009)

  15. J. Schemmel, J. Fieres, K. Meier, Wafer-scale integration of analog neural networks, Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, pp. 431–438 (2008)

  16. S. Cawley, F. Morgan, B. McGinley, S. Pande, J. Harkin, L. McDaid, EMBRACE-FPGA: evolving controllers and classifiers on hardware spiking neural network, FPL (2010)

  17. L. Benini, G. De Micheli, Networks on chips: a new SoC paradigm. Comput. 35, 70–78 (2002)

    Article  Google Scholar 

  18. A. DeHon, R. Rubin, Design of FPGA interconnect for multilevel metallization, very large scale integration (VLSI) systems. IEEE Trans. 12, 1038–1050 (2004)

    Google Scholar 

  19. J. Harkin, M. McElholm, Novel interconnect strategy for large scale implementations of NNs, IEEE Soft Comp. in Indust. App. (2007)

  20. S. Jovanovic, C. Tanougast, S. Weber, C. Bobda, CuNoC: a scalable dynamic NoC for dynamically reconfigurable FPGAs, Field Programmable Logic and Applications, 2007. FPL 2007. International Conference on, pp. 753–756 (2007)

  21. S. Pande, F. Morgan, S. Cawley, B. McGinley, S. Carrillo, J. Harkin, L. McDaid, EMBRACE-SysC for analysis of NoC-based spiking neural network architectures, System on Chip (SoC), 2010 International Symposium on, pp. 139–145 (2010)

  22. S. Pande, F. Morgan, S. Cawley, B. McGinley, S. Carrillo, J. Harkin, L. McDaid, EMBRACE-SysC for analysis of noc-based spiking neural network architectures, International Symposium on System-on-Chip (2010)

  23. H. El-Bakry, Modular neural networks for solving high complexity problems, Neural Networks, 2003. Proceedings of the International Joint Conference on, pp. 2202–2207 vol.3 (2003)

  24. T. Kumagai, M. Wada, R. Hashimoto, A. Utsugi, Dynamical control by recurrent neural networks through genetic algorithms. Int. J. Adapt. Control Signal Process. 13, 261–271 (1999)

    Article  MATH  Google Scholar 

  25. M. Ventresca and B. Ombuki, Search space analysis of recurrent spiking and continuous-time neural networks, Neural Networks, 2006. IJCNN ‘06. International Joint Conference on, pp. 4514–4521 (2006)

  26. B. Glackin, T. McGinnity, L. Maguire, Q. Wu, A. Belatreche, A novel approach for the implementation of large scale spiking neural networks on FPGA hardware, Comput. Int. Bioinspir. Syst., pp. 552–563 (2005)

  27. J. Schemmel, J. Fieres, K. Meier, Wafer-scale integration of analog neural networks, IEEE International Joint Conference on Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence), pp. 431–438 (2008)

  28. M. Ehrlich, C. Mayr, H. Eisenreich, S. Henker, A. Srowig, A. Grubl, J. Schemmel, R. Schuffny, Wafer-scale VLSI implementations of pulse coupled neural networks, Proceedings of the International Conference on Sensors, Circuits and Instrumentation Systems (2007)

  29. F. Morgan, S. Cawley, B. McGinley, S. Pande, L. McDaid, B. Glackin, J. Harkin, Exploring the evolution of NoC-based spiking neural networks on FPGAs (2009)

  30. E. Izhikevich, Which model to use for cortical spiking neurons? Neural Netw. IEEE Trans. 15, 1063–1070 (2004)

    Article  Google Scholar 

  31. J. Holland, Adaptation in natural and artificial systems an introductory analysis with applications to biology, control, and artificial intelligence, Cambridge Mass. [u.a.]: MIT Press (2001)

  32. S.E. Fahlman, An empirical study of learning speed in back-propagation networks

  33. J. Maher, B. McGinley, P. Rocke, F. Morgan, Intrinsic hardware evolution of neural networks in reconfigurable analogue and digital devices, Field-Programmable Custom Computing Machines, 2006. FCCM ‘06. 14th Annual IEEE Symposium on, pp. 321–322 (2006)

  34. O. Booij, H. tat Nguyen, A gradient descent rule for spiking neurons emitting multiple spikes. Inf. Process. Lett. 95, 552–558 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  35. S. Cawley, F. Morgan, B. McGinley, S. Pande, L. McDaid, J. Harkin, The impact of neural model resolution on hardware spiking neural network behaviour, ISSC (2010)

  36. E. Pasero, M. Perri, Hw-Sw codesign of a flexible neural controller through a FPGA-based neural network programmed in VHDL, Neural Network, 2004. Proceedings. 2004 IEEE International Joint Conference on, vol.4, pp. 3161–3165 (2004)

  37. A. Pérez-Uribe and E. Sanchez, Structure-adaptable neurocontrollers: a hardware-friendly approach, in Proceedings of the International Work-Conference on Artificial and Natural Neural Networks: Biological and Artificial Computation: From Neuroscience to Technology, (Springer, 1997), pp. 1251–1259

  38. Wisconsin, UCI Machine Learning Repository: breast cancer wisconsin (Diagnostic) data set

  39. L. Bako, Real-time classification of datasets with hardware embedded neuromorphic neural networks, Brief. Bioinfo. (2010)

Download references

Acknowledgments

This research is supported by Science Foundation Ireland under Grant No. 07/SRC/I1169 and the Irish Research Council for Science, Engineering and Technology (IRCSET), International Centre for Graduate Education in Micro and Nano Engineering (ICGEE). The authors also thank the Xilinx University Programme.

Author information

Authors and Affiliations

  1. Bio-Inspired and Reconfigurable Computing Research Group, National University of Ireland, Galway, Galway, Ireland

    Seamus Cawley, Fearghal Morgan, Brian McGinley & Sandeep Pande

  2. Intelligent Systems Research Centre, University of Ulster, Magee Campus, Derry, Northern Ireland,UK

    Liam McDaid, Snaider Carrillo & Jim Harkin

Authors
  1. Seamus Cawley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fearghal Morgan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brian McGinley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sandeep Pande
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liam McDaid
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Snaider Carrillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jim Harkin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seamus Cawley.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cawley, S., Morgan, F., McGinley, B. et al. Hardware spiking neural network prototyping and application. Genet Program Evolvable Mach 12, 257–280 (2011). https://doi.org/10.1007/s10710-011-9130-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-011-9130-9

Keywords

Search

Navigation