skip to main content
research-article

Programmable Spike-Timing-Dependent Plasticity Learning Circuits in Neuromorphic VLSI Architectures

Published: 02 September 2015 Publication History

Abstract

Hardware implementations of spiking neural networks offer promising solutions for computational tasks that require compact and low-power computing technologies. As these solutions depend on both the specific network architecture and the type of learning algorithm used, it is important to develop spiking neural network devices that offer the possibility to reconfigure their network topology and to implement different types of learning mechanisms. Here we present a neuromorphic multi-neuron VLSI device with on-chip programmable event-based hybrid analog/digital circuits; the event-based nature of the input/output signals allows the use of address-event representation infrastructures for configuring arbitrary network architectures, while the programmable synaptic efficacy circuits allow the implementation of different types of spike-based learning mechanisms. The main contributions of this article are to demonstrate how the programmable neuromorphic system proposed can be configured to implement specific spike-based synaptic plasticity rules and to depict how it can be utilised in a cognitive task. Specifically, we explore the implementation of different spike-timing plasticity learning rules online in a hybrid system comprising a workstation and when the neuromorphic VLSI device is interfaced to it, and we demonstrate how, after training, the VLSI device can perform as a standalone component (i.e., without requiring a computer), binary classification of correlated patterns.

References

[1]
L. F. Abbott and W. Gerstner. 2004. Homeostasis and learning through spike-timing dependent plasticity. In Methods and Models in Neurophysics, D. Hansel, C. Chow, B. Gutkin, and C. Meunier (Eds.). Elsevier Science. http://infoscience.epfl.ch/record/114304/files/Abbott04.pdf.
[2]
J. V. Arthur, P. Merolla, F. Akopyan, R. Alvarez, A. Cassidy, A. Chandra, S. Esser, N. Imam, W. Risk, D. B. D. Rubin, R. Manohar, and D. S. Modha. 2012. Building block of a programmable neuromorphic substrate: A digital neurosynaptic core. In Proceedings of the International Joint Conference on Neural Networks (IJCNN'12).
[3]
M. Rahimi Azghadi, S. Al-Sarawi, D. Abbott, and N. Iannella. 2013a. A neuromorphic VLSI design for spike timing and rate based synaptic plasticity. Neural Netw. 45, 70--82.
[4]
M. Rahimi Azghadi, S. Moradi, and G. Indiveri. 2013b. Programmable neuromorphic circuits for spike-based neural dynamics. In Proceedings of the 11th IEEE International New Circuits and Systems Conference (NEWCAS'13).
[5]
M. Rahimi Azghadi, S. Al-Sarawi, N. Iannella, and D. Abbott. 2012. Efficient design of triplet based spike-timing dependent plasticity. In Proceedings of the International Joint Conference on Neural Networks (IJCNN'12).
[6]
M. Rahimi Azghadi, S. AL-Sarawi, N. Iannella, and D. Abbott. 2014a. Tunable low energy, compact and high performance neuromorphic circuit for spike-based synaptic plasticity. PLoS ONE 9, 2.
[7]
M. Rahimi Azghadi, N. Iannella, S. F. Al-Sarawi, G. Indiveri, and D. Abbott. 2014b. Spike-based synaptic plasticity in silicon: Design, implementation, application, and challenges. Proc. IEEE 102, 5, 717--737.
[8]
S. Bamford, A. F. Murray, and D. J. Willshaw. 2012. Spike-timing-dependent plasticity with weight dependence evoked from physical constraints. IEEE Trans. Biomed. Circ. Syst. 6, 4, 385--398.
[9]
C. Bartolozzi and G. Indiveri. 2007. Synaptic dynamics in analog VLSI. Neural Comput. 19, 10, 2581--2603.
[10]
A. Belatreche, L. P. Maguire, and M. Mcginnity. 2006. Advances in design and application of spiking neural networks. Soft Comput. 11, 3, 239--248.
[11]
E. L. Bienenstock, L. N. Cooper, and P. W. Munro. 1982. Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 1, 32--48.
[12]
A. Bofill-I-Petit and A. F. Murray. 2004. Synchrony detection and amplification by silicon neurons with STDP synapses. IEEE Trans. Neural Netw. 15, 5, 1296--1304.
[13]
J. Brader, W. Senn, and S. Fusi. 2007. Learning real world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881--2912.
[14]
R. Brette and W. Gerstner. 2005. Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J. Neurophys. 94, 3637--3642.
[15]
E. Chicca, F. Stefanini, C. Bartolozzi, and G. Indiveri. 2014. Neuromorphic electronic circuits for building autonomous cognitive systems. Proc. IEEE 102, 9, 1367--1388.
[16]
E. Chicca, A. M. Whatley, P. Lichtsteiner, V. Dante, T. DelbrüCk, P. Del Giudice, R. J. Douglas, and G. Indiveri. 2007. A multi-chip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity. IEEE Trans. Circ. Syst. I 5, 54, 981--993.
[17]
T. Y. W. Choi, B. E. Shi, and K. Boahen. 2004. An on-off orientation selective address event representation image transceiver chip. IEEE Trans. Circ. Syst. I51, 2, 342--353.
[18]
C. Clopath, L. Büsing, E. Vasilaki, and W. Gerstner. 2010. Connectivity reflects coding: A model of voltage-based STDP with homeostasis. Nature Neurosci. 13, 3, 344--352.
[19]
P. Dayan and L. F. Abbott. 2001. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Taylor and Francis.
[20]
T. Delbruck, R. Berner, P. Lichtsteiner, and C. Dualibe. 2010. 32-bit configurable bias current generator with sub-off-current capability. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'10). 1647--1650.
[21]
D. B. Fasnacht, A. M. Whatley, and G. Indiveri. 2008. A serial communication infrastructure for multi-chip address event system. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'08). 648--651.
[22]
A. Fidjeland, D. Gamez, M. P. Shanahan, and E. Lazdins. 2013. Three tools for the real-time simulation of embodied spiking neural networks using GPUs. Neuroinf. 11, 3, 267--290.
[23]
A. Fidjeland, E. B. Roesch, M. P. Shanahan, and W. Luk. 2009. NeMo: A platform for neural modelling of spiking neurons using GPUs. In Proceedings of the IEEE Application-Specific Systems, Architectures and Processors Conference (ASAP'09). 137--144.
[24]
S. B. Furber, D. R. Lester, L. A. Plana, J. D. Garside, E. Painkras, S. Temple, and A. D. Brown. 2013. Overview of the SpiNNaker system architecture. IEEE Trans. Comput. 62, 12, 2454--2467.
[25]
W. Gerstner and W. M. Kistler. 2002. Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press.
[26]
M. Giulioni, M. Pannunzi, D. Badoni, V. Dante, and P. Del Giudice. 2009. Classification of correlated patterns with a configurable analog VLSI neural network of spiking neurons and self-regulating plastic synapses. Neural Comput. 21, 11, 3106--3129.
[27]
J. Gjorgjieva, C. Clopath, J. Audet, and J.-P. Pfister. 2011. A triplet spike-timing--dependent plasticity model generalizes the Bienenstock--Cooper--Munro rule to higher-order spatiotemporal correlations. Proc. Nat. Acad. Sci. 108, 48, 19383--19388.
[28]
M. Graupner and N. Brunel. 2012. Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location. Proc. Nat. Acad. Sci. 109, 3991--3996.
[29]
G. Indiveri. 2002. Neuromorphic bistable VLSI synapses with spike-timing-dependent plasticity. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS'02). Vol. 15. 1091--1098.
[30]
G. Indiveri, B. Linares-Barranco, T. J. Hamilton, A. Van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P, Häfliger, S. Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele, S. Saighi, T. Serrano-Gotarredona, J. Wijekoon, Y. Wang, and K. Boahen. 2011. Neuromorphic silicon neuron circuits. Frontiers Neurosci. 5, 1--23.
[31]
G. Indiveri, F. Stefanini, and E. Chicca. 2010. Spike-based learning with a generalized integrate and fire silicon neuron. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'10). 1951--1954.
[32]
R. Kempter, W. Gerstner, and J. L. Van Hemmen. 1999. Hebbian learning and spiking neurons. Phys. Rev. E59, 4, 4498--4514.
[33]
D. Khosla, D. J. Huber, and C. Kanan. 2014. A neuromorphic system for visual object recognition. Biologic. Inspired Cogn. Archit. 8, 33--45.
[34]
C. Koch. 1999. Biophysics of Computation: Information Processing in Single Neurons. Oxford University Press.
[35]
S. B. Laughlin and T. J. Sejnowski. 2003. Communication in neuronal networks. Sci. 301, 5641, 1870--1874.
[36]
S.-C. Liu and T. Delbruck. 2010. Neuromorphic sensory systems. Current Opinion Neurobiol. 20, 3, 288--295.
[37]
C. G. Mayr and J. Partzsch. 2010. Rate and pulse based plasticity governed by local synaptic state variables. Frontiers Synaptic Neurosci. 2, 33.
[38]
C. Mead. 1990. Neuromorphic electronic systems. Proc. IEEE 78, 10, 1629--36.
[39]
S. Mitra, S. Fusi, and G. Indiveri. 2009. Real-time classification of complex patterns using spike-based learning in neuro-morphic VLSI. IEEE Trans. Biomed. Circ. Syst. 3, 1, 32--42.
[40]
S. Moradi and G. Indiveri. 2011. A VLSI network of spiking neurons with an asynchronous static random access memory. In Proceedings of the Biomedical Circuits and Systems Conference (BioCAS'11). 277--280.
[41]
S. Moradi and G. Indiveri. 2014. An event-based neural network architecture with an asynchronous programmable synaptic memory. IEEE Trans. Biomed. Circ. Syst. 8, 1, 98--107.
[42]
S. Moradi, N. Imam, R. Manohar, and G. Indiveri. 2013. A memory-efficient routing method for large-scale spiking neural networks. In Proceedings of the European Conference on Circuit Theory and Design (ECCTD'13). 1--4.
[43]
J. M. Nageswaran, N. Dutt, J. L. Krichmar, A. Nicolau, and A. V. Veidenbaum. 2009. A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Netw. 22, 5--6, 791--800.
[44]
A. Nere, U. Olcese, D. Balduzzi, and G. Tononi. 2012. A neuromorphic architecture for object recognition and motion anticipation using burst-STDP. PloS ONE 7, 5.
[45]
J. P. Pfister and W. Gerstner. 2006. Triplets of spikes in a model of spike timing-dependent plasticity. J. Neurosci. 26, 38, 9673--9682.
[46]
P. Rowcliffe and J. Feng. 2008. Training spiking neuronal networks with applications in engineering tasks. IEEE Trans. Neural Netw. 19, 9, 1626--1640.
[47]
J. Schemmel, D. Bruderle, A. Grubl, M. Hock, K. Meier, and S. Millner. 2010. A wafer-scale neuromorphic hardware system for large-scale neural modeling. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS'10). 1947--1950.
[48]
M. Schmuker, T. Pfeil, and M. P. Nawrot. 2014. A neuromorphic network for generic multivariate data classification. Proc. Nat. Acad. Sci. 111, 6, 2081--2086.
[49]
P. J. Sjöström, E. A. Rancz, A. Roth, and M. Häusser. 2008. Dendritic excitability and synaptic plasticity. Physiol. Rev. 88, 2, 769--840.
[50]
S. Song, K. D. Miller, and L. F. Abbot. 2000. Competitive hebbian learning through spike-timing-dependent plasticity. Nature Neurosci. 3, 9, 919--926.
[51]
R. J. Vogelstein, U. Mallik, J. T. Vogelstein, and G. Cauwenberghs. 2007. Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses. IEEE Trans. Neural Netw. 18, 1, 253--265.
[52]
H. X. Wang, R. C. Gerkin, D. W. Nauen, and G. Q. Bi. 2005. Coactivation and timing-dependent integration of synaptic potentiation and depression. Nature Neurosci. 8, 2, 187--193.

Cited By

View all
  • (2024)Unsupervised end-to-end training with a self-defined targetNeuromorphic Computing and Engineering10.1088/2634-4386/ad8c784:4(044005)Online publication date: 21-Nov-2024
  • (2024)Unsupervised character recognition with graphene memristive synapsesNeural Computing and Applications10.1007/s00521-023-09135-236:4(1569-1584)Online publication date: 1-Feb-2024
  • (2023)Memristive Devices for Neuromorphic and Deep Learning ApplicationsAdvanced Memory Technology10.1039/BK9781839169946-00680(680-704)Online publication date: 9-Oct-2023
  • Show More Cited By

Index Terms

  1. Programmable Spike-Timing-Dependent Plasticity Learning Circuits in Neuromorphic VLSI Architectures

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Journal on Emerging Technologies in Computing Systems
      ACM Journal on Emerging Technologies in Computing Systems  Volume 12, Issue 2
      Special Issue on Advances in Design of Ultra-Low Power Circuits and Systems in Emerging Technologies
      August 2015
      191 pages
      ISSN:1550-4832
      EISSN:1550-4840
      DOI:10.1145/2820112
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Journal Family

      Publication History

      Published: 02 September 2015
      Accepted: 01 July 2014
      Revised: 01 May 2014
      Received: 01 December 2013
      Published in JETC Volume 12, Issue 2

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. AER
      2. STDP
      3. VLSI
      4. asynchronous
      5. emerging technologies
      6. learning
      7. neuromorphic
      8. plasticity
      9. realtime
      10. subthreshold

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Funding Sources

      • European Community's Seventh Framework Programme ERC

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)22
      • Downloads (Last 6 weeks)2
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Unsupervised end-to-end training with a self-defined targetNeuromorphic Computing and Engineering10.1088/2634-4386/ad8c784:4(044005)Online publication date: 21-Nov-2024
      • (2024)Unsupervised character recognition with graphene memristive synapsesNeural Computing and Applications10.1007/s00521-023-09135-236:4(1569-1584)Online publication date: 1-Feb-2024
      • (2023)Memristive Devices for Neuromorphic and Deep Learning ApplicationsAdvanced Memory Technology10.1039/BK9781839169946-00680(680-704)Online publication date: 9-Oct-2023
      • (2022)A Survey of Intelligent Chip Design Research Based on Spiking Neural NetworksIEEE Access10.1109/ACCESS.2022.320045410(89663-89686)Online publication date: 2022
      • (2020)Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical ApplicationsIEEE Transactions on Biomedical Circuits and Systems10.1109/TBCAS.2020.303608114:6(1138-1159)Online publication date: Dec-2020
      • (2020)Complementary Metal‐Oxide Semiconductor and Memristive Hardware for Neuromorphic ComputingAdvanced Intelligent Systems10.1002/aisy.2019001892:5Online publication date: 14-Apr-2020
      • (2019)CORDIC-SNN: On-FPGA STDP Learning With Izhikevich NeuronsIEEE Transactions on Circuits and Systems I: Regular Papers10.1109/TCSI.2019.289935666:7(2651-2661)Online publication date: Jul-2019
      • (2019)Efficient FPGA Implementations of Pair and Triplet-Based STDP for Neuromorphic ArchitecturesIEEE Transactions on Circuits and Systems I: Regular Papers10.1109/TCSI.2018.288175366:4(1558-1570)Online publication date: Apr-2019
      • (2019)Modeling triplet spike timing dependent plasticity using a hybrid TFT-memristor neuromorphic synapseIntegration10.1016/j.vlsi.2018.10.00464(184-191)Online publication date: Jan-2019
      • (2018)Neural and Synaptic Array Transceiver: A Brain-Inspired Computing Framework for Embedded LearningFrontiers in Neuroscience10.3389/fnins.2018.0058312Online publication date: 29-Aug-2018
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media