Skip to main content

Advertisement

SpringerLink
Log in

Analysis of Effect of Weight Variation on SNN Chip with PCM-Refresh Method

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Neuromorphic computing using analog non-volatile memory (NVM) devices has been the subject of various studies due to its potential ability to achieve extremely low power consumption less than that of traditional von Neumann architecture. However, using NVM devices, such as phase change memory (PCM) and resistive-RAM devices, presents various challenges, such as limitations in the number of conductance steps and device variability resulting from process variation and electro/thermo-dynamics. Limitations in the number of conductance steps and device variability could reduce the accuracy of neural network training. It is necessary to quantitatively analyze the effect of the number of conductance steps and synaptic device variability on the accuracy of neural network training and assess requirements for NVM devices to make NVM-based neuromorphic computing successful. We conducted the analysis using simulations focusing on a spiking neural network (SNN) based restricted Boltzmann machine (RBM) with PCM devices using the PCM-refresh method. The results of our quantitative simulation, which used the MNIST dataset, showed that having more than 500 conductance steps achieves comparable performance to that when there are more than 1000 conductance steps. We also found that less than 10% conductance update variation in the synaptic devices is required to achieve the comparable accuracy with the no variation case. These results can provide guidelines for designing and optimizing a synaptic device for realizing NVM-based neuromorphic computing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Burr GW, Shelby RM, Sebastian A, Kim S, Kim S, Sidler S, Virwani K, Ishii M, Narayanan P, Fumarola A et al (2017) Neuromorphic computing using non-volatile memory. Adv Phys X 2(1):89

    Google Scholar 

  2. Kim S, Ishii M, Lewis S, Perri T, BrightSky M, Kim W, Jordan R, Burr G, Sosa N, Ray A et al (2015) NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning, In: 2015 IEEE international electron devices meeting (IEDM). (IEEE), pp 17.1.1–17.1.4

  3. Merolla PA, Arthur JV, Alvarez-Icaza R, Cassidy AS, Sawada J, Akopyan F, Jackson BL, Imam N, Guo C, Nakamura Y et al (2014) A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197):668

    Article  Google Scholar 

  4. Davies M, Srinivasa N, Lin TH, Chinya G, Cao Y, Choday SH, Dimou G, Joshi P, Imam N, Jain S et al (2018) Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1):82

    Article  Google Scholar 

  5. Ito M, Ishii M, Okazaki A, Kim S, Okazawa J, Nomura A, Hosokawa K, Haensch W (2018) Lightweight refresh method for PCM-based neuromorphic circuits. In: 2018 IEEE 18th international conference on nanotechnology (IEEE-NANO). (IEEE), pp 1–4

  6. Bichler O, Suri M, Querlioz D, Vuillaume D, DeSalvo B, Gamrat C (2012) Visual pattern extraction using energy-efficient “2-PCM Synapse” neuromorphic architecture. IEEE Trans Electron Devices 59(8):2206

    Article  Google Scholar 

  7. Zhang W, Li T (2009) Characterizing and mitigating the impact of process variations on phase change based memory systems. In: 2009 42nd Annual IEEE/ACM international symposium on microarchitecture (MICRO). (IEEE), pp 2–13

  8. Suri M, Bichler O, Querlioz D, Cueto O, Perniola L, Sousa V, Vuillaume D, Gamrat C, DeSalvo B (2011) Phase change memory as synapse for ultra-dense neuromorphic systems: application to complex visual pattern extraction. In: 2011 IEEE international electron devices meeting (IEDM). (IEEE), pp 4.4.1–4.4.4

  9. Kuzum D, Jeyasingh RG, Lee B, Wong HSP (2012) Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett 12(5):2179

    Article  Google Scholar 

  10. Burr GW, Shelby RM, Sidler S, Di Nolfo C, Jang J, Boybat I, Shenoy RS, Narayanan P, Virwani K, Giacometti EU et al (2015) Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses) using phase-change memory as the synaptic weight element. IEEE Trans Electron Devices 62(11):3498

    Article  Google Scholar 

  11. Ielmini D (2011) Modeling the universal set/reset characteristics of bipolar RRAM by field-and temperature-driven filament growth. IEEE Trans Electron Devices 58(12):4309

    Article  Google Scholar 

  12. Neftci E, Das S, Pedroni B, Kreutz-Delgado K, Cauwenberghs G (2014) Event-driven contrastive divergence for spiking neuromorphic systems. Front Neurosci 7:272

    Article  Google Scholar 

  13. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278

    Article  Google Scholar 

  14. Nomura A, Ito M, Okazaki A, Ishii M, Kim S, Okazawa J, Hosokawa K, Haensch W (2018) NVM weight variation impact on analog spiking neural network chip. In International conference on neural information processing. (Springer), pp 676–685

  15. Gokmen T, Vlasov Y (2016) Acceleration of deep neural network training with resistive cross-point devices: design considerations. Front Neurosci 10:333

    Article  Google Scholar 

  16. Eryilmaz SB, Kuzum D, Yu S, Wong HSP (2015) Device and system level design considerations for analog-non-volatile-memory based neuromorphic architectures. In: 2015 IEEE international electron devices meeting (IEDM). (IEEE), pp 4.1.1–1.1.4

Download references

Author information

Authors and Affiliations

  1. IBM Research - Tokyo, Kawasaki, Kanagawa, Japan

    Akiyo Nomura, Megumi Ito, Atsuya Okazaki, Masatoshi Ishii, Junka Okazawa & Kohji Hosokawa

  2. Seoul National University, Seoul, Korea

    Sangbum Kim

  3. IBM Research, Yorktown Heights, NY, USA

    Wilfried Haensch

Authors
  1. Akiyo Nomura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Megumi Ito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Atsuya Okazaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masatoshi Ishii
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sangbum Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Junka Okazawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kohji Hosokawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wilfried Haensch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akiyo Nomura.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nomura, A., Ito, M., Okazaki, A. et al. Analysis of Effect of Weight Variation on SNN Chip with PCM-Refresh Method. Neural Process Lett 53, 1741–1751 (2021). https://doi.org/10.1007/s11063-019-10139-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10139-0

Keywords

Search

Navigation