research-article

Public Access

RESPARC: A Reconfigurable and Energy-Efficient Architecture with Memristive Crossbars for Deep Spiking Neural Networks

Authors:

Abhronil Sengupta,

Priyadarshini Panda,

Kaushik RoyAuthors Info & Claims

DAC '17: Proceedings of the 54th Annual Design Automation Conference 2017

Article No.: 27, Pages 1 - 6

https://doi.org/10.1145/3061639.3062311

Published: 18 June 2017 Publication History

Abstract

Neuromorphic computing using post-CMOS technologies is gaining immense popularity due to its promising abilities to address the memory and power bottlenecks in von-Neumann computing systems. In this paper, we propose RESPARC - a reconfigurable and energy efficient architecture built-on Memristive Crossbar Arrays (MCA) for deep Spiking Neural Networks (SNNs). Prior works were primarily focused on device and circuit implementations of SNNs on crossbars. RESPARC advances this by proposing a complete system for SNN acceleration and its subsequent analysis. RESPARC utilizes the energy-efficiency of MCAs for inner-product computation and realizes a hierarchical reconfigurable design to incorporate the data-flow patterns in an SNN in a scalable fashion. We evaluate the proposed architecture on different SNNs ranging in complexity from 2k-230k neurons and 1.2M-5.5M synapses. Simulation results on these networks show that compared to the baseline digital CMOS architecture, RESPARC achieves 500x (15x) efficiency in energy benefits at 300x (60x) higher throughput for multi-layer perceptrons (deep convolutional networks). Furthermore, RESPARC is a technology-aware architecture that maps a given SNN topology to the most optimized MCA size for the given crossbar technology.

References

[1]

A. Krizhevsky et al. Imagenet classification with deep convolutional neural networks. NIPS, 2012.

Digital Library

[2]

S. Chetlur et al. cudnn: Efficient primitives for deep learning. arXiv:1410.0759, 2014.

[3]

T. Chen et al. Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning. ACM Sigplan Notices, 2014.

Digital Library

[4]

P. U. Diehl et al. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. IJCNN, 2015.

[5]

F. Akopyan et al. Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. TCAD, 2015.

[6]

S. H. Jo et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano letters, 2010.

[7]

M. Prezioso et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature, 2015.

[8]

X. Liu et al. Reno: A high-efficient reconfigurable neuromorphic computing accelerator design. DAC, 2015.

Digital Library

[9]

B. L. Jackson et al. Nanoscale electronic synapses using phase change devices. JETC, 2013.

Digital Library

[10]

A. Sengupta et al. Proposal for an all-spin artificial neural network: Emulating neural and synaptic functionalities through domain wall motion in ferromagnets. TBioCAS, 2016.

[11]

J. Liang et al. Cross-point memory array without cell selectors-device characteristics and data storage pattern dependencies. TED, 2010.

[12]

B. Liu et al. Vortex: variation-aware training for memristor x-bar. DAC, 2015.

Digital Library

[13]

P. Chi et al. Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory. ISCA, 2016.

Digital Library

[14]

A. Shafiee et al. Isaac: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. ISCA, 2016.

Digital Library

[15]

P. Panda at al. Falcon: Feature driven selective classification for energy-efficient image recognition. arXiv:1609.03396, 2016.

[16]

B. Rajendran et al. Specifications of nanoscale devices and circuits for neuromorphic computational systems. TED, 2013.

[17]

A. Joubert et al. Hardware spiking neurons design: Analog or digital? IJCNN, 2012.

[18]

N. Muralimanohar et al. Optimizing nuca organizations and wiring alternatives for large caches with cacti 6.0. MICRO, 2007.

Digital Library

[19]

Y. Netzer et al. Reading digits in natural images with unsupervised feature learning. 2011.

[20]

Y. LeCun et al. Gradient-based learning applied to document recognition. IEEE, 1998.

[21]

A. Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.

Cited By

Lofù DSorino PDi Noia TDi Sciascio E(2024)Towards a Federated Intrusion Detection System based on Neuromorphic Computing2024 9th International Conference on Smart and Sustainable Technologies (SpliTech)10.23919/SpliTech61897.2024.10612534(1-5)Online publication date: 25-Jun-2024
https://doi.org/10.23919/SpliTech61897.2024.10612534
Chowdhury SKosta ASharma DApolinario MRoy K(2024)Unearthing the Potential of Spiking Neural Networks2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546699(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546699
Mustafazade IKandasamy NDas A(2024)Clustering and Allocation of Spiking Neural Networks on Crossbar-Based Neuromorphic ArchitectureProceedings of the 21st ACM International Conference on Computing Frontiers10.1145/3649153.3649199(164-171)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3649153.3649199
Show More Cited By

Index Terms

RESPARC: A Reconfigurable and Energy-Efficient Architecture with Memristive Crossbars for Deep Spiking Neural Networks
1. Hardware
  1. Emerging technologies
    1. Analysis and design of emerging devices and systems
      1. Emerging architectures

Recommendations

Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars
ISLPED '22: Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design

Spiking Neural Networks (SNNs) have recently emerged as the low-power alternative to Artificial Neural Networks (ANNs) owing to their asynchronous, sparse, and binary information processing. To improve the energy-efficiency and throughput, SNNs can be ...
TraNNsformer: Neural Network Transformation for memristive crossbar based neuromorphic system design
ICCAD '17: Proceedings of the 36th International Conference on Computer-Aided Design

Implementation of Neuromorphic Systems using post Complementary Metal-Oxide-Semiconductor (CMOS) technology based Memristive Crossbar Array (MCA) has emerged as a promising solution to enable low-power acceleration of neural networks. However, the ...
Improved integrate-and-fire neuron models for inference acceleration of spiking neural networks
Abstract
We study the effects of different bio-synaptic membrane potential mechanisms on the inference speed of both spiking feed-forward neural networks and spiking convolutional neural networks. These mechanisms are inspired by biological neuron ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DAC '17: Proceedings of the 54th Annual Design Automation Conference 2017

June 2017

533 pages

ISBN:9781450349277

DOI:10.1145/3061639

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

EDAC: Electronic Design Automation Consortium
SIGDA: ACM Special Interest Group on Design Automation
IEEE-CEDA

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation
Vannevar Bush Faculty Fellowship
Semiconductor Research Corporation
Intel Corporation
Center for Spintronic Materials, Interfaces and Novel Architectures (C-SPIN), a MARCO and DARPA sponsored StarNet center

Conference

DAC '17

Sponsor:

EDAC
SIGDA

DAC '17: The 54th Annual Design Automation Conference 2017

June 18 - 22, 2017

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Upcoming Conference

DAC '25

Sponsor:
sigda

62nd ACM/IEEE Design Automation Conference

June 22 - 26, 2025

San Francisco , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

74
Total Citations
View Citations
1,344
Total Downloads

Downloads (Last 12 months)187
Downloads (Last 6 weeks)27

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lofù DSorino PDi Noia TDi Sciascio E(2024)Towards a Federated Intrusion Detection System based on Neuromorphic Computing2024 9th International Conference on Smart and Sustainable Technologies (SpliTech)10.23919/SpliTech61897.2024.10612534(1-5)Online publication date: 25-Jun-2024
https://doi.org/10.23919/SpliTech61897.2024.10612534
Chowdhury SKosta ASharma DApolinario MRoy K(2024)Unearthing the Potential of Spiking Neural Networks2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546699(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546699
Mustafazade IKandasamy NDas A(2024)Clustering and Allocation of Spiking Neural Networks on Crossbar-Based Neuromorphic ArchitectureProceedings of the 21st ACM International Conference on Computing Frontiers10.1145/3649153.3649199(164-171)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3649153.3649199
Li ZZheng QKu JTaylor BLi H(2024)TFSRAM: A 249.8TOPS/W Timing-to-First-Spike Compute-in-Memory Neuromorphic Processing Engine With Twin-Column SRAM SynapsesIEEE Transactions on Circuits and Systems for Artificial Intelligence10.1109/TCASAI.2024.34526491:1(26-36)Online publication date: Sep-2024
https://doi.org/10.1109/TCASAI.2024.3452649
Mustafazade IDas AKandasamy N(2024)Efficient Built-In Self-Test Strategy for Neuromorphic Hardware Based On Alarm Placement2024 IEEE 29th Pacific Rim International Symposium on Dependable Computing (PRDC)10.1109/PRDC63035.2024.00014(1-10)Online publication date: 13-Nov-2024
https://doi.org/10.1109/PRDC63035.2024.00014
Bhattacharjee AYin RMoitra APanda P(2024)Are SNNs Truly Energy-efficient? — A Hardware PerspectiveICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10448269(13311-13315)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10448269
Saha AAmarnath CMa KChatterjee A(2024)Signature Driven Post-Manufacture Testing and Tuning of RRAM Spiking Neural Networks for Yield Recovery2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC)10.1109/ASP-DAC58780.2024.10473874(740-745)Online publication date: 22-Jan-2024
https://doi.org/10.1109/ASP-DAC58780.2024.10473874
AbuHamra NMohammad B(2024)Memory-Centric Computing for Image Classification Using SNN with RRAM2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS)10.1109/AICAS59952.2024.10595912(105-109)Online publication date: 22-Apr-2024
https://doi.org/10.1109/AICAS59952.2024.10595912
Kim YBaek JIm ILee DPark MJang H(2024)Two-Terminal Neuromorphic Devices for Spiking Neural Networks: Neurons, Synapses, and Array IntegrationACS Nano10.1021/acsnano.4c12884Online publication date: 12-Dec-2024
https://doi.org/10.1021/acsnano.4c12884
Peng HGan LGuo XPeng HGan LGuo X(2024)Memristor based Spiking Neural Networks: Cooperative Development of Neural Network Architecture/Algorithms and MemristorsChip10.1016/j.chip.2024.100093(100093)Online publication date: Apr-2024
https://doi.org/10.1016/j.chip.2024.100093
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten