skip to main content
10.1145/3566097.3568359acmconferencesArticle/Chapter ViewAbstractPublication PagesaspdacConference Proceedingsconference-collections
invited-talk
Public Access

Hardware-Software Co-Design for On-Chip Learning in AI Systems

Published: 31 January 2023 Publication History

Abstract

Spike-based convolutional neural networks (CNNs) are empowered with on-chip learning in their convolution layers, enabling the layer to learn to detect features by combining those extracted in the previous layer. We propose ECHELON, a generalized design template for a tile-based neuromorphic hardware with on-chip learning capabilities. Each tile in ECHELON consists of a neural processing units (NPU) to implement convolution and dense layers of a CNN model, an on-chip learning unit (OLU) to facilitate spike-timing dependent plasticity (STDP) in the convolution layer, and a special function unit (SFU) to implement other CNN functions such as pooling, concatenation, and residual computation. These tile resources are interconnected using a shared bus, which is segmented and configured via the software to facilitate parallel communication inside the tile. Tiles are themselves interconnected using a classical Network-on-Chip (NoC) interconnect. We propose a system software to map CNN models to ECHELON, maximizing the performance. We integrate the hardware design and software optimization within a co-design loop to obtain the hardware and software architectures for a target CNN, satisfying both performance and resource constraints. In this preliminary work, we show the implementation of a tile on a FPGA and some early evaluations. Using 8 STDP-enabled CNN models, we show the potential of our co-design methodology to optimize hardware resources.

References

[1]
Adarsha Balaji, Anup Das, Yuefeng Wu, Khanh Huynh, Francesco G Dell'Anna, Giacomo Indiveri, Jeffrey L Krichmar, Nikil D Dutt, Siebren Schaafsma, et al. 2019. Mapping spiking neural networks to neuromorphic hardware. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 28, 1 (2019), 76--86.
[2]
Adarsha Balaji, Phu Huynh, Francky Catthoor, Nikil D. Dutt, Jeffrey Krichmar, and Anup Das. 2023. NeuSB: A scalable interconnect architecture for spiking neuromorphic hardware. arXiv (2023).
[3]
Adarsha Balaji, Shihao Song, Twisha Titirsha, Anup Das, Jeffrey Krichmar, Nikil Dutt, James Shackleford, Nagarajan Kandasamy, et al. 2021. NeuroXplorer 1.0: An extensible framework for architectural exploration with spiking neural networks. In International Conference on Neuromorphic Systems (ICONS).
[4]
Adarsha Balaji, Yuefeng Wu, Anup Das, Francky Catthoor, and Siebren Schaafsma. 2019. Exploration of segmented bus as scalable global interconnect for neuromorphic computing. In Great Lakes Symposium on VLSI.
[5]
Ben Varkey Benjamin, Peiran Gao, Emmett McQuinn, Swadesh Choudhary, Anand R Chandrasekaran, Jean-Marie Bussat, Rodrigo Alvarez-Icaza, John V Arthur, et al. 2014. Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proc. of the IEEE 102, 5 (2014), 699--716.
[6]
Geoffrey W. Burr, Robert M. Shelby, Abu Sebastian, Sangbum Kim, Seyoung Kim, Severin Sidler, Kumar Virwani, Masatoshi Ishii, Pritish Narayanan, Alessandro Fumarola, Lucas L. Sanches, Irem Boybat, Manuel Le Gallo, Kibong Moon, Jiyoo Woo, Hyunsang Hwang, and Yusuf Leblebici. 2017. Neuromorphic computing using non-volatile memory. Advances in Physics: X 2, 1 (2017), 89--124.
[7]
Natalia Caporale, Yang Dan, et al. 2008. Spike timing-dependent plasticity: a Hebbian learning rule. Annual Review of Neuroscience 31, 1 (2008), 25--46.
[8]
Francky Catthoor, Srinjoy Mitra, Anup Das, and Siebren Schaafsma. 2018. Very large-scale neuromorphic systems for biological signal processing. In CMOS Circuits for Biological Sensing and Processing. Springer, 315--340.
[9]
Federico Corradi, Guido Adriaans, and Sander Stuijk. 2021. Gyro: A digital spiking neural network architecture for multi-sensory data analytics. In Drone Systems Engineering and Rapid Simulation and Performance Evaluation: Methods and Tools Proceedings. 9--15.
[10]
Gaël Daoudal and Dominique Debanne. 2003. Long-term plasticity of intrinsic excitability: learning rules and mechanisms. Learning & Memory 10, 6 (2003).
[11]
Anup Das. 2022. Real-Time Scheduling of Machine Learning Operations on Heterogeneous Neuromorphic SoC. In International Symposium on Formal Methods and Models for System Design (MEMOCODE).
[12]
Anup Das, Akash Kumar, and Bharadwaj Veeravalli. 2014. Energy-aware task mapping and scheduling for reliable embedded computing systems. ACM Transactions on Embedded Computing Systems 13, 2s (2014), 1--27.
[13]
Anup Das, Paruthi Pradhapan, Willemijn Groenendaal, Prathyusha Adiraju, Raj Thilak Rajan, Francky Catthoor, Siebren Schaafsma, Jeffrey L Krichmar, Nikil Dutt, and Chris Van Hoof. 2018. Unsupervised heart-rate estimation in wearables with liquid states and a probabilistic readout. Neural Networks 99 (2018), 134--147.
[14]
Mike Davies, Narayan Srinivasa, Tsung Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, Yuyun Liao, Chit Kwan Lin, Andrew Lines, Ruokun Liu, Deepak Mathaikutty, Steven McCoy, Arnab Paul, Jonathan Tse, Guruguhanathan Venkataramanan, Yi Hsin Weng, Andreas Wild, et al. 2018. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro 38, 1 (2018), 82--99.
[15]
Michael V. Debole, Brian Taba, Arnon Amir, et al. 2019. TrueNorth: Accelerating from zero to 64 million neurons in 10 years. Computer 52, 5 (2019), 20--29.
[16]
Steve Furber, Francesco Galluppi, Steve Temple, and Luis A Plana. 2014. The SpiNNaker project. Proc. of the IEEE 102, 5 (2014), 652--665.
[17]
Eric Hunsberger and Chris Eliasmith. 2015. Spiking deep networks with LIF neurons. arXiv (2015).
[18]
Phu Khanh Huynh, M Lakshmi Varshika, Ankita Paul, Murat Isik, Adarsha Balaji, and Anup Das. 2022. Implementing spiking neural networks on neuromorphic architectures: A review. arXiv (2022).
[19]
Yu Ji, YouHui Zhang, ShuangChen Li, Ping Chi, CiHang Jiang, Peng Qu, Yuan Xie, and WenGuang Chen. 2016. NEUTRAMS: Neural network transformation and co-design under neuromorphic hardware constraints. In International Symposium on Microarchitecture (MICRO). 1--13.
[20]
Xin Jin, Alexander Rast, Francesco Galluppi, Sergio Davies, and Steve Furber. 2010. Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware. In International Joint Conference on Neural Networks (IJCNN).
[21]
Saeed Reza Kheradpisheh, Mohammad Ganjtabesh, Simon J Thorpe, and Timothée Masquelier. 2018. STDP-based spiking deep convolutional neural networks for object recognition. Neural Networks 99 (2018), 56--67.
[22]
M Lakshmi Varshika, Adarsha Balaji, Federico Corradi, Anup Das, Jan Stuijt, and Francky Catthoor. 2022. Design of Many-Core Big Little μBrain for Energy-Efficient Embedded Neuromorphic Computing. In Design, Automation & Test in Europe Conference & Exhibition (DATE).
[23]
Corey Lammie, Tara Julia Hamilton, André van Schaik, and Mostafa Rahimi Azghadi. 2018. Efficient FPGA implementations of pair and triplet-based STDP for neuromorphic architectures. IEEE Transactions on Circuits and Systems I: Regular Papers 66, 4 (2018), 1558--1570.
[24]
Edward A Lee and David G Messerschmitt. 1987. Synchronous data flow. Proc. of the IEEE 75, 9 (1987), 1235--1245.
[25]
Chit-Kwan Lin, Andreas Wild, Gautham N. Chinya, Tsung-Han Lin, Mike Davies, and Hong Wang. 2018. Mapping Spiking Neural Networks onto a Manycore Neuromorphic Architecture. In Conference on Programming Language Design and Implementation (PLDI). 78--89.
[26]
Xiaoxiao Liu, Wei Wen, Xuehai Qian, Hai Li, and Yiran Chen. 2018. Neu-NoC: A high-efficient interconnection network for accelerated neuromorphic systems. In Asia and South Pacific Design Automation Conference (ASP-DAC). 141--146.
[27]
Carver Mead. 1990. Neuromorphic electronic systems. Proc. of the IEEE 78, 10 (1990), 1629--1636.
[28]
Lars Niedermeier, Kexin Chen, Jinwei Xing, Anup Das, Jeffrey Kopsick, Eric Scott, Nate Sutton, Killian Weber, Nikil Dutt, and Jeffrey L Krichmar. 2022. CARLsim 6: An Open Source Library for Large-Scale, Biologically Detailed Spiking Neural Network Simulation. In International Joint Conference on Neural Networks.
[29]
Ning Qiao, Hesham Mostafa, Federico Corradi, Marc Osswald, Fabio Stefanini, Dora Sumislawska, and Giacomo Indiveri. 2015. A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses. Frontiers in Neuroscience 9 (2015), 141.
[30]
Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. 2019. Going deeper in spiking neural networks: VGG and residual architectures. Frontiers in Neuroscience 13 (2019), 95.
[31]
Teresa Serrano-Gotarredona, Timothée Masquelier, Themistoklis Prodromakis, Giacomo Indiveri, and Bernabe Linares-Barranco. 2013. STDP and STDP variations with memristors for spiking neuromorphic learning systems. Frontiers in Neuroscience 7 (2013), 2.
[32]
Luping Shi, Jing Pei, Ning Deng, Dong Wang, Lei Deng, Yu Wang, Youhui Zhang, Feng Chen, Mingguo Zhao, Sen Song, et al. 2015. Development of a neuromorphic computing system. In International Electron Devices Meeting (IEDM). 4--3.
[33]
Shihao Song, Adarsha Balaji, Anup Das, and Nagarajan Kandasamy. 2022. Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements. ACM Transactions on Embedded Computing Systems (TECS) (2022).
[34]
Shihao Song, Adarsha Balaji, Anup Das, Nagarajan Kandasamy, et al. 2020. Compiling spiking neural networks to neuromorphic hardware. In International Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES).
[35]
Shihao Song, Lakshmi Mirtinti, Anup Das, and N. Kandasamy. 2021. A Design Flow for Mapping Spiking Neural Networks to Many-Core Neuromorphic Hardware. In International Conference On Computer Aided Design (ICCAD). 1--9.
[36]
Amirreza Yousefzadeh, Timothée Masquelier, Teresa Serrano-Gotarredona, and Bernabé Linares-Barranco. 2017. Hardware implementation of convolutional STDP for on-line visual feature learning. In International Symposium on Circuits and Systems (ISCAS).
[37]
Robert S Zucker, Wade G Regehr, et al. 2002. Short-term synaptic plasticity. Annual Review of Physiology 64, 1 (2002), 355--405.

Cited By

View all
  • (2024)CMOS-Memristor Hybrid Design of A Neuromorphic Crossbar Array with Integrated Inference and Training2024 IEEE 67th International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS60917.2024.10658962(442-446)Online publication date: 11-Aug-2024
  • (2023)Design of a Tunable Astrocyte Neuromorphic Circuitry with Adaptable Fault Tolerance2023 IEEE 66th International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS57524.2023.10405978(904-908)Online publication date: 6-Aug-2023
  • (2023)DesignSystemsJS - Building a Design Systems API for aiding standardization and AI integration2023 International Conference on Computing, Networking, Telecommunications & Engineering Sciences Applications (CoNTESA)10.1109/CoNTESA61248.2023.10384889(83-89)Online publication date: 14-Dec-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ASPDAC '23: Proceedings of the 28th Asia and South Pacific Design Automation Conference
January 2023
807 pages
ISBN:9781450397834
DOI:10.1145/3566097
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

In-Cooperation

  • IPSJ
  • IEEE CAS
  • IEEE CEDA
  • IEICE

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 January 2023

Check for updates

Author Tags

  1. FPGA
  2. neuromorphic computing
  3. spike timing dependent plasticity (STDP)
  4. spiking neural network (SNN)

Qualifiers

  • Invited-talk

Funding Sources

  • US Department of Energy
  • US National Science Foundation

Conference

ASPDAC '23
Sponsor:

Acceptance Rates

ASPDAC '23 Paper Acceptance Rate 102 of 328 submissions, 31%;
Overall Acceptance Rate 466 of 1,454 submissions, 32%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)265
  • Downloads (Last 6 weeks)20
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)CMOS-Memristor Hybrid Design of A Neuromorphic Crossbar Array with Integrated Inference and Training2024 IEEE 67th International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS60917.2024.10658962(442-446)Online publication date: 11-Aug-2024
  • (2023)Design of a Tunable Astrocyte Neuromorphic Circuitry with Adaptable Fault Tolerance2023 IEEE 66th International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS57524.2023.10405978(904-908)Online publication date: 6-Aug-2023
  • (2023)DesignSystemsJS - Building a Design Systems API for aiding standardization and AI integration2023 International Conference on Computing, Networking, Telecommunications & Engineering Sciences Applications (CoNTESA)10.1109/CoNTESA61248.2023.10384889(83-89)Online publication date: 14-Dec-2023
  • (2023)Fault-Tolerant Spiking Neural Network Mapping Algorithm and Architecture to 3D-NoC-Based Neuromorphic SystemsIEEE Access10.1109/ACCESS.2023.3278802(1-1)Online publication date: 2023
  • (undefined)A Design Flow for Scheduling Spiking Deep Convolutional Neural Networks on Heterogeneous Neuromorphic System-on-ChipACM Transactions on Embedded Computing Systems10.1145/3635032

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media