skip to main content
research-article

DL-RSIM: A Reliability and Deployment Strategy Simulation Framework for ReRAM-based CNN Accelerators

Published: 28 May 2022 Publication History

Abstract

Memristor-based deep learning accelerators provide a promising solution to improve the energy efficiency of neuromorphic computing systems. However, the electrical properties and crossbar structure of memristors make these accelerators error-prone. In addition, due to the hardware constraints, the way to deploy neural network models on memristor crossbar arrays affects the computation parallelism and communication overheads. To enable reliable and energy-efficient memristor-based accelerators, a simulation platform is needed to precisely analyze the impact of non-ideal circuit/device properties on the inference accuracy and the influence of different deployment strategies on performance and energy consumption. In this paper, we propose a flexible simulation framework, DL-RSIM, to tackle this challenge. A rich set of reliability impact factors and deployment strategies are explored by DL-RSIM, and it can be incorporated with any deep learning neural networks implemented by TensorFlow. Using several representative convolutional neural networks as case studies, we show that DL-RSIM can guide chip designers to choose a reliability-friendly design option and energy-efficient deployment strategies and develop optimization techniques accordingly.

References

[1]
K. Alex. 2012. Learning multiple layers of features from tiny images. University of Toronto (2012).
[2]
R. Balasubramonian, A. B. Kahng, N. Muralimanohar, A. Shafiee, and V. Srinivas. 2017. CACTI 7: New tools for interconnect exploration in innovative off-chip memories. ACM Trans. Archit. Code Optim. 14, 2, Article 14 (2017), 25 pages.
[3]
M. N. Bojnordi and E. Ipek. 2016. Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning. In IEEE International Symposium on High Performance Computer Architecture (HPCA). 1–13.
[4]
I. Chakraborty, M. Fayez Ali, D. Eun Kim, A. Ankit, and K. Roy. 2020. GENIEx: A Generalized Approach to Emulating Non-Ideality in Memristive Xbars using Neural Networks. 6 pages.
[5]
P.-Y. Chen, X. Peng, and S. Yu. 2018. NeuroSim: A circuit-level macro model for benchmarking neuro-inspired architectures in online learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD) 37, 12 (2018), 3067–3080.
[6]
W.-H. Chen, K.-X. Li, W.-Y. Lin, K.-H. Hsu, P.-Y. Li, C.-H. Yang, C.-X. Xue, E.-Y. Yang, Y.-K. Chen, Y.-S. Chang, T.-H. Hsu, Y.-C. King, C.-J. Lin, R.-S. Liu, C.-C. Hsieh, K.-T. Tang, and M.-F. Chang. 2018. A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors. In IEEE International Solid - State Circuits Conference (ISSCC). 494–496.
[7]
P. Chi, S. Li, C. Xu, T. Zhang, J. Zhao, Y. Liu, Y. Wang, and Y. Xie. 2016. PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM-based main memory. In ACM/IEEE International Symposium on Computer Architecture (ISCA). 27–39.
[8]
J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei.2009. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 248–255.
[9]
H. Esmaeilzadeh, A. Sampson, L. Ceze, and D. Burger. 2012. Neural acceleration for general-purpose approximate programs. In IEEE/ACM International Symposium on Microarchitecture (MICRO). 449–460.
[10]
B. Feinberg, S. Wang, and E. Ipek. 2018. Making memristive neural network accelerators reliable. In IEEE International Symposium on High Performance Computer Architecture (HPCA). 52–65.
[11]
K. C. Hsu, F. M. Lee, Y. Y. Lin, E. K. Lai, J. Y. Wu, D. Y. Lee, M. H. Lee, H. L. Lung, K. Y. Hsieh, and C. Y. Lu. 2015. A study of array resistance distribution and a novel operation algorithm for WOx ReRAM memory. In International Conference on Solid State Devices and Materials (SSDM). 1168–1169.
[12]
M. Hu, J. P. Strachan, Z. Li, E. M. Grafals, N. Davila, C. Graves, S. Lam, N. Ge, J. J. Yang, and R. S. Williams. 2016. Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication. In ACM/IEEE Design Automation Conference (DAC). 1–6.
[13]
Y. Ji, Y. Zhang, X. Xie, S. Li, P. Wang, X. Hu, Y. Zhang, and Y. Xie. 2019. FPSA: A full system stack solution for reconfigurable ReRAM-Based NN accelerator architecture. In International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). 733–747.
[14]
S. W. Keckler, W. J. Dally, B. Khailany, M. Garland, and D. Glasco. 2011. GPUs and the future of parallel computing. IEEE Micro 31, 5 (2011), 7–17.
[15]
A. Krizhevsky, I. Sutskever, and G. E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS).
[16]
C. Lammie and M. R. Azghadi. 2020. MemTorch: An open-source simulation framework for memristive deep learning systems. In IEEE International Symposium on Circuits and Systems (ISCAS). 1–5.
[17]
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.
[18]
M. K. F. Lee, Y. Cui, T. Somu, T. Luo, J. Zhou, W. T. Tang, W.-F. Wong, and R. S. M. Goh. 2019. A system-level simulator for RRAM-based neuromorphic computing chips. ACM Trans. Archit. Code Optim. 15, 4, Article 64 (2019), 24 pages.
[19]
B. Li, Y. Wang, and Y. Chen. 2020. HitM: High-throughput ReRAM-based PIM for multi-modal neural networks. In IEEE/ACM International Conference On Computer Aided Design (ICCAD). 1–7.
[20]
M. Saberi, R. Lotfi, K. Mafinezhad, and W. A. Serdijn. 2011. Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation ADCs. IEEE Transactions on Circuits and Systems I: Regular Papers 58, 8 (2011), 1736–1748.
[21]
P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. 2014. OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks. arxiv:cs.CV/1312.6229.
[22]
A. Shafiee, A. Nag, N. Muralimanohar, R. Balasubramonian, J. P. Strachan, M. Hu, R. S. Williams, and V. Srikumar. 2016. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In ACM/IEEE International Symposium on Computer Architecture (ISCA). 14–26.
[23]
F. Su, W.-H. Chen, L. Xia, C.-P. Lo, T. Tang, Z. Wang, K.-H. Hsu, M. Cheng, J.-Y. Li, Y. Xie, Y. Wang, M.-F. Chang, H. Yang, and Y. Liu. 2017. A 462GOPs/J RRAM-based nonvolatile intelligent processor for energy harvesting IoE system featuring nonvolatile logics and processing-in-memory. In Symposium on VLSI Technology (VLSIT). T260–T261.
[24]
Y. Sun, X. Wang, and X. Tang. 2014. Deep learning face representation from predicting 10,000 classes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1891–1898.
[25]
H.-S. P. Wong, H.-Y. Lee, S. Yu, Y.-S. Chen, Y. Wu, P.-S. Chen, B. Lee, F. T. Chen, and M.-J. Tsai. 2012. Metal-oxide RRAM. Proc. IEEE 100, 6 (2012), 1951–1970.
[26]
Y. N. Wu, V. Sze, and J. S. Emer. 2020. An architecture-level energy and area estimator for processing-in-memory accelerator designs. In IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 116–118.
[27]
L. Xia, B. Li, T. Tang, P. Gu, P.-Y. Chen, S. Yu, Y. Cao, Y. Wang, Y. Xie, and H. Yang. 2018. MNSIM: Simulation platform for memristor-based neuromorphic computing system. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD) 37, 5 (2018), 1009–1022.

Cited By

View all
  • (2024)Side-Channel Attack Analysis on In-Memory Computing ArchitecturesIEEE Transactions on Emerging Topics in Computing10.1109/TETC.2023.325768412:1(109-121)Online publication date: Jan-2024
  • (2024)PointCIM: A Computing-in-Memory Architecture for Accelerating Deep Point Cloud Analytics2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00097(1309-1322)Online publication date: 2-Nov-2024
  • (2024)ReAIM: A ReRAM-based Adaptive Ising Machine for Solving Combinatorial Optimization Problems2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00015(58-72)Online publication date: 29-Jun-2024
  • Show More Cited By

Index Terms

  1. DL-RSIM: A Reliability and Deployment Strategy Simulation Framework for ReRAM-based CNN Accelerators

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Embedded Computing Systems
      ACM Transactions on Embedded Computing Systems  Volume 21, Issue 3
      May 2022
      365 pages
      ISSN:1539-9087
      EISSN:1558-3465
      DOI:10.1145/3530307
      • Editor:
      • Tulika Mitra
      Issue’s Table of Contents

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Journal Family

      Publication History

      Published: 28 May 2022
      Online AM: 31 January 2022
      Accepted: 01 December 2021
      Revised: 01 October 2021
      Received: 01 February 2021
      Published in TECS Volume 21, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Simulation framework
      2. processing-in-memory
      3. resistive random access memory
      4. deep learning accelerator
      5. reliability
      6. energy efficiency

      Qualifiers

      • Research-article
      • Refereed

      Funding Sources

      • Ministry of Science and Technology of Taiwan
      • Delta Electronics
      • Macronix Inc., Hsin-chu, Taiwan

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)222
      • Downloads (Last 6 weeks)17
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Side-Channel Attack Analysis on In-Memory Computing ArchitecturesIEEE Transactions on Emerging Topics in Computing10.1109/TETC.2023.325768412:1(109-121)Online publication date: Jan-2024
      • (2024)PointCIM: A Computing-in-Memory Architecture for Accelerating Deep Point Cloud Analytics2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00097(1309-1322)Online publication date: 2-Nov-2024
      • (2024)ReAIM: A ReRAM-based Adaptive Ising Machine for Solving Combinatorial Optimization Problems2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00015(58-72)Online publication date: 29-Jun-2024
      • (2023)Special Session - Non-Volatile Memories: Challenges and Opportunities for Embedded System Architectures with Focus on Machine Learning ApplicationsProceedings of the International Conference on Compilers, Architecture, and Synthesis for Embedded Systems10.1145/3607889.3609088(11-20)Online publication date: 17-Sep-2023
      • (2023)HyperMetric: Robust Hyperdimensional Computing on Error-prone Memories using Metric Learning2023 IEEE 41st International Conference on Computer Design (ICCD)10.1109/ICCD58817.2023.00045(243-246)Online publication date: 6-Nov-2023
      • (2023)Unified Agile Accuracy Assessment in Computing-in-Memory Neural Accelerators by Layerwise Dynamical Isometry2023 60th ACM/IEEE Design Automation Conference (DAC)10.1109/DAC56929.2023.10247782(1-6)Online publication date: 9-Jul-2023
      • (2023)Three Challenges in ReRAM-Based Process-In-Memory for Neural Network2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS)10.1109/AICAS57966.2023.10168640(1-5)Online publication date: 11-Jun-2023

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      Full Text

      HTML Format

      View this article in HTML Format.

      HTML Format

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media