Lei Zhao
ABSTRACT
Deep Neural Network (DNN) accelerators are increasingly developed to pursue high efficiency in DNN computing. However, the IP protection of the DNNs deployed on such accelerators is an important topic that has been less addressed. Although there are previous works that targeted this problem for CMOS-based designs, there is still no solution for ReRAM-based accelerators which pose new security challenges due to their crossbar structure and non-volatility. ReRAM's non-volatility retains data even after the system is powered off, making the stored DNN model vulnerable to attacks by simply reading out the ReRAM content. Because the crossbar structure can only compute on plaintext data, encrypting the ReRAM content is no longer a feasible solution in this scenario.
In this paper, we propose SRA - a secure ReRAM-based DNN accelerator that stores DNN weights on crossbars in an encrypted format while still maintaining ReRAM's in-memory computing capability. The proposed encryption scheme also supports sharing bits among multiple weights, significantly reducing the storage overhead. In addition, SRA uses a novel high-bandwidth SC conversion scheme to protect each layer's intermediate results, which also contain sensitive information of the model. Our experimental results show that SRA can effectively prevent pirating the deployed DNN weights as well as the intermediate results with negligible accuracy loss, and achieves 1.14X performance speedup and 9% energy reduction compared to ISAAC - a non-secure ReRAM-based baseline.
- Y. Chen et al., "Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural network," in ISCA, 2016Google Scholar
Digital Library
- Y. Chen et al., "Dadiannao: A machine-learning supercomputer," in MICRO, 2014Google Scholar
- N. Jouppi et al., "In-datacenter performance analysis of a tensor processing unit," in ISCA, 2017Google Scholar
- H. Wong et al., "Metal-oxide RRAM," in Proceedings of IEEE, 2012Google Scholar
- A. Vincent et al., "Spin-transfer torque magnetic memory as a stochastic memristive synapse," in ISCAS, 2014Google Scholar
- G. Burr et al., "Experimental demonstration and tolerancing of a large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element," in IEEE Transactions on Electron Devices, 2015Google Scholar
- L. Zhao et al., "AEP: An error-bearing neural network accelerator for energy efficiency and model protection," in ICCAD, 2017Google Scholar
- L. Zhao et al., "SCA: a secure CNN accelerator for both training and inference," in DAC, 2020Google Scholar
- W. Li et al., "P3M: a PIM-based neural network model protection scheme for deep learning accelerator," in ASPDAC, 2019Google ScholarDigital Library
- C. Yeh et al., "Compact one-transistor-N-RRAM array architecture for advanced CMOS technology," in IEEE Journal of Solid-State Circuits, 2015Google Scholar
- A. Ren et al., "SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing," in ASPLOS, 2017Google ScholarDigital Library
- S. Li et al., "Scope: A stochastic computing engine for dram-based in-situ accelerator," in MICRO, 2018Google Scholar
- M. Bojnordi et al., "Memristive boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning," in HPCA, 2016Google Scholar
- L. Zhao et al., "Flipping Bits to Share Crossbars in ReRAM-Based DNN Accelerator," in ICCD, 2021Google Scholar
- A. Shafiee et al., "ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars," in ISCA, 2016Google Scholar
- N. Muralimanohar et al., "CACTI 6.0: A tool to model large caches," Technical Report, 2009.Google Scholar
- X. Dong et al., "Nvsim: A circuit-level performance, energy, and area model for emerging nonvolatile memory," TCAD, 2012.Google Scholar
- M. Saberi et al., "Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation ADCs," TCAS1, 2011.Google Scholar
- Y. LeCun et al., "The mnist database of handwritten digits," http://yann.lecun.com/exdb/mnist/, 2011.Google Scholar
- Y. Netzer et al., "Reading digits in natural images with unsupervised feature learning," in NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.Google Scholar
- A. Krizhevsky et al., "Learning multiple layers of features from tiny images," technical report, 2009.Google Scholar
- J. Deng et al., "Imagenet: A large-scale hierarchical image database," in CVPR, 2009.Google Scholar
- Y. LeCun et al., "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, 1998.Google Scholar
- K. He et al., "Deep residual learning for image recognition," in CVPR, 2016.Google Scholar
- S. Xie et al., "Aggregated residual transformations for deep neural networks," in CVPR, 2017.Google Scholar
- C. Szegedy et al., "Going deeper with convolutions," in CVPR, 2015.Google Scholar
- G. Huang et al., "Densely connected convolutional networks," in CVPR, 2017.Google Scholar
- T. Yang et al., "Sparse reram engine: Joint exploration of activation and weight sparsity in compressed neural networks," in ISCA, 2019.Google ScholarDigital Library
Recommendations
Emerging NVM: A Survey on Architectural Integration and Research Challenges
There has been a surge of interest in Non-Volatile Memory (NVM) in recent years. With many advantages, such as density and power consumption, NVM is carving out a place in the memory hierarchy and may eventually change our view of computer architecture. ...
A Novel ReRAM-based Main Memory Structure for Optimizing Access Latency and Reliability
DAC '17: Proceedings of the 54th Annual Design Automation Conference 2017Emerging Resistive Memory (ReRAM) is a promising candidate as the replacement for DRAM because of its low power consumption, high density and high endurance. Due to the unique crossbar structure, ReRAM can be constructed with a very high density. ...
3D-ReG: A 3D ReRAM-based Heterogeneous Architecture for Training Deep Neural Networks
Deep neural network (DNN) models are being expanded to a broader range of applications. The computational capability of traditional hardware platforms cannot accommodate the growth of model complexity. Among recent technologies to accelerate DNN, ...
Comments