Baogang Zhang
ABSTRACT
The dominating computational workload in the inference phase of deep neural networks (DNNs) is matrix-vector multiplication. An arising solution to accelerate the inference phase is to perform analog matrix-vector multiplication using memristor crossbar arrays (MCAs). A key challenge is that stuck-at-fault defects may degrade the classification accuracy of the memristor-based DNNs. A common technique to reduce the negative impact of stuck-at-faults is to utilize redundant synapses, i.e, each row in a weight matrix is realized using two (or r) parallel rows in an MCA. In this paper, we propose to handle stuck-at-faults by inserting redundant neurons and by sharing redundant synapses. The first technique is based on inserting redundant neurons to surgically repair neurons connected to rows and columns in the MCAs with many stuck-at-faults. The second technique is focused on sharing redundant synapses between different neurons to reduce the hardware overhead, which generalizes (1:r) synapse redundancy in previous studies to (q:r) synapse redundancy. The experimental results demonstrate new trade-offs between robustness and hardware overhead without requiring the neural networks to be retrained. Compared with state-of-the-art, the power and area overhead for a neural network can be reduced with up to 16% and 25%, respectively.
Supplemental Material
- C. Y. Chen et al. RRAM defect modeling and failure analysis based on march test and a novel squeeze-search scheme. Tran. on Computers, 64(1):180--190, 2015.Google Scholar
Cross Ref
- L. Chen et al. Accelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar. DATE'17, pages 19--24, 2017.Google ScholarDigital Library
- P. Chi et al. PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM -based main memory. ISCA'16, pages 27--39, 2016.Google ScholarDigital Library
- T. H. Cormen et al. Introduction to Algorithms. McGraw-Hill Higher Education, 2001.Google ScholarDigital Library
- Z. He et al. Noise injection adaption: End-to-end reram crossbar non-ideal effect adaption for neural network mapping. DAC'19, pages 57:1--57:6, 2019.Google ScholarDigital Library
- M. Hu et al. Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication. DAC'16, pages 1--6, 2016.Google ScholarDigital Library
- S. Kannan et al. Modeling, detection, and diagnosis of faults in multilevel memristor memories. IEEE TCAD, 34(5):822--834, 2015.Google Scholar
- Y. LeCun et al. Deep learning. Nature, 521(7553):436, 2015.Google ScholarCross Ref
- B. Liu et al. Vortex: Variation-aware training for memristor X-bar. DAC'15, pages 1--6, 2015.Google Scholar
- X. Liu et al. Harmonica: A framework of heterogeneous computing systems with memristor-based neuromorphic computing accelerators. IEEE TCAS, 63(5):617--628, 2016.Google Scholar
- H. Miao et al. Memristor-based analog computation and neural network classification with a dot product engine. Advanced Materials, 30(9):1705914.Google Scholar
- A. Shafiee et al. ISAAC: A convolutional neural network accelerator with In-Situ analog arithmetic in crossbars. ISCA'16, pages 14--26, 2016.Google ScholarDigital Library
- P. Simon. Too Big to Ignore: The Business Case for Big Data. Wiley Publishing, 1st edition, 2013.Google Scholar
- L. Song et al. Pipelayer: A pipelined ReRAM-based accelerator for deep learning. HPCA'17, pages 541--552, 2017.Google ScholarCross Ref
- A. J. Van de Goor and Y. Zorian. Effective march algorithms for testing single-order addressed memories. Journal of Electronic Testing, 5(4):337--345, 1994.Google ScholarDigital Library
- W. A. Wulf and S. A. McKee. Hitting the memory wall: Implications of the obvious. SIGARCH Computing Architecture News, 23(1):20--24, 1995.Google ScholarDigital Library
- L. Xia et al. Fault-tolerant training with on-line fault detection for rram-based neural computing systems. DAC'17, pages 1--6, 2017.Google ScholarDigital Library
- L. Xia et al. Stuck-at fault tolerance in rram computing systems. IEEE JETCAS, 8(1):102--115, 2018.Google Scholar
- B. Yan et al. A closed-loop design to enhance weight stability of memristor based neural network chips. ICCAD'17, pages 541--548, 2017.Google ScholarDigital Library
- B. Zhang et al. Handling stuck-at-faults in memristor crossbar arrays using matrix transformations. ASP-DAC'19, pages 438--443, 2019.Google ScholarDigital Library
Index Terms
- Redundant Neurons and Shared Redundant Synapses for Robust Memristor-based DNNs with Reduced Overhead
Recommendations
Associative Learning of Integrate-and-Fire Neurons with Memristor-Based Synapses
A memrsitor is a two-terminal electronic device whose conductance can be precisely modulated by charge or flux through it. In this paper, we present a class of memristor-based neural circuits comprising leaky integrate-and-fire (I & F) neurons and ...
Self-organization of Hebbian synapses in Hippocampal neurons
NIPS'90: Proceedings of the 3rd International Conference on Neural Information Processing SystemsWe are exploring the significance of biological complexity for neuronal computation. Here we demonstrate that Hebbian synapses in realistically-modeled hippocampal pyramidal cells may give rise to two novel forms of self-organization in response to ...
Modelling reduced excitability in aged CA1 neurons as a calcium-dependent process
We use a multi-compartmental model of a CA1 pyramidal cell to study changes in hippocampal excitability that result from aging-induced alterations in calcium-dependent membrane mechanisms. The model incorporates N- and L-type calcium channels which are ...
Comments