Suyong Lee
ABSTRACT
Deep Neural Networks (DNNs) surpass the human-level performance on specific tasks. The outperforming capability accelerate an adoption of DNNs to safety-critical applications such as autonomous vehicles and medical diagnosis. Millions of parameters in DNN requires a high memory capacity. A process technology scaling allows increasing memory density, however, the memory reliability confronts significant reliability issues causing errors in the memory. This can make stored weights in memory erroneous. Studies show that the erroneous weights can cause a significant accuracy loss. This motivates research on fault-tolerant DNN architectures. Despite of these efforts, DNNs are still vulnerable to errors, especially error in DNN classifier. In the worst case, because a classifier in convolutional neural network (CNN) is the last stage determining an input class, a single error in the classifier can cause a significant accuracy drop. To enhance the fault tolerance in CNN, this paper proposes a novel bipolar vector classifier which can be easily integrated with any CNN structures and can be incorporated with other fault tolerance approaches. Experimental results show that the proposed method stably maintains an accuracy with a high bit error rate up to 10−3 in the classifier.
- Sanguhn Cha et al. 2017. Defect analysis and cost-effective resilience architecture for future DRAM devices. In 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 61--72.Google Scholar
- Kaiming He et al. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.Google Scholar
- Le-Ha Hoang et al. 2020. FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving Their Fault Tolerance Using Clipped Activation. In Proceedings of the 23rd Conference on Design, Automation and Test in Europe (Grenoble, France) (DATE '20). EDA Consortium, San Jose, CA, USA, 1241--1246.Google Scholar
- Gao Huang et al. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700--4708.Google Scholar
- Muhammad Imran et al. 2020. Effective Write Disturbance Mitigation Encoding Scheme for High-density PCM. In 2020 Design, Automation Test in Europe Conference Exhibition (DATE). 1490--1495.Google Scholar
- Sangmok Jeong et al. 2020. PAIR: Pin-aligned In-DRAM ECC architecture using expandability of Reed-Solomon code. In 2020 57th ACM/IEEE Design Automation Conference (DAC). 1--6.Google Scholar
- Jae-San Kim et al. 2019. DRIS-3: Deep Neural Network Reliability Improvement Scheme in 3D Die-Stacked Memory based on Fault Analysis. In 2019 56th ACM/IEEE Design Automation Conference (DAC). 1--6.Google Scholar
- Alex Krizhevsky et al. 2009. Learning multiple layers of features from tiny images. (2009).Google Scholar
- Taehyun Kwon et al. 2020. Cost-Effective Reliable MLC PCM Architecture Using Virtual Data Based Error Correction. IEEE Access 8 (2020), 44006--44018.Google ScholarCross Ref
- Taehyun Kwon et al. 2021. Reliability Enhanced Heterogeneous Phase Change Memory Architecture for Performance and Energy Efficiency. IEEE Trans. Comput. 70, 9 (2021), 1388--1400.Google ScholarDigital Library
- Elbruz Ozen et al. 2020. Boosting Bit-Error Resilience of DNN Accelerators Through Median Feature Selection. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 11 (2020), 3250--3262.Google ScholarCross Ref
- Elbruz Ozen et al. 2020. Just Say Zero: Containing Critical Bit-Error Propagation in Deep Neural Networks with Anomalous Feature Suppression. In Proceedings of the 39th International Conference on Computer-Aided Design (Virtual Event, USA) (ICCAD '20). Association for Computing Machinery, New York, NY, USA, Article 75, 9 pages.Google Scholar
- Adam Paszke et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019), 8026--8037.Google Scholar
- Brandon Reagen et al. 2018. Ares: A framework for quantifying the resilience of deep neural networks. In 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC). 1--6.Google Scholar
- Mark Sandler et al. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4510--4520.Google Scholar
- Christian Szegedy et al. 2016. Rethinking the Inception Architecture for Computer Vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Inayat Ullah et al. 2020. Factored Radix-8 Systolic Array for Tensor Processing. In 2020 57th ACM/IEEE Design Automation Conference (DAC). 1--6.Google Scholar
- Jung Min You et al. 2019. MRLoc: Mitigating Row-Hammering Based on Memory Locality. In Proceedings of the 56th Annual Design Automation Conference 2019 (Las Vegas, NV, USA) (DAC '19). Association for Computing Machinery, New York, NY, USA, Article 19, 6 pages.Google Scholar
Index Terms
- Bipolar vector classifier for fault-tolerant deep neural networks
Recommendations
Symmetric Power Activation Functions for Deep Neural Networks
LOPAL '18: Proceedings of the International Conference on Learning and Optimization Algorithms: Theory and ApplicationsCommon nonlinear activation functions with large saturation regions, like Sigmoid and Tanh, used for Deep Neural Networks (DNNs) can not guarantee useful and efficient training since they suffer from vanishing gradients problem. Rectified Linear Units ...
Deep Elman recurrent neural networks for statistical parametric speech synthesis
Owing to the success of deep learning techniques in automatic speech recognition, deep neural networks (DNNs) have been used as acoustic models for statistical parametric speech synthesis (SPSS). DNNs do not inherently model the temporal structure in ...
Detect and Remove Watermark in Deep Neural Networks via Generative Adversarial Networks
Information SecurityAbstractDeep neural networks (DNN) have achieved remarkable performance in various fields. However, training a DNN model from scratch requires expensive computing resources and a lot of training data, which are difficult to obtain for most individual ...
Comments