Abstract
Process-in-memory (PIM) engines based on Resistive random-access memory (RRAM) are used to accelerate the convolutional neural network (CNN). RRAM performs computation by mapping weights on its crossbars and applying a high voltage to get results. The computing process degrades RRAM from the fresh status where RRAM can support high data precision to the aged status where RRAM only can support low precision, potentially leading to a significant CNN training accuracy degradation. Fortunately, many previous studies show that the impact of loss caused by the RRAM precision limitation across various weights is different for CNN training accuracy, which motivates us to consider mapping different weights on RRAM with different statuses to keep high CNN training accuracy and extending the high CNN training accuracy iterations of PIM engines based on RRAM, which is regarded as the lifetime of the RRAM on CNN training. In this paper, we propose a method to evaluate the performance of the weights mapping on extending the lifetime of the RRAM and present a weights mapping framework specifically designed for the hybrid of aged and fresh RRAM to extend the lifetime of the RRAM engines on CNN training. Experimental results demonstrate that our weights mapping framework brings up to 6.3\(\times \) on average lifetime enhancement compared to the random weights mapping.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cokun, M., et al.: Face recognition based on convolutional neural network. In: International Conference on Modern Electrical and Energy Systems (2017)
Li, Q., et al.: Medical image classification with convolutional neural network. In: 13th International Conference on Control Automation Robotics & Vision (2014)
Chen, L., et al.: Beyond human recognition: a CNN-based framework for handwritten character recognition. In: 3rd IAPR Asian Conference on Pattern Recognition (2015)
Tang, T., et al.: Binary convolutional neural network on RRAM. In: 22nd Asia and South Pacific Design Automation Conference (2017)
Shafiee, A., et al.: ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In: ACM SIGARCH Computer Architecture News, vol. 44, no. 3, pp. 14–26 (2016)
Chi, P., et al.: Prime: a novel processing-in-memory architecture for neural network computation in reram-based main memory. In: ACM SIGARCH Computer Architecture News, vol. 44, no. 3, pp. 27–39 (2016)
Li, W., et al.: Timely: pushing data movements and interfaces in pim accelerators towards local and in time domain. In: ACM/IEEE 47th Annual International Symposium on Computer Architecture (2020)
Chen, P.Y., et al.: WRAP: weight RemApping and processing in RRAM-based neural network accelerators considering thermal effect. In: Design, Automation & Test in Europe Conference & Exhibition (2022)
Zhang, G.L., et al.: Reliable and robust RRAM-based neuromorphic computing. In Proceedings of the 2020 on Great Lakes Symposium on VLSI (2020)
Zhang, S., et al.: Lifetime enhancement for rram-based computing-in-memory engine considering aging and thermal effects. In: 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (2020)
Zhu, Z., et al.: Mixed size crossbar based RRAM CNN accelerator with overlapped mapping method. In: International Conference on Computer-Aided Design (2018)
Liang, T., et al.: Pruning and quantization for deep neural network acceleration: a survey. Neurocomputing 461, 370–403 (2021)
Chen, B., et al.: Physical mechanisms of endurance degradation in TMO-RRAM. In: International Electron Devices (2011)
Le, B.Q., et al.: Resistive RAM with multiple bits per cell: array-level demonstration of 3 bits per cell. IEEE Trans. Electron Dev. 66(1), 641–646 (2018)
Yong, H., Huang, J., Hua, X., Zhang, L.: Gradient centralization: a new optimization technique for deep neural networks. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 635–652. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_37
Huang, Z., Shao, W., Wang, X., Lin, L., Luo, P.: Rethinking the pruning criteria for convolutional neural network. Adv. Neural Inf. Process. Syst. 34, 16305–16318 (2021)
Molchanov, P., et al.: Pruning convolutional neural networks for resource efficient inference. In: International Conference on Learning Representations (2017)
LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. In: Conference on Neural Information Processing Systems (1989)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv Machine Learning (2017)
Walt, S., Colbert, S.C., Varoquaux, G.: The NumPy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)
Zhang, S., et al.: Lifetime enhancement for rram-based computing-in-memory engine considering aging and thermal effects. In: IEEE International Conference on Artificial Intelligence Circuits and Systems (2020)
Tanaka, H., et al.: Pruning neural networks without any data by iteratively conserving synaptic flow. Adv. Neural Inf. Process. Syst. 33, 6377–6389 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yang, F., Li, Y., Niu, Z., Wang, G., Liu, X. (2024). ExtendLife: Weights Mapping Framework to Improve RRAM Lifetime for Accelerating CNN. In: Li, C., Li, Z., Shen, L., Wu, F., Gong, X. (eds) Advanced Parallel Processing Technologies. APPT 2023. Lecture Notes in Computer Science, vol 14103. Springer, Singapore. https://doi.org/10.1007/978-981-99-7872-4_3
Download citation
DOI: https://doi.org/10.1007/978-981-99-7872-4_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-7871-7
Online ISBN: 978-981-99-7872-4
eBook Packages: Computer ScienceComputer Science (R0)
