Raqibul Hasan
ABSTRACT
Memristor crossbar-based neural networks perform parallel operation in the analog domain. Ex-situ training approach needs to program the predetermined resistance values in the memristor crossbar. Because of the stochasticity of the memristor devices, programming a memristor needs to read the device resistance value iteratively. Reading a single memristor in a crossbar (without isolation transistor) is challenging due to the sneak path current. Programming a memristor in a crossbar to either RON or ROFF state is relatively straightforward. A neural network implemented using higher precision weights provides higher classification accuracy compared to a Ternary Neural Network (TNN). This paper demonstrates the implementation of memristor-based neural networks using only the two resistance values (RON, ROFF). At the same time, it achieves higher weight precision. The experimental result shows that the proposed higher precision synapses are easy to program and provide better classification accuracy compared to a TNN.
- T. M. Taha, R. Hasan, C. Yakopcic, and M. R. McLean, "Exploring the Design Space of Specialized Multicore Neural Processors," IEEE International Joint Conference on Neural Networks (IJCNN), 2013.Google Scholar
- B. Belhadj, A. J. L. Zheng, R. Héliot, and O. Temam. “Continuous real-world inputs can open up alternative accelerator designs,” ISCA 2013.Google Scholar
- L. O. Chua, "Memristor—The Missing Circuit Element," IEEE Transactions on Circuit Theory, vol. 18, no. 5, pp. 507–519 (1971).Google ScholarCross Ref
- D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, “The missing Memristor found,” Nature, vol. 453, pp. 80–83 (2008).Google ScholarCross Ref
- D. Chabi, W. Zhao, D. Querlioz, J.-O. Klein, “Robust Neural Logic Block (NLB) Based on Memristor Crossbar Array” IEEE/ACM International Symposium on Nanoscale Architectures, pp.137-143, 2011.Google ScholarDigital Library
- T. M. Taha, R. Hasan, C. Yakopcic, “Memristor crossbar based multicore neuromorphic processors,” in IEEE International System-on-Chip Conference (SOCC), vol., no., pp.383-389, 2-5 Sept. 2014.Google Scholar
- D. Soudry, D. D. Castro, A. Gal, A. Kolodny, and S. Kvatinsky, "Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training,” IEEE Trans. on Neural Networks and Learning Systems, issue 99, 2015.Google Scholar
- Boquet, G., Macias, E., Morell, A., Serrano, J., Miranda, E., & Vicario, J. L. (2021, January). Offline training for memristor-based neural networks. In 2020 28th European Signal Processing Conference (EUSIPCO) (pp. 1547-1551). IEEE.Google ScholarCross Ref
- S. Yu, Y. Wu, and H.-S. P. Wong, "Investigating the switching dynamics and multilevel capability of bipolar metal oxide resistive switching memory," Applied Physics Letters 98, 103514 (2011).Google ScholarCross Ref
- M. A. Zidan, H. A. H. Fahmy, M. M. Hussain, and K. N. Salama, “Memristor-based memory: The sneak paths problem and solutions,” Microelectronics Journal, vol. 44, no. 2, pp. 176–183, 2012.Google ScholarDigital Library
- Li, Yesheng, and Kah-Wee Ang. "Hardware implementation of neuromorphic computing using large‐scale memristor crossbar arrays." Advanced Intelligent Systems 3, no. 1 (2021): 2000137.Google ScholarCross Ref
- Zhao, Yiyang, Yongjia Wang, Ruibo Wang, Yuan Rong, and Xianyang Jiang. "A highly robust binary neural network inference accelerator based on binary memristors." Electronics 10, no. 21 (2021): 2600.Google Scholar
- Li, Fengfu, Bin Liu, Xiaoxing Wang, Bo Zhang, and Junchi Yan. "Ternary weight networks." arXiv preprint arXiv:1605.04711 (2016).Google Scholar
- X. Dong, C. Xu, S. Member, Y. Xie, and N. P. Jouppi, “NVSim: A Circuit-Level Performance, Energy, and Area Model for Emerging Nonvolatile Memory,” IEEE Trans. on Computer Aided Design of Integrated Circuits and Systems, vol. 31, no. 7, pp. 994-1007, July, 2012.Google ScholarDigital Library
- Alibart, F., Zamanidoost, E., & Strukov, D. B. (2013). Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nature communications, 4(1), 2072.Google Scholar
- C. Yakopcic, T. M. Taha, M. R. McLean, “Method for ex-situ training in a memristor-based neuromorphic circuit using a robust weight programming method,” Electronics Letters, vol. 51, no. 12, pp. 899-900, 2015.Google ScholarCross Ref
- C. Yakopcic, R. Hasan, and T. M. Taha, “Memristor Based Neuromorphic Circuit for Ex-Situ Training of Multi-Layer Neural Network Algorithms,” IEEE IJCNN, 2015.Google ScholarCross Ref
- Hasan, R., Yakopcic, C., & Taha, T. M. (2019). Ex-situ training of large memristor crossbars for neural network applications. Analog Integrated Circuits and Signal Processing, 99, 1-10.Google Scholar
- Russell, S. & Norvig, P. (2002). Artificial Intelligence: A Modern Approach (2nd Edition). Prentice Hall, ISBN-13: 978-01379039555.Google Scholar
- Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., & Bengio, Y. (2017). Quantized neural networks: Training neural networks with low precision weights and activations. The Journal of Machine Learning Research, 18(1), 6869-6898.Google Scholar
- Courbariaux, M., Bengio, Y., & David, J. P. (2015). Binaryconnect: Training deep neural networks with binary weights during propagations. Advances in neural information processing systems, 28.Google Scholar
- http://yann.lecun.com/exdb/mnist/Google Scholar
Index Terms
- Robust Ex-situ Training of Memristor Crossbar-based Neural Network with Limited Precision Weights
Recommendations
Ex-situ training of large memristor crossbars for neural network applications
Memristor crossbar arrays carry out multiply---add operations in parallel in the analog domain, and can enable neuromorphic systems with high throughput at low energy and area consumption. Neural networks need to be trained prior to use. This work ...
A fast training method for memristor crossbar based multi-layer neural networks
Memristor crossbar arrays carry out multiply---add operations in parallel in the analog domain which is the dominant operation in a neural network application. On-chip training of memristor neural network systems have the significant advantage of being ...
On-chip training of memristor crossbar based multi-layer neural networks
Memristor crossbar arrays carry out multiply-add operations in parallel in the analog domain, and so can enable neuromorphic systems with high throughput at low energy and area consumption. On-chip training of these systems have the significant ...
Comments