Abstract
The conventional computing system with the architecture of von Neumann has greatly benefited our humans for past decades, while it is also suffered from low efficiency due to the separation between a memory block and a processing unit. Memristor, which is an emerging electron device with the capability of data storage and processing information simultaneously, can be employed to construct a bioinspired neuromorphic computing system. Simulation as one of the most powerful methods to obtain the optimizing result for the memristor-based neuromorphic network has been extensively focused to realize the high precision calculation. It becomes very difficult because the pulse-to-pulse (P2P) model is limited by the updating process. The memristor-based multi-layer Perceptron (MLP) network online training generally presents a low accuracy. Therefore, an efficiency training schedual is urgently desired to improve the accuracy. Based on the resistive switching behavior observed in the Ag/TiOx/F-doped SnO2 memristor, the weight update by the P2P model enables the MLP network online training in the low accuracy memristor with high performance. The low bits MLP optimized by a novel weight update schedual can realize high precision identification and classification. By that, the time and power consumption of memristor can be largely reduced. The experiment result illustrates that the high accuracy of 90.82% and 95.44% can be obtained at the first and final epoch of the MNIST handwritten digital datasets, respectively. Importantly, the number of the weight update, and the online training time and power consumption can be reduced by 81% and 93.7%, respectively. The scheme provides high precision, low power consumption, and fast convergence solution for the in-situ training of the imprecise memristor-based neuromorphic network.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agarwal, S. et al. (2017) In: 2017 Symposium on VLSI technology. T174-T175 (IEEE)
Ambrogio S et al (2018) Equivalent-accuracy accelerated neural-network training using analogue memory. Nature 558:60–67
Borghetti J et al (2010) ‘Memristive’switches enable ‘stateful’logic operations via material implication. Nature 464:873–876
Chen P.-Y, Peng X, Yu S (2017) In: 2017 IEEE International electron devices meeting (IEDM). 6.1. 1–6.1. 4 (IEEE)
Chua L (1971) Memristor-the missing circuit element. IEEE Trans Circuits Syst 18:507–519
Devlin J, Chang M.-W, Lee K, Toutanova K. (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
Gao L, Chen P-Y, Yu S (2015) Programming protocol optimization for analog weight tuning in resistive memories. IEEE Electr Device Lett 36:1157–1159
He K, Zhang X, Ren S, Sun J (2016) In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778
Howard AG et al. (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861
Hu H, Wang L, Peng L, Zeng Y-R (2020) Effective energy consumption forecasting using enhanced bagged echo state network. Energy 193:116778
Kautz WH (1969) Cellular logic-in-memory arrays. IEEE Trans Comput 100:719–727
Khan A, Sohail A, Zahoora U, Qureshi AS (2020) A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev 53:5455–5516
Krestinskaya O, Choubey B, James AP (2020) Memristive GAN in analog. Sci Rep. https://doi.org/10.1038/s41598-020-62676-7
LeCun, Y. (1998) The MNIST database of handwritten digits. https://yann.lecun.com/exdb/mnist/.
Li C et al (2020) A review for cervical histopathology image analysis using machine vision approaches. Artif Intell Rev 53:1–42
Liu ZW et al (2020) Neural signal analysis with memristor arrays towards high-efficiency brain-machine interfaces. Nat Commun. https://doi.org/10.1038/s41467-020-18105-4
Nair V, Hinton GE (2010) In: ICML
Nogueira MA et al (2017) Image descriptors in radiology images: a systematic review. Artif Intell Rev 47:531–559
Peng L, Zhu Q, Lv S-X, Wang L (2020) Effective long short-term memory with fruit fly optimization algorithm for time series forecasting. Soft Comput 24:1–21
Prezioso M et al (2015) Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521:61–64
Redmon J, Divvala S, Girshick R, Farhadi A (2016) In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 779–788
Sheridan PM et al (2017) Sparse coding with memristor networks. Nat Nanotechnol 12:784. https://doi.org/10.1038/Nnano.2017.83
Silver D et al (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–489
Strukov DB, Snider GS, Stewart DR, Williams RS (2008) The missing memristor found. Nature 453:80–83. https://doi.org/10.1038/nature06932
Sun B et al (2019) A unified capacitive-coupled memristive model for the nonpinched current–voltage hysteresis loop. Nano Lett 19:6461–6465
Waldrop MM (2016) The chips are down for Moore’s law. Nat News 530:144
Wang L, Lv S-X, Zeng Y-R (2018) Effective sparse adaboost method with ESN and FOA for industrial electricity consumption forecasting in China. Energy 155:1013–1031
Wang ZR et al (2019) Reinforcement learning with analogue memristor arrays. Nat Electron 2:115–124. https://doi.org/10.1038/s41928-019-0221-6
Wang Z, Wang E, Zhu Y (2020) Image segmentation evaluation: a survey of methods. Artif Intell Rev 53:1–38
Wang YY, Wu S, Tian L, Shi LP (2020b) SSM: a high-performance scheme for in situ training of imprecise memristor neural networks. Neurocomputing 407:270–280. https://doi.org/10.1016/j.neucom.2020.04.130
Yao P et al (2017) Face classification using electronic synapses. Nat Commun 8:1–8
Yao P et al (2020) Fully hardware-implemented memristor convolutional neural network. Nature 577:641. https://doi.org/10.1038/s41586-020-1942-4
Zhang Q et al (2018) Sign backpropagation: an on-chip learning algorithm for analog RRAM neuromorphic computing systems. Neural Netw 108:217–223
Zhang W et al (2019) In: 2019 56th ACM/IEEE Design automation conference (DAC). pp. 1–6 (IEEE)
Zhou G et al (2018) Coexistence of negative differential resistance and resistive switching memory at room temperature in TiOx modulated by moisture. Adv Electron Mater 4:1700567
Zhou G et al (2019) Resistive switching memory integrated with amorphous carbon-based nanogenerators for self-powered device. Nano Energy 63:103793
Zhou G et al (2019) Artificial and wearable albumen protein memristor arrays with integrated memory logic gate functionality. Mater Horiz 6:1877–1882
Zhou G et al (2020) Capacitive effect: an original of the resistive switching memory. Nano Energy 68:104386
Zhou G et al (2021) Negative photoconductance effect: an extension function of the TiOx-based memristor. Adv Sci 8:2003765
Zhu F et al (2020) In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 1969–1979
Acknowledgements
This work was supported by the National Key R&D Program of China (Grant No. 2018YFB1306600), the National Natural Science Foundation of China (Grant Nos.62076207, 62076208), the Fundamental Science and Advanced Technology Research Foundation of Chongqing, China (Grant Nos. cstc2017jcyjBX0050).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, J., Zhou, G., Li, Y. et al. Reduction 93.7% time and power consumption using a memristor-based imprecise gradient update algorithm. Artif Intell Rev 55, 657–677 (2022). https://doi.org/10.1007/s10462-021-10060-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-021-10060-w