Abstract
Real-time data processing using recurrent neural networks (NN) is non-trivial task, due to tight timing constraints requirements. It is proposed hardware implementation of recurrent echo state NN (ESN) on the basis of the Cyclone IV FPGA. Advantages of the hardware implementation are high computational parallelism and low power consumption. To solve the problem of neuron weight storage, it is proposed to reduce the space of their values to a set of integers of low capacity. It was determined that the proposed NN model decreases need in hardware resources for the reservoir implementation in 2–3 orders of magnitude in comparison with conventional NN. Modeling results, implementation and testing of the FPGA project confirmed effectiveness of the proposed integer NN in hardware applications #CSOC1120.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jawandhiya, P.: Hardware design for machine learning. Int. J. Artif. Intell. Appl. 9(1), 63–84 (2018)
Lukosevicius, M., Jaeger, H.: Reservoir computing approaches to RNN training. Comput. Sci. Rev. 3(3), 127–149 (2009)
Maass, W., Natschlager, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002)
Kanerva, P.: Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cogn. Comput. 1(2), 139–159 (2009)
Kleyko, D., Frady, E.P., Osipov, E.: Integer echo state networks: hyperdimensional reservoir computing. arXiv preprint arXiv:1706.00280 (2017)
Kleyko, D., Osipov, E., De Silva, D., Wiklund, U., Alahakoon, D.: Integer self-organizing maps for digital hardware. In: 2019 International Joint Conference on Neural Networks (IJCNN) 2019, pp. 1–8. IEEE, Budapest (2019)
Kleyko, D., Kheffache, M., Frady, E.P., Wiklund, U., Osipov, E.: Density encoding enables resource-efficient randomly connected neural networks. arXiv preprint arXiv:1909.09153 (2019)
Kleyko, D., Osipov, E., De Silva, D., Wiklund, U., Vyatkin, V., Alahakoon, D.: Distributed representation of n-gram statistics for boosting self-organizing maps with hyperdimensional computing. In: Bjørner, N., Virbitskaite, I., Voronkov, A. (eds.) Perspectives of System Informatics (PSI) 2019. Lecture Notes in Computer Science, vol. 11964, pp. 64–79. Springer, Cham (2019)
Gallant, S.I., Culliton, P.: Positional binding with distributed representations. In: International Conference on Image, Vision and Computing (ICIVC), pp. 108–113 (2016)
Widdows, D., Cohen, N.: Reasoning with vectors: a continuous model for fast robust inference. Log. J. IGPL 23(2), 141–173 (2016)
Wang, H., Wu, Y., Zhang, B., Du, K.L.: Recurrent neural networks: associative memory and optimization. Inf. Technol. Softw. Eng. 1(2), 1–15 (2019)
Acknowledgements
Co-funded by the Erasmus + programme of the European Union: Joint project Capacity Building in the field of Higher Education 573545-EPP-1-2016-1-DE-EPPKA2-CBHE-JP “Applied curricula in space exploration and intelligent robotic systems”.
The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Nepomnyashchiy, O., Khantimirov, A., Galayko, D., Sirotinina, N. (2020). Method of Recurrent Neural Network Hardware Implementation. In: Silhavy, R. (eds) Artificial Intelligence and Bioinspired Computational Methods. CSOC 2020. Advances in Intelligent Systems and Computing, vol 1225. Springer, Cham. https://doi.org/10.1007/978-3-030-51971-1_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-51971-1_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-51970-4
Online ISBN: 978-3-030-51971-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)