Abstract
In this paper we propose three modifications of Differential Neural Computer aiming at the improvement of convergence and speed of the network training. The first one relies on simplifying the mechanism of erasing old information; the second increases the influence of the link between data, and the last one increases the utility of the memory module. We evaluate the proposed modifications using five bAbI tasks. The results analysis gave some insights into modification effects, and the most promising results achieved the DNC modification without erasing vector.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Csordás, R., Schmidhuber, J.: Improving differentiable neural computers through memory masking, de-allocation, and link distribution sharpness control. CoRR abs/1904.10278 (2019). http://arxiv.org/abs/1904.10278
Franke, J., Niehues, J., Waibel, A.: Robust and scalable differentiable neural computer for question answering. CoRR abs/1807.02658 (2018). http://arxiv.org/abs/1807.02658
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT (2016). http://www.deeplearningbook.org
Graves, A., Wayne, G., Danihelka, I.: Neural turing machines. CoRR abs/1410.5401 (2014). http://arxiv.org/abs/1410.5401
Graves, A., et al.: Hybrid computing using a neural network with dynamic external memory. Nature, 471–476 (2016). https://doi.org/10.1038/nature20101
Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. CoRR abs/1503.04069 (2015). http://arxiv.org/abs/1503.04069
Jozefowicz, R., Zaremba, W., Sutskever, I.: An empirical exploration of recurrent network architectures. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning ICML2015, vol. 37, pp. 2342–2350. JMLR.org (2015). http://dl.acm.org/citation.cfm?id=3045118.3045367
Weston, J., Chopra, S., Bordes, A.: Memory networks (2014). https://arxiv.org/abs/1410.3916
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Markowska-Kaczmar, U., Kułakowski, G. (2020). Inference Ability Assessment of Modified Differential Neural Computer. In: Nguyen, N., Jearanaitanakij, K., Selamat, A., Trawiński, B., Chittayasothorn, S. (eds) Intelligent Information and Database Systems. ACIIDS 2020. Lecture Notes in Computer Science(), vol 12033. Springer, Cham. https://doi.org/10.1007/978-3-030-41964-6_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-41964-6_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41963-9
Online ISBN: 978-3-030-41964-6
eBook Packages: Computer ScienceComputer Science (R0)