Abstract
An artificial short term memory, the binary kernel function, is presented to facilitate the learning of complex sequences of integers by Neural Networks, requiring far fewer weights than are usually needed. This is achieved by using only a single weight to encode repeat occurrences of an integer in a sequence. The coding used allows a complex sequence to be learned in only one presentation. The kernel's exponential complexity growth is overcome with hierarchical architectures which chunk the sequences to be learnt. Architectures are introduced for recognition and reproduction of complex sequences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D.Wang and M.A.Arbib, “Timing and chunking in processing temporal order”, IEEE Transactions on Systems, Man, and Cybernetics, Vol. 23, No. 4, pp. 993–1008, 1993.
M.C.Mozer, “Neural net architectures for temporal sequence processing”, in A. S.Weigand and N. A.Gershenfeld (eds) Time Series Prediction: Forecasting the Future and Predicting the Past, SFI Studies in the Sciences of Complexity, Proc. Vol. XV, Addison Wesley: Reading, MA, 1993.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kirke, A.J. A short term memory for Neural Networds which allows recognition and reproduction of complex sequences of integers with the minimum number of weights. Neural Process Lett 3, 49–54 (1996). https://doi.org/10.1007/BF00417789
Issue Date:
DOI: https://doi.org/10.1007/BF00417789