Skip to main content
SpringerLink
Log in

STORE working memory networks for storage and recall of arbitrary temporal sequences

  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

Neural network models of working memory, called “sustained temporal order recurrent” (STORE) models, are described. They encode the invariant temporal order of sequential events in short-term memory (STM) in a way that mimics cognitive data about working memory, including primacy, recency, and bowed order and error gradients. As new items are presented, the pattern of previously stored items remains invariant in the sense that relative activations remain constant through time. This invariant temporal order code enables all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system. Such competence is needed to design self-organizing temporal recognition and planning systems in which any subsequence of events may need to be categorized in order to control and predict future behavior or external events. STORE models show how arbitrary event sequences may be invariantly stored, including repeated events. A preprocessor interacts with the working memory to represent event repeats in spatially separate locations. It is shown why at least two processing levels are needed to invariantly store events presented with variable durations and interstimulus intervals. It is also shown how network parameters control the type and shape of primacy, recency, or bowed temporal order gradients that will be stored.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Atkinson RC, Shiffrin RM (1971) The control of short term memory. Sci Am, August:82–90

  • Baddeley AD (1986) Working memory. Clarendon Press, Oxford

    Google Scholar 

  • Boardman I, Bullock D (1991) A neural network model of serial order recall from short-term memory. In: Proceedings of the international joint conference on neural networks, Seattle, II:879–884 IEEE Service Center, Piscataway, NJ

    Google Scholar 

  • Boardman I, Cohen MA, Grossberg S (1993) Variable rate working memories for phonetic categorization and invariant speech perception. In: Proceedings of the international joint conference on neural networks, Portland, III:2–5. IEEE Service Center, Piscataway, NJ

    Google Scholar 

  • Bradski G, Carpenter GA, Grossberg S (1991) Working memory networks for learning multiple groupings of temporally ordered events: applications to 3-D visual object recognition. In: Proceedings of the international joint conference on neural networks, Seattle, 1: 723–728. IEEE Service Center, Piscataway, NJ

  • Bradski G, Carpenter, GA, Grossberg, S (1992) Working memory networks for learning temporal order with application to 3-D visual object recognition. Neural Comp 4:270–286

    Google Scholar 

  • Carpenter GA, Grossberg S (eds) (1991) Pattern recognition by self-organizing neural networks. MIT Press, Cambridge, MA

    Google Scholar 

  • Carpenter GA, Grossberg S (1992) A self-organizing neural network for supervised learning, recognition, and prediction. IEEE Commun 30:38–49

    Article  Google Scholar 

  • Carpenter GA, Grossberg S, Reynolds JH (1991) ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks 4:565–588

    Article  Google Scholar 

  • Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Networks 3:698–713

    Article  Google Scholar 

  • Cohen MA, Grossberg S (1986) Neural dynamics of speech and language coding: developmental programs, perceptual grouping, and competition for short term memory. Hum Neurobiol 5:1–22

    PubMed  Google Scholar 

  • Elman JL (1990) Finding structure in time. Cogn 14:179–211

    Article  Google Scholar 

  • Gaudiano P, Grossberg S (1991) Vector associative maps: unsupervised real-time error-based learning and control of movement trajectories. Neural Networks 4:147–183

    Article  Google Scholar 

  • Grossberg S (1970) Some networks that can learn, remember, and reproduce any number of complicated space-time patterns. II. Stud Appl Math 49:135–166

    Google Scholar 

  • Grossberg S (1973) Contour enhancement, short term memory, and constancies in reverberating neural networks. Stud Appl Math 52:217–257

    Google Scholar 

  • Grossberg S (1976) Adaptive pattern classification and universal recoding. I. Parallel development and coding of neural feature detectors. Biol Cybern 23:121–134

    Article  PubMed  Google Scholar 

  • Grossberg S (1978a) A theory of human memory: self-organization and performance of sensory-motor codes, maps, and plans. In: Rosen R, Snell F (Eds) Progress in theoretical biology, Vol 5. Academic Press, New York, pp 233–374

    Google Scholar 

  • Grossberg S (1987b) Behavioral contrast in short term memory: serial binary memory models or parallel continuous memory models? J Math Psychol 17:199–219

    Article  Google Scholar 

  • Grossberg S (1982) Studies of mind and brain: neural principles of learning, perception, development, cognition, and motor control. Reidel Press, Boston

    Google Scholar 

  • Grossberg S (1986) The adaptive self-organization of serial order in behavior: speech, language, and motor control. Schwab EC, Nusbaum, HC (Eds) Pattern recognition by humans and machines, Vol 1. Speech perception. Academic Press, New York, pp 187–294. Reprinted in: Grossberg S (eds) The adaptive brain. II. Vision, Speech, language, and motor control. Elsevier/North-Holland, Amsterdam, pp 311–400

    Google Scholar 

  • Grossberg S, Kuperstein M (1989) Neural dynamics of adaptive sensory-motor control: expanded edition. Pergamon Press, Elmsford, NY

    Google Scholar 

  • Grossberg S, Pepe J (1971) Spiking threshold and overarousal effects in serial learning. J Stat Phys 3:95–125

    Article  Google Scholar 

  • Grossberg S, Stone GO (1986a) Neural dynamics of attention switching and temporal order information in short term memory. Memory Cogn 14:451–468

    Google Scholar 

  • Grossberg S, Stone GO (1986b) Neural dynamics of word recognition and recall: attentional priming, learning, and resonance. Psychol Rev 93:46–74

    Article  PubMed  Google Scholar 

  • Gutfreund H, Mezard M (1988) Processing of temporal sequences in neural networks. Physiol Rev Lett 61:235–238

    Article  Google Scholar 

  • Guyon I, Personnaz L, Nadal JP, Dreyfus G (1988) Storage retrieval of complex sequences in neural networks. Physiol Rev A 38:6365–6372

    Article  Google Scholar 

  • Healy AF (1975) Separating item from order information in short-term memory. J Verbal Learn Verbal Behav 13:644–655

    Article  Google Scholar 

  • Jordan MI (1986) Serial order: a parallel distributed processing approach. (Institute for Cognitive Science, Report 8604) University of California, San Diego

    Google Scholar 

  • Lee C, Estes WK (1981) Item and order information in short-term memory: evidence for multilevel perturbation process J Exp Psychol Hum Learn Memory 1:149–169

    Google Scholar 

  • Miller JL (1981) Effects of speaking rate on segmental distinctions In: Eimas PD, Miller JL (Eds) Perspectives on the study of speech. Lawrence Erlbaum, Hillsdale, NJ

    Google Scholar 

  • Miller JL, Liberman AM (1979) Some effects of later-occurring information on the perception of stop consonant and semivowel. Percept Psychophys 25:457–465

    PubMed  Google Scholar 

  • Ratcliff, R (1978) A theory of memory retrieval. Psychol Rev 85:59–108

    Article  Google Scholar 

  • Reeves A, Sperling G (1986) Attention gating in short-term visual memory. Psychol Rev 93:180–206

    Article  PubMed  Google Scholar 

  • Repp BH (1980) A range-frequency effect on perception of silence in speech. Haskins Laboratories Status Report on Speech Research, SR-61:151–165

    Google Scholar 

  • Repp BH (1983) Bi directional contrast effects in the perception of VC-CV sequences. Percept Psychophys 33:147–155

    PubMed  Google Scholar 

  • Repp B, Liberman A, Eccardt T, Pestsky D (1978) Perceptual integration of acoustic cues for stop fricative and affiricate manner. J Exp Psychol Hum Percept Perform 4:621–637

    Article  PubMed  Google Scholar 

  • Schreter Z, Pfeifer R (1989) Short-term memory/long-term memory interactions in connectionist simulations of psychological experiments on list learning. In: Personnaz L, Dreyfus G (Eds) Neural networks from models to applications. IDSET, Paris

    Google Scholar 

  • Seibert MC (1991) Neural networks for machine vision: learning three-dimensional object recognition. PhD Dissertation, Boston University

  • Seibert MC, Waxman AM (1990a) Learning aspect graph representations from view sequences. In: Touretzky DS (Eds) Advances in neural information processing systems 2. Morgan Kaufmann, San Mateo, CA, pp 258–265

    Google Scholar 

  • Seibert MC, Waxman AM (1990b) Learning aspect graph representations of 3D objects in a neural network. In: Caudill M (eds) Proceedings of the international joint conference on neural networks, Washington, DC, 2. Lawrence Erlbaum, Hillsdale, NJ, 233–236

    Google Scholar 

  • Tartter VC, Kat D, Samuel AG, Repp BH (1983) Perception of intervocalic stop consonants: the contributions of closure duration and formant transitions. J Acoust Soc Am 73:715–725

    Article  Google Scholar 

  • Wang D, Arbib MA (1990) Complex temporal sequence learning based on short-term memory. Proc IEE 78:1536–1543

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Adaptive Systems and Department of Cognitive and Neural Systems, Boston University, 111 Cummington Street, 02215, Boston, MA, USA

    Gary Bradski, Gail A. Carpenter & Stephen Grossberg

Authors
  1. Gary Bradski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gail A. Carpenter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephen Grossberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Supported in part by the Air Force Office of Scientific Research (AFOSR 90-0128) and the Office of Naval Research (ONR N00014-91-J-4100 and ONR N00014-92-J-1309).

Supported in part by ARPA (AFOSR 90-0083 and ONR N00014-92-J-4015), the National Science Foundation (NSF IRI-94-01659), and the Office of Naval Research (ONR N00014-91-J-4100).

Supported in part by the Air Force Office of Scientific Research (AFOSR F49620-92-J0225), ARPA (AFOSR 90-0083 and ONR N00014-92-J4015), and the Office of Naval Research (ONR N00014-91-J-4100 and ONR N00014-92-J-1309).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bradski, G., Carpenter, G.A. & Grossberg, S. STORE working memory networks for storage and recall of arbitrary temporal sequences. Biol. Cybern. 71, 469–480 (1994). https://doi.org/10.1007/BF00198465

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00198465

Keywords

Search

Navigation