Skip to main content
SpringerLink
Log in

Can’t Get You Out of My Head: A Connectionist Model of Cyclic Rehearsal

  • Conference paper
Modeling Communication with Robots and Virtual Humans

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4930))

Abstract

Humans are able to perform a large variety of periodic activities in different modes, for instance cyclic rehearsal of phone numbers, humming a melody sniplet over and over again. These performances are, to a certain degree, robust against perturbations, and it often suffices to present a new pattern a few times only until it can be “picked up”. From an abstract mathematical perspective, this implies that the brain, as a dynamical system, (1) hosts a very large number of cyclic attractors, such that (2) if the system is driven by external input with a cyclic motif, it can entrain to a closely corresponding attractor in a very short time. This chapter proposes a simple recurrent neural network architecture which displays these dynamical phenomena. The model builds on echo state networks (ESNs), which have recently become popular in machine learning and computational neuroscience.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baddeley, A.: Working memory: looking back and looking forward. Nature Reviews: Neuroscience 4(10), 829–839 (2003)

    Article  Google Scholar 

  2. Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 5(2), 157–166 (1994)

    Article  Google Scholar 

  3. Buonomano, D.V.: A learning rule for the emergence of stable dynamics and timing in recurrent networks. Journal of Neurophysiology 94, 2275–2283 (2005), http://www.neurobio.ucla.edu/~dbuono/BuonoJNphy05.pdf

    Article  Google Scholar 

  4. Cariani, P.: Temporal codes, timing nets, and music perception. Journal of New Music Research 30(2), 107–135 (2001)

    Article  Google Scholar 

  5. Cooper, G., Meyer, L.B.: The Rhythmic Structure of Music. The Univ. of Chicago Press, Chicago (1960)

    Google Scholar 

  6. Daucé, E., Quoy, M., Doyon, B.: Resonant spatiotemporal learning in large random recurrent networks. Bological Cybernetics 87, 185–198 (2002)

    Article  MATH  Google Scholar 

  7. Eck, D.: Finding downbeats with a relaxation oscillator. Psychological Research 66(1), 18–25 (2002), http://www.iro.umontreal.ca/~eckdoug/papers/2002_psyres.pdf

    Article  MathSciNet  Google Scholar 

  8. Eck, D.: Finding long-timescale musical structure with an autocorrelation phase matrix. Music Perception 24(2), 167–176 (2006)

    Article  Google Scholar 

  9. Eck, D.: Generating music sequences with an echo state network. In: NIPS 2006 Workshop on Echo State Networks and Liquid State Machines, Whistler, British Columbia (2006)

    Google Scholar 

  10. Eck, D., Schmidhuber, J.: Finding temporal structure in music: Blues improvisation with LSTM recurrent networks. In: Bourlard, H. (ed.) Neural Networks for Signal Processing XII, Proceedings of the 2002 IEEE Workshop, pp. 747–756. IEEE, New York (2002)

    Chapter  Google Scholar 

  11. Gouyon, F.: A computational approach to rhythm detection. Phd thesis, Dpt. of Technology of the University Pompeu Fabra, Barcelona (2005), http://www.iua.upf.edu/mtg/publications/9d0455-PhD-Gouyon.pdf

  12. Hickok, G., Buchsbaum, B., Humphries, C., Muftuler, T.: Auditory-motor interaction revealed by fMRI: Speech, music, and working memory in area spt. Journal of Cognitive Neuroscience 15(5), 673–682 (2003)

    Google Scholar 

  13. Hochreiter, S., Schmidhuber, J.: Long Short-Term Memory. Neural Computation 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  14. Jaeger, H.: The ”echo state” approach to analysing and training recurrent neural networks. GMD Report 148, GMD - German National Research Institute for Computer Science (2001), http://www.faculty.jacobs-university.de/hjaeger/pubs/EchoStatesTechRep.pdf

  15. Jaeger, H.: Short term memory in echo state networks. GMD-Report 152, GMD - German National Research Institute for Computer Science (2002), http://www.faculty.jacobs-university.de/hjaeger/pubs/STMEchoStatesTechRep.pdf

  16. Jaeger, H.: Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the echo state network approach. GMD Report 159, Fraunhofer Institute AIS (2002), http://www.faculty.jacobs-university.de/hjaeger/pubs/ESNTutorial.pdf

  17. Jaeger, H.: Generating exponentially many periodic attractors with linearly growing echo state networks. IUB Technical Report 3, International University Bremen (2006), http://www.faculty.jacobs-university.de/hjaeger/pubs/techrep3.pdf

  18. Jaeger, H., Haas, H.: Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004), http://www.faculty.jacobs-university.de/hjaeger/pubs/ESNScience04.pdf

  19. Kaske, A., Maass, W.: A model for the interaction of oscillations and pattern generation with real-time computing in generic microcircuit models. Neural Networks 19(5), 600–609 (2006)

    Article  MATH  Google Scholar 

  20. Krumhansl, C.: Cognitive Foundations of Muiscal Pitch. Oxford University Press, Oxford (1990)

    Google Scholar 

  21. Large, E., Palmer, C.: Perceiving temporal regularity in music. Cognitive Science 26, 1–37 (2002)

    Article  Google Scholar 

  22. Large, E., Kolen, J.: Resonance and the perception of musical meter. Connection Science 6(2/3), 177–208 (1994)

    Article  Google Scholar 

  23. Maass, W., Joshi, P., Sontag, E.: Computational aspects of feedback in neural circuits. PLOS Computational Biology 3(1), 1–20 (2007)

    Article  MathSciNet  Google Scholar 

  24. Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14(11), 2531–2560 (2002), http://www.lsm.tugraz.at/papers/lsm-nc-130.pdf

    Article  MATH  Google Scholar 

  25. Mauk, M., Buonomano, D.: The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004)

    Article  Google Scholar 

  26. Mozer, M.C.: Neural network composition by prediction: Exploring the benefits of psychophysical constraints and multiscale processing. Cognitive Science 6, 247–280 (1994)

    Google Scholar 

  27. Palmer, C., Pfordresher, P.: Incremental planning in sequence production. Psychological Review 110, 683–712 (2003)

    Article  Google Scholar 

  28. Pasemann, F.: Characterization of periodic attractors in neural ring networks. Neural Networks 8(3), 421–429 (1995)

    Article  Google Scholar 

  29. Pongas, D., Billard, A., Schaal, S.: Rapid synchronization and accurate phase-locking of rhythmic motor primitives. In: Proc. of 2005 IEEE Conf. on Intelligent Robotics and Systems (IROS 2005), pp. 2911–2916 (2005)

    Google Scholar 

  30. Stevens, C., Wiles, J.: Representations of tonal music: A case study in the development of temporal relationship. In: Mozer, M., Smolensky, P., Touretsky, D., Elman, J., Weigend, A.S. (eds.) Proceedings of the 1993 Connectionist Models Summer School, pp. 228–235. Erlbaum, Hillsdale (1994)

    Google Scholar 

  31. Thaut, M., Kenyon, G., Schauer, M., McIntosh, G.: The connection between rhythmicity and brain function. IEEE Engineering in Medicine and Biology Magazine 18(2), 101–108 (1999)

    Article  Google Scholar 

  32. Todd, P.M.: A connectionist approach to algorithmic composition. Computer Music Journal 13(4), 27–43 (1989)

    Article  Google Scholar 

  33. van Gelder, T.: The dynamical hypothesis in cognitive science. Behavioural and Brain Sciences 21(5), 615–628 (1998)

    Article  Google Scholar 

  34. van Gelder, T., Port, R. (eds.): Mind as Motion: Explorations in the Dynamics of Cognition. Bradford/MIT Press (1995)

    Google Scholar 

  35. White, O.L., Lee, D.D., Sompolinsky, H.S.: Short-term memory in orthogonal neural networks. Phys. Rev. Lett. 92(14), 102–148 (2004)

    Article  Google Scholar 

  36. Widrow, B., Lehr, M.A.: Perceptrons, adalines, and backpropagation. In: Arbib, M. (ed.) The Handbook of Brain Theory and Neural Networks, pp. 719–724. MIT Press/Bradford Books (1995)

    Google Scholar 

  37. Williams, R.J., Zipser, D.: Gradient-based learning algorithms for recurrent networks and their computational complexity. In: Chauvin, Y., Rumelhart, D.E. (eds.) Back-propagation: Theory, Architectures and Applications, pp. 433–486. Erlbaum, Hillsdale (1992)

    Google Scholar 

  38. Yang, W., Chong, N.Y.: Goal-directed imitation with self-adjusting adaptor based on a neural oscillator network. In: Proceedings, 12th International Conference on Advanced Robotics, ICAR 2005, pp. 404–410 (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Jacobs University Bremen,  

    Herbert Jaeger

  2. Department of Computer Science, University of Montreal, Canada

    Douglas Eck

Authors
  1. Herbert Jaeger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Douglas Eck
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Ipke Wachsmuth Günther Knoblich

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Jaeger, H., Eck, D. (2008). Can’t Get You Out of My Head: A Connectionist Model of Cyclic Rehearsal. In: Wachsmuth, I., Knoblich, G. (eds) Modeling Communication with Robots and Virtual Humans. Lecture Notes in Computer Science(), vol 4930. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79037-2_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-79037-2_17

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-79036-5

  • Online ISBN: 978-3-540-79037-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation