Skip to main content
SpringerLink
Log in

Oscillatory Network and Deep Value Network Based Memory Replay Model of Hippocampus

  • Conference paper
  • First Online:
Pattern Recognition and Machine Intelligence (PReMI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14301))

  • 578 Accesses

Abstract

Memory replay is crucial for learning and consolidation. Hippocampal place cells demonstrate neuronal replay of behavioral sequences at a faster timescale in forward and reverse directions during resting states (awake and sleep). We propose a model of the hippocampus to demonstrate replay characteristics. The model comprises two parts - a Neural Oscillator Network to simulate replay and a Deep Value network to learn value function. The Neural Oscillator Network learns the input signal and allows modulation of the speed and direction of replay of the learned signal by modifying a single parameter. Combining reward information with the input signal and when trained with the Deep Value Network, reverse replay achieves faster learning of associations than forward replay in case of a rewarding sequence. The proposed model also explains the changes observed in the replay rate in an experimental study in which a rodent explores a linear track with changing reward conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Carr, M.F., Jadhav, S.P., Frank, L.M.: Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nat. Neurosci. 14, 147–153 (2011). https://doi.org/10.1038/NN.2732

    Article  Google Scholar 

  2. Foster, D.J., Knierim, J.J.: Sequence learning and the role of the hippocampus in rodent navigation. Curr. Opin. Neurobiol. 22, 294–300 (2012). https://doi.org/10.1016/J.CONB.2011.12.005

    Article  Google Scholar 

  3. Pfeiffer, B.E., Foster, D.J.: Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497, 74–79 (2013). https://doi.org/10.1038/NATURE12112

    Article  Google Scholar 

  4. Foster, D.J., Wilson, M.A.: Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680–683 (2006). https://doi.org/10.1038/NATURE04587

    Article  Google Scholar 

  5. Wilson, M.A., McNaughton, B.L.: Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679 (1994). https://doi.org/10.1126/SCIENCE.8036517

    Article  Google Scholar 

  6. Buzsáki, G.: Hippocampal sharp wave-ripple: a cognitive biomarker for episodic memory and planning. Hippocampus 25, 1073–1188 (2015). https://doi.org/10.1002/HIPO.22488

    Article  Google Scholar 

  7. Levy, W.B.: A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal-like tasks. Hippocampus 6, 570–590 (1996)

    Article  Google Scholar 

  8. Kali, S., Dayan, P.: The involvement of recurrent connections in area CA3 in establishing the properties of place fields: a model. J. Neurosci. 20, 7463–7477 (2000). https://doi.org/10.1523/JNEUROSCI.20-19-07463.2000

    Article  Google Scholar 

  9. Davoudi, H., Foster, D.J.: Acute silencing of hippocampal CA3 reveals a dominant role in place field responses. Nat. Neurosci. 22, 337–342 (2019). https://doi.org/10.1038/S41593-018-0321-Z

    Article  Google Scholar 

  10. Michelmann, S., Staresina, B.P., Bowman, H., Hanslmayr, S.: Speed of time-compressed forward replay flexibly changes in human episodic memory. Nat. Hum. Behav. 3, 143–154 (2019). https://doi.org/10.1038/S41562-018-0491-4

    Article  Google Scholar 

  11. Diba, K., Buzsáki, G.: Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241–1242 (2007). https://doi.org/10.1038/NN1961

    Article  Google Scholar 

  12. Ambrose, R.E., Pfeiffer, B.E., Foster, D.J.: Reverse replay of hippocampal place cells is uniquely modulated by changing reward. Neuron 91, 1124–1136 (2016). https://doi.org/10.1016/J.NEURON.2016.07.047

    Article  Google Scholar 

  13. Singer, A.C., Frank, L.M.: Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 64, 910–921 (2009). https://doi.org/10.1016/J.NEURON.2009.11.016

    Article  Google Scholar 

  14. Momennejad, I., Otto, A.R., Daw, N.D., Norman, K.A.: Offline replay supports planning in human reinforcement learning. Elife 7 (2018). https://doi.org/10.7554/ELIFE.32548

  15. Schapiro, A.C., McDevitt, E.A., Rogers, T.T., Mednick, S.C., Norman, K.A.: Human hippocampal replay during rest prioritizes weakly learned information and predicts memory performance. Nat Commun. 9 (2018). https://doi.org/10.1038/S41467-018-06213-1

  16. Liu, Y., Dolan, R.J., Kurth-Nelson, Z., Behrens, T.E.J.: Human replay spontaneously reorganizes experience. Cell 178, 640-652.e14 (2019). https://doi.org/10.1016/J.CELL.2019.06.012

    Article  Google Scholar 

  17. Schuck, N.W., Niv, Y.: Sequential replay of nonspatial task states in the human hippocampus. Science. 364 (2019). https://doi.org/10.1126/SCIENCE.AAW5181

  18. Buhry, L., Azizi, A.H., Cheng, S.: Reactivation, replay, and preplay: how it might all fit together. Neural Plast. 2011 (2011). https://doi.org/10.1155/2011/203462

  19. Findlay, G., Tononi, G., Cirelli, C.: The evolving view of replay and its functions in wake and sleep. Sleep Adv. 1 (2021). https://doi.org/10.1093/SLEEPADVANCES/ZPAB002

  20. August, D.A., Levy, W.B.: Temporal sequence compression by an integrate-and-fire model of hippocampal area CA3. J. Comput. Neurosci. 6, 71–90 (1999). https://doi.org/10.1023/A:1008861001091

    Article  MATH  Google Scholar 

  21. Shen, B., McNaughton, B.L.: Modeling the spontaneous reactivation of experience-specific hippocampal cell assembles during sleep. Hippocampus 6, 685–692 (1996)

    Article  Google Scholar 

  22. Milstein, A.D., Tran, S., Ng, G., Soltesz, I.: Offline memory replay in recurrent neuronal networks emerges from constraints on online dynamics. J. Physiol. (2022). https://doi.org/10.1113/JP283216

  23. Jahnke, S., Timme, M., Memmesheimer, R.M.: A unified dynamic model for learning, replay, and sharp-wave/ripples. J. Neurosci. 35, 16236–16258 (2015). https://doi.org/10.1523/JNEUROSCI.3977-14.2015

    Article  Google Scholar 

  24. Biswas, D., Pallikkulath, S., Chakravarthy, V.S.: A complex-valued oscillatory neural network for storage and retrieval of multidimensional aperiodic signals. Front. Comput. Neurosci. 15 (2021). https://doi.org/10.3389/FNCOM.2021.551111

  25. Mattar, M.G., Daw, N.D.: Prioritized memory access explains planning and hippocampal replay. Nat. Neurosci. 21, 1609–1617 (2018). https://doi.org/10.1038/S41593-018-0232-Z

    Article  Google Scholar 

  26. Righetti, L., Buchli, J., Ijspeert, A.J.: From Dynamic hebbian learning for oscillators to adaptive central pattern generators. In: Proceedings of 3rd International Symposium on Adaptive Motion in Animals and Machines -- AMAM 2005, p. 45 (2005)

    Google Scholar 

  27. Righetti, L., Buchli, J., Ijspeert, A.J.: Adaptive frequency oscillators and applications. Open Cybern. Syst. J. 3, 64–69 (2009). https://doi.org/10.2174/1874110X00903020064

    Article  MathSciNet  MATH  Google Scholar 

  28. Atherton, L.A., Dupret, D., Mellor, J.R.: Memory trace replay: the shaping of memory consolidation by neuromodulation. Trends Neurosci. 38, 560 (2015). https://doi.org/10.1016/J.TINS.2015.07.004

    Article  Google Scholar 

  29. Foster, D.J.: Replay comes of age. Annu. Rev. Neurosci. 40, 581–602 (2017). https://doi.org/10.1146/ANNUREV-NEURO-072116-031538

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory for Computational Neuroscience, Department of Biotechnology, Bhupat and Jyoti Mehta School of Biosciences, Indian Institute of Technology Madras, Chennai, Tamil Nadu, India

    Tamizharasan Kanagamani, Madhuvanthi Muliya & V. Srinivasa Chakravarthy

  2. Department of Computer Science and Engineering, Robert Bosch Centre for Data Science and AI, Indian Institute of Technology Madras, Chennai, Tamil Nadu, India

    Balaraman Ravindran

  3. Cognition and Behavioural Neurology Section, Department of Neurology, Sree Chitra Tirunal Institute for Medical Sciences and Technology, Trivandrum, Kerala, India

    Ramshekhar N. Menon

Authors
  1. Tamizharasan Kanagamani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Madhuvanthi Muliya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. Srinivasa Chakravarthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Balaraman Ravindran
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ramshekhar N. Menon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tamizharasan Kanagamani .

Editor information

Editors and Affiliations

  1. Indian Statistical Institute, Kolkata, India

    Pradipta Maji

  2. Texas A&M University at Qatar, Doha, Qatar

    Tingwen Huang

  3. Indian Statistical Institute, Kolkata, West Bengal, India

    Nikhil R. Pal

  4. Indian Institute of Technology Jodhpur, Jodhpur, India

    Santanu Chaudhury

  5. Indian Statistical Institute, Kolkata, West Bengal, India

    Rajat K. De

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kanagamani, T., Muliya, M., Chakravarthy, V.S., Ravindran, B., Menon, R.N. (2023). Oscillatory Network and Deep Value Network Based Memory Replay Model of Hippocampus. In: Maji, P., Huang, T., Pal, N.R., Chaudhury, S., De, R.K. (eds) Pattern Recognition and Machine Intelligence. PReMI 2023. Lecture Notes in Computer Science, vol 14301. Springer, Cham. https://doi.org/10.1007/978-3-031-45170-6_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45170-6_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45169-0

  • Online ISBN: 978-3-031-45170-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation