Skip to main content
SpringerLink
Log in

Computational Models of Neuromodulation

  • Living reference work entry
  • First Online:
Encyclopedia of Computational Neuroscience

  • 206 Accesses

Definition

Neuromodulatory systems serve a special meta-processing role in the brain. Due to their anatomical privileges of having massively extensive arborization patterns throughout the central and peripheral nervous systems and to their physiological capacity of finely controlling how other neurons communicate with each other and plasticize, they are ideally positioned to regulate the way information is acquired, processed, utilized, and stored in the brain. As such, neuromodulation has been a fertile ground for computational models for neural information processing, which have strived to explain not only how the major neuromodulators coordinate to enable normal sensory, motor, and cognitive functions but also how dysfunctions arise in psychiatric and neurological conditions when these neuromodulatory systems are impaired.

Detailed Description

Although much still remains unknown or opaque about neuromodulatory functions, a computationally sophisticated and coherent picture is...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Ammassari-Teule M, Maho C, Sara SJ (1991) Clonidine reverses spatial learning deficits and reinstates θ frequencies in rats with partial fornix section. Behav Brain Res 45:1–8

    Article  CAS  PubMed  Google Scholar 

  • Aston-Jones G, Cohen JD (2005) An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci 28:403–450

    Article  CAS  PubMed  Google Scholar 

  • Battaglia PW, Jacobs RA, Aslin RN (2003) Bayesian integration of visual and auditory signals for spatial localization. J Opt Soc Am A Opt Image Sci Vis 20(7):1391–1397

    Article  PubMed  Google Scholar 

  • Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS (2007) Learning the value of information in an uncertain world. Nature Neurosci 10(9):1214–1221

    Article  CAS  PubMed  Google Scholar 

  • Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev 28:309–369

    Article  CAS  PubMed  Google Scholar 

  • Boureau Y-L, Dayan P (2011) Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36:74–97

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Cohen JD, Frank MJ (2009) Neurocomputational models of basal ganglia function in learning, memory and choice. Behav Brain Res 199:141–156

    Article  PubMed Central  PubMed  Google Scholar 

  • Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N (2012) Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482:85–88

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Cools R (2011) Dopaminergic control of the striatum for high-level cognition. Curr Opin Neurobiol 21:402–407

    Article  CAS  PubMed  Google Scholar 

  • Córdova, Yu, Chiba (2004) Soc Neurosci Abstr

    Google Scholar 

  • Coull JT, Frith CD, Dolan RJ, Frackowiak RS, Grasby PM (1997) The neural correlates of the noradrenergic modulation of human attention, arousal and learning. Eur J Neurosci 9(3):589–598

    Article  CAS  PubMed  Google Scholar 

  • Daw ND, Kakade S, Dayan P (2002) Opponent interactions between serotonin and dopamine. Neural Netw 15:603–616

    Article  PubMed  Google Scholar 

  • Dayan P (2012) Twenty-five lessons from computational neuromodulation. Neuron 76:240–256

    Article  CAS  PubMed  Google Scholar 

  • Dayanik S, Yu AJ (2012) Reward-rate maximization in sequential identification under a stochastic deadline. SIAM J Control Optim 51(4):2922–2948

    Article  Google Scholar 

  • Deakin JFW (1983) Roles of brain serotonergic neurons in escape, avoidance and other behaviors. J Psychopharmacol 43:563–577

    Google Scholar 

  • Doya K (2002) Metalearning and neuromodulation. Neural Netw 15(4–6):495–506

    Article  PubMed  Google Scholar 

  • Dyon-Laurent C, Romand S, Biegon A, Sara SJ (1993) Functional reorganization of the noradrenergic system after partial fornix section: a behavioral and autoradiographic study. Exp Brain Res 96:203–211

    Article  CAS  PubMed  Google Scholar 

  • Dyon-Laurent C, Hervé A, Sara SJ (1994) Noradrenergic hyperactivity in hippocampus after partial denervation: pharmacological, behavioral, and electrophysiological studies. Exp Brain Res 99:259–266

    Article  CAS  PubMed  Google Scholar 

  • Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429–433

    Article  CAS  PubMed  Google Scholar 

  • Fotiou DF, Stergiou V, Tsiptsios D, Lithari C, Nakou M, Karlovasitou A (2009) Cholinergic deficiency in Alzheimer’s and Parkinson’s disease: evaluation with pupillometry. Int J Psychophysiol 73(2):143–149

    Article  CAS  PubMed  Google Scholar 

  • Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci 17:51–72

    Article  PubMed  Google Scholar 

  • Frank MJ, Seeberger LC, OReilly RC (2004) By carrot or by stick: cognitive reinforcement learning in Parkinsonism. Science 306:1940–1943

    Article  CAS  PubMed  Google Scholar 

  • Gerfen CR, Engber TM, Mahan LC, Susel Z, Chase TN, Monsma FJJ, Sibley DR (1990) D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250:1429–1432

    Article  CAS  PubMed  Google Scholar 

  • Gilzenrat MS, Nieuwenhuis S, Jepma M, Cohen JD (2010) Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn Affect Behav Neurosci 10:252–269

    Article  PubMed Central  PubMed  Google Scholar 

  • Hutchins JB, Corbett JJ (1997) The visual system. In: Haines DE (ed) Fundamental neuroscience. Churchill Livingstone, New York, pp 265–284

    Google Scholar 

  • Ikemoto S, Panksepp J (1999) The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Rev 31:6–41

    Article  CAS  PubMed  Google Scholar 

  • Jepma M, Nieuwenhuis S (2011) Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory. J Cogn Neurosci 23:1587–1596

    Article  PubMed  Google Scholar 

  • Johnson J, Li W, Li J, Klopf A (2001) A computational model of learned avoidance behavior in a one-way avoidance experiment. Adapt Behav 9:91

    Article  Google Scholar 

  • Körding KP, Wolpert DM (2004) Bayesian integration in sensorimotor learning. Nature 427:244–247

    Article  PubMed  Google Scholar 

  • Kravitz AV, Tye LD, Kreitzer AC (2012) Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci 15:816–818

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Little JT, Johnson DN, Minichiello M, Weingartner H, Sunderland T (1998) Combined nicotinic and muscarinic blockade in elderly normal volunteers: cognitive, behavioral, and physiological responses. Neuropsychopharmacology 19(1):60–69

    Article  CAS  PubMed  Google Scholar 

  • Maia TV (2010) Two-factor theory, the actor-critic model, and conditioned avoidance. Learn Behav 38:50–67

    Article  PubMed  Google Scholar 

  • McClure SM, Daw ND, Montague PR (2003) A computational substrate for incentive salience. Trends Neurosci 26:423–428

    Article  CAS  PubMed  Google Scholar 

  • Miyazaki K, Miyazaki KW, Doya K (2011) Activation of dorsal Raphe serotonin neurons underlies waiting for delayed rewards. J Neurosci 31:469–479

    Article  CAS  PubMed  Google Scholar 

  • Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive hebbian learning. J Neurosci 16:1936–1947

    CAS  PubMed  Google Scholar 

  • Moutoussis M, Bentall RP, Williams J, Dayan P (2008) A temporal difference account of avoidance learning. Network 19:137–160

    Article  PubMed  Google Scholar 

  • Murschall A, Hauber W (2006) Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performance. Learn Mem 13:123–126

    Article  CAS  PubMed  Google Scholar 

  • Nakamura K, Hikosaka O (2006) Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J Neurosci 26:5360–5369

    Article  CAS  PubMed  Google Scholar 

  • Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI (2012) Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15:1040–1046

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity cost and the control of response vigor. Psychopharmacology (Berl) 191:507–520

    Article  CAS  Google Scholar 

  • Sara SJ (1989) Noradrenergic-cholinergic interaction: its possible role in memory dysfunction associated with senile dementia. Arch Gerontol Geriatr Suppl 1:99–108

    CAS  PubMed  Google Scholar 

  • Sara SJ, Dyon-Laurent C, Guibert B, Leviel V (1992) Noradrenergic hyperactivity after fornix section: role in cholinergic dependent memory performance. Exp Brain Res 89:125–132

    Article  CAS  PubMed  Google Scholar 

  • Satoh T, Nakai S, Sat T, Kimura M (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23:9913–9923

    CAS  PubMed  Google Scholar 

  • Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599

    Article  CAS  PubMed  Google Scholar 

  • Schweimer JV, Ungless MA (2010) Phasic responses in dorsal raphe serotonin neurons to noxious stimuli. Neuroscience 171:1209–1215

    Article  CAS  PubMed  Google Scholar 

  • Smith Y, Bevan MD, Shink E, Bolam JP (1998) Microcircuitry of the direct and indirect pathways of the basal ganglia. Neuroscience 86:353–387

    Article  CAS  PubMed  Google Scholar 

  • Surmeier DJ, Ding J, Day M, Wang Z, Shen W (2007) D1 and D2 dopamine receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci 30:228–235

    Article  CAS  PubMed  Google Scholar 

  • Surmeier DJ, Shen W, Day M, Gertler T, Chan S, Tian X, Plotkin JL (2010) The role of dopamine in modulating the structure and function of striatal circuits. Prog Brain Res 183:149–167

    CAS  PubMed  Google Scholar 

  • Sutton RS (1988) Learning to predict by the methods of temporal differences. Mach Learn 3(1):9–44

    Google Scholar 

  • Talmi D, Seymour B, Dayan P, Dolan RJ (2008) Human Pavlovian-instrumental transfer. J Neurosci 28:360–368

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Thiel CM, Fink GR (2008) Effects of the cholinergic agonist nicotine on reorienting of visual spatial attention and top-down attentional control. Neuroscience 152(2):381–390

    Article  CAS  PubMed  Google Scholar 

  • Ungless MA, Magill PJ, Bolam JP (2004) Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303:2040–2042

    Article  CAS  PubMed  Google Scholar 

  • Yu AJ, Dayan P (2003) Expected and unexpected uncertainty: ACh and NE in the neocortex. In: Becker STS, Obermayer K (eds) Advances in neural information processing systems 15. MIT Press, Cambridge, MA, pp 157–164

    Google Scholar 

  • Yu AJ, Dayan P (2005a) Inference, attention, and decision in a Bayesian neural architecture. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, Cambridge, MA

    Google Scholar 

  • Yu AJ, Dayan P (2005b) Uncertainty, neuromodulation, and attention. Neuron 46:681–692

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cognitive Science, University of California, San Diego 9500 Gilman Drive, Mail Code 0515, La Jolla, CA, 92103-0515, USA

    Prof. Angela J. Yu

Authors
  1. Prof. Angela J. Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Angela J. Yu .

Editor information

Editors and Affiliations

  1. Atlanta, Georgia, USA

    Dieter Jaeger

  2. Department of Biomedical Engineering, Florida International University, Miami, Florida, USA

    Ranu Jung

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this entry

Cite this entry

Yu, A.J. (2014). Computational Models of Neuromodulation. In: Jaeger, D., Jung, R. (eds) Encyclopedia of Computational Neuroscience. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7320-6_625-1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7320-6_625-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, New York, NY

  • Online ISBN: 978-1-4614-7320-6

  • eBook Packages: Springer Reference Biomedicine and Life SciencesReference Module Biomedical and Life Sciences

Publish with us

Policies and ethics

Search

Navigation