Abstract
Neuromodulators can have a strong effect on how organisms cooperate and compete for resources. To better understand the effect of neuromodulation on cooperative behavior, a computational model of the dopaminergic and serotonergic systems was constructed and tested in games of conflict and cooperation. This neural model was based on the assumptions that dopaminergic activity increases as expected reward increases, and serotonergic activity increases as the expected cost of an action increases. The neural model guided the behavior of an agent that played a series of Hawk-Dove games against an opponent. The agent adapted its behavior appropriately to changes in environmental conditions and to changes in its opponent’s strategy. The neural agent tended to engage in Hawk-like behavior in low-risk situations and Dove-like behavior in high-risk situations. When the simulated dopaminergic activity was greater than the serotonergic activity, the agent tended to escalate a fight. These results suggest how the neuromodulatory systems shape decision-making and adaptive behavior in competitive and cooperative situations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)
Berridge, K.C.: Motivation concepts in behavioral neuroscience. Physiol. Behav. 81, 179–209 (2004)
Millan, M.J.: The neurobiology and control of anxious states. Prog. Neurobiol. 70, 83–244 (2003)
Crockett, M.J., Clark, L., Tabibnia, G., Lieberman, M.D., Robbins, T.W.: Serotonin modulates behavioral reactions to unfairness. Science 320, 1739 (2008)
Maynard Smith, J.: Evolution and the theory of games. Cambridge University Press, Cambridge (1982)
Wood, R.M., Rilling, J.K., Sanfey, A.G., Bhagwagar, Z., Rogers, R.D.: Effects of tryptophan depletion on the performance of an iterated Prisoner’s Dilemma game in healthy adults. Neuropsychopharmacology 31, 1075–1084 (2006)
Roussos, P., Giakoumaki, S.G., Pavlakis, S., Bitsios, P.: Planning, decision-making and the COMT rs4818 polymorphism in healthy males. Neuropsychologia 46, 757–763 (2008)
Axelrod, R., Hamilton, W.D.: The evolution of cooperation. Science 211, 1390–1396 (1981)
Hasselmo, M.E., McGaughy, J.: High acetylcholine levels set circuit dynamics for attention and encoding and low acetylcholine levels set dynamics for consolidation. Prog. Brain Res. 145, 207–231 (2004)
Krichmar, J.L.: The Neuromodulatory System – A Framework for Survival and Adaptive Behavior in a Challenging World. Adaptive Behavior 16, 385–399 (2008)
Cox, B.R., Krichmar, J.L.: Neuromodulation as a Robot Controller: A Brain Inspired Design Strategy for Controlling Autonomous Robots. IEEE Robotics & Automation Magazine 16, 72–80 (2009)
Nowak, M., Sigmund, K.: A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner’s Dilemma game. Nature 364, 56–58 (1993)
Amat, J., Paul, E., Zarza, C., Watkins, L.R., Maier, S.F.: Previous experience with behavioral control over stress blocks the behavioral and dorsal raphe nucleus activating effects of later uncontrollable stress: role of the ventral medial prefrontal cortex. J. Neurosci. 26, 13264–13272 (2006)
Watson, K.K., Ghodasra, J.H., Platt, M.L.: Serotonin transporter genotype modulates social reward and punishment in rhesus macaques. PLoS ONE e4156 (2009)
Cools, R., Roberts, A.C., Robbins, T.W.: Serotoninergic regulation of emotional and behavioural control processes. Trends Cogn. Sci. 12, 31–40 (2008)
Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002)
Denk, F., Walton, M.E., Jennings, K.A., Sharp, T., Rushworth, M.F., Bannerman, D.M.: Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology (Berl) 179, 587–596 (2005)
Pauli, W.M., O’Reilly, R.C.: Attentional control of associative learning-A possible role of the central cholinergic system. Brain Res. 1202, 43–53 (2008)
Aston-Jones, G., Cohen, J.D.: An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annual Review of Neuroscience 28, 403–450 (2005)
Doya, K.: Metalearning and neuromodulation. Neural Netw. 15, 495–506 (2002)
Schweighofer, N., Tanaka, S.C., Doya, K.: Serotonin and the evaluation of future rewards: theory, experiments, and possible neural mechanisms. Annals of the New York Academy of Sciences 1104, 289–300 (2007)
Redgrave, P., Gurney, K.: The short-latency dopamine signal: a role in discovering novel actions? Nature Reviews 7, 967–975 (2006)
Sandholm, T.W., Crites, R.H.: Multiagent reinforcement learning in the Iterated Prisoner’s Dilemma. Biosystems 37, 147–166 (1996)
Harrald, P.G., Fogel, D.B.: Evolving continuous behaviors in the Iterated Prisoner’s Dilemma. Biosystems 37, 135–145 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zaldivar, A., Asher, D.E., Krichmar, J.L. (2010). Simulation of How Neuromodulation Influences Cooperative Behavior. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, JA., Mouret, JB. (eds) From Animals to Animats 11. SAB 2010. Lecture Notes in Computer Science(), vol 6226. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15193-4_61
Download citation
DOI: https://doi.org/10.1007/978-3-642-15193-4_61
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15192-7
Online ISBN: 978-3-642-15193-4
eBook Packages: Computer ScienceComputer Science (R0)