Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Evolution of cooperation through cumulative reciprocity

Nature Computational Science volume 2, pages 677–686 (2022)Cite this article

Subjects

Abstract

Reciprocity is a simple principle for cooperation that explains many of the patterns of how humans seek and receive help from each other. To capture reciprocity, traditional models often assume that individuals use simple strategies with restricted memory. These memory-1 strategies are mathematically convenient, but they miss important aspects of human reciprocity, where defections can have lasting effects. Here we instead propose a strategy of cumulative reciprocity. Cumulative reciprocators count the imbalance of cooperation across their previous interactions with their opponent. They cooperate as long as this imbalance is sufficiently small. Using analytical and computational methods, we show that this strategy can sustain cooperation in the presence of errors, that it enforces fair outcomes and that it evolves in hostile environments. Using an economic experiment, we confirm that cumulative reciprocity is more predictive of human behaviour than several classical strategies. The basic principle of cumulative reciprocity is versatile and can be extended to a range of social dilemmas.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: CURE in the repeated prisoner’s dilemma.
Fig. 2: The evolutionary dynamics of pairwise strategy competitions.
Fig. 3: Evolution of CURE in populations of memory-1 players.
Fig. 4: CURE, GTFT and WSLS dominate the population in different payoff regions.
Fig. 5: CURE in stochastic games and the repeated public goods game.
Fig. 6: CURE in an economic experiment.

Similar content being viewed by others

Data availability

Source data for Figs. 2, 3, 4 and 6 are available with this manuscript. Source data for Fig. 5 are available on Zenodo58.

Code availability

All Java codes based on Eclipse (version Oxygen.3a release 4.7.3a), MATLAB codes (MATLAB 2020b) and oTree software (version 5.8.0b5, based on Python 3.8.10) can be obtained from Code Ocean57.

References

  1. Trivers, R. L. The evolution of reciprocal altruism. Q. Rev. Biol. 46, 35–57 (1971).

    Article  Google Scholar 

  2. Axelrod, R. & Hamilton, W. D. The evolution of cooperation. Science 211, 1390–1396 (1981).

    Article  MathSciNet  Google Scholar 

  3. Nowak, M. A. Five rules for the evolution of cooperation. Science 314, 1560–1563 (2006).

    Article  Google Scholar 

  4. Rapoport, A., Chammah, A. M. & Orwant, C. J. Prisoner’s Dilemma: a Study in Conflict and Cooperation (Univ. of Michigan Press, 1965).

  5. Neilson, W. S. The economics of favors. J. Econ. Behav. Organ. 39, 387–397 (1999).

    Article  Google Scholar 

  6. Schweinfurth, M. K. & Call, J. Reciprocity: different behavioural strategies, cognitive mechanisms and psychological processes. Learn. Behav. 47, 284–301 (2019).

    Article  Google Scholar 

  7. Keohane, R. O. Reciprocity in international relations. Int. Organ. 40, 1–27 (1986).

    Article  Google Scholar 

  8. Friedman, J. W. A non-cooperative equilibrium for supergames. Rev. Econ. Stud. 38, 1–12 (1971).

    Article  MATH  Google Scholar 

  9. Hilbe, C., Chatterjee, K. & Nowak, M. A. Partners and rivals in direct reciprocity. Nat. Hum. Behav. 2, 469–477 (2018).

    Article  Google Scholar 

  10. Nowak, M. A. & Sigmund, K. Tit for tat in heterogeneous populations. Nature 355, 250–253 (1992).

    Article  Google Scholar 

  11. Molander, P. The optimal level of generosity in a selfish, uncertain environment. J. Confl. Resolut. 29, 611–618 (1985).

    Article  Google Scholar 

  12. Kraines, D. & Kraines, V. Pavlov and the prisoner’s dilemma. Theory Decis. 26, 47–79 (1989).

    Article  MathSciNet  MATH  Google Scholar 

  13. Nowak, M. & Sigmund, K. A strategy of win-stay, lose-shift that outperforms tit-for-tat in the prisoner’s dilemma game. Nature 364, 56–58 (1993).

    Article  Google Scholar 

  14. Hauert, C. & Schuster, H. G. Effects of increasing the number of players and memory size in the iterated prisoner’s dilemma: a numerical approach. Proc. R. Soc. Lond. Ser. B 264, 513–519 (1997).

    Article  Google Scholar 

  15. Grujić, J., Cuesta, J. A. & Sanchez, A. On the coexistence of cooperators, defectors and conditional cooperators in the multiplayer iterated prisoner’s dilemma. J. Theor. Biol. 300, 299–308 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  16. Stewart, A. J. & Plotkin, J. B. From extortion to generosity, evolution in the iterated prisoner’s dilemma. Proc. Natl Acad. Sci. U. S. A. 110, 15348–15353 (2013).

    Article  MathSciNet  MATH  Google Scholar 

  17. Stewart, A. J. & Plotkin, J. B. Collapse of cooperation in evolving games. Proc. Natl Acad. Sci. U. S. A. 111, 17558–17563 (2014).

    Article  Google Scholar 

  18. Akin, E. What you gotta know to play good in the iterated prisoner’s dilemma. Games 6, 175–190 (2015).

    Article  MathSciNet  MATH  Google Scholar 

  19. Akin, E. in Ergodic Theory: Advances in Dynamical Systems (ed. Idris Assani) 77–107 (De Gruyter, 2016).

  20. Hilbe, C., Martinez-Vaquero, L. A., Chatterjee, K. & Nowak, M. A. Memory-n strategies of direct reciprocity. Proc. Natl Acad. Sci. U. S. A. 114, 4715–4720 (2017).

    Article  Google Scholar 

  21. Murase, Y. & Baek, S. K. Five rules for friendly rivalry in direct reciprocity. Sci. Rep. 10, 1–9 (2020).

    Article  Google Scholar 

  22. Brandt, H. & Sigmund, K. The good, the bad and the discriminator—errors in direct and indirect reciprocity. J. Theor. Biol. 239, 183–194 (2006).

    Article  MathSciNet  MATH  Google Scholar 

  23. Baek, S. K., Jeong, H.-C., Hilbe, C. & Nowak, M. A. Comparing reactive and memory-one strategies of direct reciprocity. Sci. Rep. 6, 1–13 (2016).

    Article  Google Scholar 

  24. Hilbe, C., Wu, B., Traulsen, A. & Nowak, M. A. Cooperation and control in multiplayer social dilemmas. Proc. Natl Acad. Sci. U. S. A. 111, 16425–16430 (2014).

    Article  Google Scholar 

  25. Hilbe, C., Šimsa, Š., Chatterjee, K. & Nowak, M. A. Evolution of cooperation in stochastic games. Nature 559, 246–249 (2018).

    Article  Google Scholar 

  26. Su, Q., McAvoy, A., Wang, L. & Nowak, M. A. Evolutionary dynamics with game transitions. Proc. Natl Acad. Sci. U. S. A. 116, 25398–25404 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  27. Sigmund, K. The Calculus of Selfishness (Princeton Univ. Press, 2010).

  28. Do, Yi,S., Baek, S. K. & Choi, J.-K. Combination with anti-tit-for-tat remedies problems of tit-for-tat. J. Theor. Biol. 412, 1–7 (2017).

    Article  MathSciNet  MATH  Google Scholar 

  29. Press, W. H. & Dyson, F. J. Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent. Proc. Natl Acad. Sci. U .S. A. 109, 10409–10413 (2012).

    Article  MATH  Google Scholar 

  30. Hao, D., Rong, Z. & Zhou, T. Extortion under uncertainty: zero-determinant strategies in noisy games. Phys. Rev. E 91, 052803 (2015).

    Article  MathSciNet  Google Scholar 

  31. McAvoy, A. & Hauert, C. Autocratic strategies for iterated games with arbitrary action spaces. Proc. Natl Acad. Sci. U. S. A. 113, 3573–3578 (2016).

    Article  Google Scholar 

  32. Ichinose, G. & Masuda, N. Zero-determinant strategies in finitely repeated games. J. Theor. Biol. 438, 61–77 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  33. Reiter, J. G., Hilbe, C., Rand, D. G., Chatterjee, K. & Nowak, M. A. Crosstalk in concurrent repeated games impedes direct reciprocity and requires stronger levels of forgiveness. Nat. Commun. 9, 1–8 (2018).

    Article  Google Scholar 

  34. Schmid, L., Chatterjee, K., Hilbe, C. & Nowak, M. A. A unified framework of direct and indirect reciprocity. Nat. Hum. Behav. 5, 1292–1302 (2021).

    Article  Google Scholar 

  35. Boyd, R. Mistakes allow evolutionary stability in the repeated prisoner’s dilemma game. J. Theor. Biol. 136, 47–56 (1989).

    Article  MathSciNet  Google Scholar 

  36. Stewart, A. J. & Plotkin, J. B. Small groups and long memories promote cooperation. Sci. Rep. 6, 1–11 (2016).

    Article  Google Scholar 

  37. Xia, C., Gracia-Lázaro, C. & Moreno, Y. Effect of memory, intolerance, and second-order reputation on cooperation. Chaos 30, 063122 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  38. Van Segbroeck, S., Pacheco, J. M., Lenaerts, T. & Santos, F. C. Emergence of fairness in repeated group interactions. Phys. Rev. Lett. 108, 158104 (2012).

    Article  Google Scholar 

  39. Pinheiro, F. L., Vasconcelos, V. V., Santos, F. C., Pacheco, J. M. & Daunizeau, J. Evolution of all-or-none strategies in repeated public goods dilemmas. PLoS Comput. Biol. 10, e1003945 (2014).

    Article  Google Scholar 

  40. Van Veelen, M., García, J., Rand, D. G. & Nowak, M. A. Direct reciprocity in structured populations. Proc. Natl Acad. Sci. U. S. A. 109, 9929–9934 (2012).

    Article  MATH  Google Scholar 

  41. García, J. & van Veelen, M. In and out of equilibrium I: evolution of strategies in repeated games with discounting. J. Econ. Theory 161, 161–189 (2016).

    Article  MathSciNet  MATH  Google Scholar 

  42. Duersch, P., Oechssler, J. & Schipper, B. C. Unbeatable imitation. Games Econ. Behav. 76, 88–96 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  43. Duersch, P., Oechssler, J. & Schipper, B. C. When is tit-for-tat unbeatable? Int. J. Game Theory 43, 25–36 (2014).

    Article  MathSciNet  MATH  Google Scholar 

  44. Fudenberg, D. & Tirole, J. Game Theory (MIT Press, 1991).

  45. Nowak, M. A., Sasaki, A., Taylor, C. & Fudenberg, D. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646–650 (2004).

    Article  Google Scholar 

  46. Fudenberg, D., Rand, D. G. & Dreber, A. Slow to anger and fast to forgive: cooperation in an uncertain world. Am. Econ. Rev. 102, 720–749 (2012).

    Article  Google Scholar 

  47. Dal Bó, P. & Fréchette, G. R. Strategy choice in the infinitely repeated prisoner’s dilemma. Am. Econ. Rev. 109, 3929–3952 (2019).

    Article  Google Scholar 

  48. Shackelford, T. K. & Buss, D. M. Betrayal in mateships, friendships, and coalitions. Personal. Soc. Psychol. Bull. 22, 1151–1164 (1996).

    Article  Google Scholar 

  49. Smith, J. M. & Price, G. R. The logic of animal conflict. Nature 246, 15–18 (1973).

    Article  MATH  Google Scholar 

  50. Boyd, R. & Lorberbaum, J. P. No pure strategy is evolutionarily stable in the repeated prisoner’s dilemma game. Nature 327, 58–59 (1987).

    Article  Google Scholar 

  51. Bendor, J. & Swistak, P. Types of evolutionary stability and the problem of cooperation. Proc. Natl Acad. Sci. U. S. A. 92, 3596–3600 (1995).

    Article  MATH  Google Scholar 

  52. García, J. & van Veelen, M. No strategy can win in the repeated prisoner’s dilemma: linking game theory and computer simulations. Front. Robot. AI 5, 102 (2018).

    Article  Google Scholar 

  53. Glynatsi, N. E. & Knight, V. A. Using a theory of mind to find best responses to memory-one strategies. Sci. Rep. 10, 1–9 (2020).

    Article  Google Scholar 

  54. Hauert, C. & Stenull, O. Simple adaptive strategy wins the prisoner’s dilemma. J. Theor. Biol. 218, 261–272 (2002).

    Article  MathSciNet  Google Scholar 

  55. Akin, E. Good strategies for the iterated prisoner’s dilemma: Smale vs. Markov. J. Dyn. Games 4, 217–253 (2017).

    Article  MathSciNet  Google Scholar 

  56. Silk, J. B. in Genetic and Cultural Evolution of Cooperation (ed. Hammerstein, P.) pp 37–54 (MIT Press, 2003).

  57. Li, J. et al. Evolution of cooperation through cumulative reciprocity. Code Ocean https://doi.org/10.24433/CO.6894248.v1 (2022).

  58. Li, J., et al. Evolution of cooperation through cumulative reciprocity. Zenodo https://doi.org/10.5281/zenodo.7074407 (2022).

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under grant 71871042 (to H.X.) and the Humanities and Social Science Project of the Ministry of Education of China grant 18YJA630118 (to H.X.). Part of this work was conducted during H.X.’s visit at Sloan School of Management, Massachusetts Institute of Technology, supported by the Fulbright Visiting Scholar Program, jointly with the Chinese Scholarship Council. H.X. thanks P. Gloor for hosting the visit and D. Rand for discussion on an early draft of this work. H.X. and J.L. thank Y. Qian for technical assistance in preparing the simulation platform. C.H. acknowledges generous support by the European Research Council starting grant 850529: E-DIRECT.

Author information

Author notes

  1. These authors contributed equally: Christian Hilbe and Haoxiang Xia.

Authors and Affiliations

  1. Institute of Systems Engineering, Dalian University of Technology, Dalian, China

    Juan Li, Xiaowei Zhao, Bing Li & Haoxiang Xia

  2. Center for Big Data and Intelligent Decision-Making, Dalian University of Technology, Dalian, China

    Juan Li & Haoxiang Xia

  3. School of Software Technology, Dalian University of Technology, Dalian, China

    Xiaowei Zhao

  4. Max Planck Institute for Evolutionary Biology, Plön, Germany

    Charlotte S. L. Rossetti & Christian Hilbe

Authors
  1. Juan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaowei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Charlotte S. L. Rossetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christian Hilbe
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haoxiang Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

H.X. conceived the original idea for this work with the help of X.Z. and designed the computational experiments, collaboratively with C.H. J.L., X.Z. and B.L. wrote the computer programs for the simulations and implemented the simulations. C.H. derived the analytical results. J.L., C.H. and H.X. analysed the results based on X.Z. and B.L.’s earlier successive contributions. C.S.L.R. designed and implemented the game software for the behavioural experiment, and collected the data. C.H. and C.S.L.R. analysed the respective results. J.L., C.H. and H.X. wrote the manuscript, and J.L. prepared the figures. H.X. and C.H. supervised the study. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Christian Hilbe or Haoxiang Xia.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks Ethan Akin, Isamu Okada and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Handling editor: Kaitlin McCardle, in collaboration with the Nature Computational Science team. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information.

Supplementary Figs. 1–19, Tables 1–10 and Sections 1–11.

Reporting summary

Peer review file

Source data

Source Data Fig. 2.

Statistical source data.

Source Data Fig. 3.

Statistical source data.

Source Data Fig. 4.

Statistical source data.

Source Data Fig. 6.

Statistical source data.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Zhao, X., Li, B. et al. Evolution of cooperation through cumulative reciprocity. Nat Comput Sci 2, 677–686 (2022). https://doi.org/10.1038/s43588-022-00334-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-022-00334-w

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing