Skip to main content
SpringerLink
Log in

Selective attention to historical comparison or social comparison in the evolutionary iterated prisoner’s dilemma game

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

This paper investigates an evolutionary iterated prisoner’s dilemma (IPD) model of multiple agents, in which agents interact in terms of the pair-wise IPD game while adapting their attitudes towards income stream risk. Specifically, agents will become more risk averse (or more risk seeking) if their game payoffs exceed (or fall below) their expectations. In particular, agents use their peers’ average payoffs as expectations (social comparison) when their payoffs are lower than their peers’ averages, but use their own historical payoffs as expectations (historical comparison) when their payoffs are higher than their peers’ averages. Such selective attention to social comparison or historical comparison manifests a desire for continuous improvement of agents. Simulations are conducted to investigate the evolution of cooperation under the selective attention mechanism. Results indicate that agents can sustain a highly cooperative equilibrium when they consider selective attention in adjusting their risk attitudes. This holds true for both the well-mixed and the network-based games, even in the presence of uncertain game payoffs. The reason is that, selective attention can significantly induce agents to adhere to conditional cooperation as well as to identify uncertainty in payoffs, which enhances the risk-averse behavior of agents in the IPD game. As a result, high levels of cooperation can be attained.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. The prospect theory used the terms expectations and aspirations interchangeably (Kahneman and Tversky 1979).

  2. Here, what we mean is that, individuals or organizations tend to use their own past performances as expectations (i.e., historical comparison) if they outperform their peers’ averages, but they tend to use their peers’ average performances as expectations (i.e., social comparison) if they underperform their peers’ averages. It is worth noting that the prerequisite for the application of historical comparison incorporates a social-comparison-like process, in which individuals or organizations need to confirm that they do be above their peers’ averages before implementing historical comparison. However, for the purpose of clarity, we explicitly term this circumstance as “historical comparison”, because it is the individuals’ or the organizations’ own past performances that are directly used as their expectations in this circumstance. In other words, it is historical comparison that mainly determines the individuals’ or the organizations’ expectations. On the other hand, when the term “social comparison” is presented, it refers to the case in which individuals or organizations directly use their peers’ average performances as their expectations.

  3. Agents with \(\alpha > 0.8\) are found to perform similarly well in the IPD game.

  4. The literature has suggested that an IPD strategy should cover histories of only recent interactions, because only recent moves have significance for the current move (Darwen and Yao 1997). The consideration of previous three moves has been widely exploited in the literature (Axelrod 1984; Franken and Engelbrecht 2005; Mittal and Deb 2009).

  5. The game-playing process is independent across generations, which means that each agent will start new IPD games with \(g_{n}\) new opponents, respectively, in a new generation. Still, each IPD game entails l encounters.

  6. Note that the reference-performance selection process is cost-effective, because each agent uses only information on its own payoff in the prior generation and information on the population average payoff in the current generation. The population average payoff, like, the average profit of an industry or a sub-industry, is always public information.

  7. According to previous work in Zeng et al. (2016a, 2017), the value of \(\gamma\) indicates the sensitivity of changes in agents’ risk attitudes. A too-small value of \(\gamma\), e.g., \(\gamma = 0\) (or a too-large value of \(\gamma\), e.g., \(\gamma = 0.5\)), means that the agents might change their risk attitudes too frequently (or too slowly) in response to the game outcomes. These inappropriate adjustment speeds may prevent agents from adapting to the game environment, and, therefore, have a disruptive impact on the evolutionary outcome. Thus, the value of \(\gamma\) is set as 0.15 in this study, as suggested in Zeng et al. (2016a). Meanwhile, because comparable results are obtained for different specifications of \(r_{up}\) and \(r_{down}\), we use \(r_{up} = r_{down} = 0.2\) for illustration. These parameter settings are also convenient for the comparison of results with selective attention (i.e., results in the current study) to those without (Zeng et al. 2016a, 2017).

  8. Simulation results are qualitatively identical with the population size and the average neighborhood size in a wide range of values, respectively. We specify N as 256 for convenience in constructing the grid network as a \(16 \times 16\) lattice. In addition, the neighborhood is defined as the \(5 \times 5\) lattice around a focal agent in the grid network, which results in \(24 = 5 \times 5 - 1\) neighbors for each agent. The construction of the other networks is relatively easy with any given population size and neighborhood size. Furthermore, the specification of population size as 256 and the definition of neighborhood size as 24 help in providing comparable results to those achieved in our previous studies (Zeng et al. 2016a, 2017), in which homogeneous historical comparison or social comparison was used for agents’ risk attitude adaptation. Note that locality of interaction can be guaranteed when the neighborhood size of 24 is compared to the population size of 256, which brings heterogeneity into interactions of agents (Zeng et al. 2017).

  9. Note that \(\xi_{1} {, }\xi_{{2}} {, }\xi_{{3}} {, }\xi_{{4}}\) are independently drawn for the four payoffs \(T,R,P,S\). The resulting \((T^{\prime},R^{\prime},P^{\prime},S^{\prime})\) that do not satisfy the constraint \(T^{\prime} > R^{\prime} > P^{\prime} > S^{\prime}\) or \(2R^{\prime} > T^{\prime} + S^{\prime}\) will be excluded for sustaining the IPD-payoff pattern.

  10. In our experiments, the average result over 20 runs is not statistically different from the average results over 30, 40, or 50 runs (with a 95% confidence level).

  11. The value of 2.6 is used here as an indication of the emergence of high levels of cooperation, which is higher than the average payoff of 2.5 that agents obtain from the move “Cooperate–Defect” or “Defect–-Cooperate” (Fogel 1995).

  12. The clustering coefficients of the ring, ring-based small-world, scale-free, grid, grid-based small-world, and random networks are 0.72, 0.63, 0.75, 0.52, 0.46, and 0.09, respectively; and their characteristic path lengths are 5.82, 2.58, 2.14, 2.93, 2.46, and 2.00, respectively. According to Jun and Sethi (2007), the clustering coefficient indicates the degree to which agents form local clusters, and the characteristic path length implies the average distance between agents in the network.

  13. Experiments using smaller neighborhood sizes or larger population sizes, which generally involve longer characteristic path lengths, also indicate that long characteristic path lengths impede global cooperation in the network-based IPD game.

  14. Although the population average risk attitude declines from about 0.9 to about 0.6 as \(\sigma\) is changed from 0 to 3 (Fig. 8a), agents with risk attitudes of around 0.6, similar to those with 0.9, can consistently cooperate with each other in the IPD game (Zeng et al. 2016b). In addition, an inspection on results by generation shows that the population average risk attitude persistently increases to and finally stabilizes around 0.6 when \(\sigma = 3\), which indicates the steady formation of global cooperation in the evolution.

References

  • Aranda C, Arellano J, Davila A (2017) Organizational learning in target setting. Acad Manag J 60(3):1189–1211

    Google Scholar 

  • Axelrod R (1984) The evolution of cooperation. Basic Books, New York

    MATH  Google Scholar 

  • Baggio JA, Papyrakis E (2014) Agent-based simulations of subjective well-being. Soc Indic Res 115(2):623–635

    Google Scholar 

  • Blettner DP, He ZL, Hu S, Bettis RA (2015) Adaptive aspirations and performance heterogeneity: attention allocation among multiple reference points. Strateg Manag J 36(7):987–1005

    Google Scholar 

  • Boyd R, Lorberbaum JP (1987) No pure strategy is evolutionarily stable in the repeated prisoner's dilemma game. Nature 327(6117):58–59

    Google Scholar 

  • Bromiley P (1991) Testing a causal model of corporate risk taking and performance. Acad Manag J 34(1):37–59

    Google Scholar 

  • Chen S-H (2008) Software-agent designs in economics: an interdisciplinary. IEEE Comput Intell Mag 3(4):18–22

    Google Scholar 

  • Chen S-H (2012) Varieties of agents in agent-based computational economics: a historical and an interdisciplinary perspective. J Econ Dyn Control 36(1):1–25

    MathSciNet  MATH  Google Scholar 

  • Chen Y-S, Yang H-X, Guo W-Z (2017) Aspiration-induced dormancy promotes cooperation in the spatial prisoner’s dilemma games. Physica A 469:625–630

    MATH  Google Scholar 

  • Chiong R, Kirley M (2012) Effects of iterated interactions in multiplayer spatial evolutionary games. IEEE Trans Evol Comput 16(4):537–555

    Google Scholar 

  • Cyert RM, March JG (1963) A behavioral theory of the firm. Prentice Hall, Englewood Cliffs, NJ

    Google Scholar 

  • Darwen PJ, Yao X (1997) Speciation as automatic categorical modularization. IEEE Trans Evol Comput 1(2):101–108

    Google Scholar 

  • Dato S, Grunewald A, Müller D, Strack P (2017) Expectation-based loss aversion and strategic interaction. Games Econ Behav 104(Supplement C):681–705

    MathSciNet  MATH  Google Scholar 

  • Daylamani-Zad D, Agius H, Angelides MC (2020) Reflective agents for personalisation in collaborative games. Artif Intell Rev 53:429–474

    Google Scholar 

  • Dreber A, Fudenberg D, Rand DG (2014) Who cooperates in repeated games: the role of altruism, inequity aversion, and demographics. J Econ Behav Organ 98:41–55

    Google Scholar 

  • Fafchamps M, Kebede B, Zizzo DJ (2015) Keep up with the winners: experimental evidence on risk taking, asset integration, and peer effects. Eur Econ Rev 79:59–79

    Google Scholar 

  • Festinger L (1954) A theory of social comparison processes. Hum Relat 7(2):117–140

    Google Scholar 

  • Fiegenbaum A (1990) Prospect theory and the risk-return association: an empirical examination in 85 industries. J Econ Behav Organ 14(2):187–203

    Google Scholar 

  • Fiegenbaum A, Thomas H (1995) Strategic groups as reference groups: theory, modeling and empirical examination of industry and competitive strategy. Strateg Manag J 16(6):461–476

    Google Scholar 

  • Fogel DB (1995) On the relationship between the duration of an encounter and the evolution of cooperation in the iterated prisoner's dilemma. Evol Comput 3(3):349–363

    Google Scholar 

  • Franken N, Engelbrecht AP (2005) Particle swarm optimization approaches to coevolve strategies for the iterated prisoner's dilemma. IEEE Trans Evol Comput 9(6):562–579

    Google Scholar 

  • Fudenberg D, Imhof LA (2008) Monotone imitation dynamics in large populations. J Econ Theory 140(1):229–245

    MathSciNet  MATH  Google Scholar 

  • Fudenberg D, Maskin E (1986) The folk theorem in repeated games with discounting or with incomplete information. Econometrica 54(3):533–554

    MathSciNet  MATH  Google Scholar 

  • García J, van Veelen M (2016) In and out of equilibrium I: evolution of strategies in repeated games with discounting. J Econ Theory 161:161–189

    MathSciNet  MATH  Google Scholar 

  • García J, van Veelen M (2018) No strategy can win in the repeated prisoner's dilemma: linking game theory and computer simulations. Front Robot AI 5:102

    Google Scholar 

  • Ghatak A, Mukherjee D, Mallikarjuna Rao KS (2018) A spatial game theoretic analysis of conflict and identity. Comput Econ 52(2):493–519

    Google Scholar 

  • Hodgson GM, Huang K (2012) Evolutionary game theory and evolutionary economics: are they different species? J Evol Econ 22(2):345–366

    Google Scholar 

  • Holton GA (2004) Defining risk. Financ Anal J 60(6):19–25

    Google Scholar 

  • Howley E, O’Riordan C (2006) The effects of viscosity in choice and refusal IPD environments. Artif Intell Rev 26(1):103–114

    Google Scholar 

  • Ioannou C (2014) Coevolution of finite automata with errors. J Evol Econ 24(3):541–571

    Google Scholar 

  • Ishibuchi H, Ohyanagi H, Nojima Y (2011) Evolution of strategies with different representation schemes in a spatial iterated prisoner's dilemma game. IEEE Trans Comput Intell AI Games 3(1):67–82

    Google Scholar 

  • Joseph J, Gaba V (2015) The fog of feedback: ambiguity and firm responses to multiple aspiration levels. Strateg Manag J 36(13):1960–1978

    Google Scholar 

  • Jun T, Sethi R (2007) Neighborhood structure and the evolution of cooperation. J Evol Econ 17(5):623–646

    Google Scholar 

  • Jun T, Sethi R (2009) Reciprocity in evolving social networks. J Evol Econ 19(3):379–396

    Google Scholar 

  • Kahneman D, Tversky A (1979) Prospect theory: an analysis of decision under risk. Econom J Econom Soc 47:263–291

    MathSciNet  MATH  Google Scholar 

  • Klemm K, Eguiluz VM (2002) Growing scale-free networks with small-world behavior. Phys Rev E 65(5):057102

    Google Scholar 

  • Li J, Zhang C, Sun Q, Chen Z, Zhang J (2017) Changing the intensity of interaction based on individual behavior in the iterated prisoner's dilemma game. IEEE Trans Evol Comput 21(4):506–517

    Google Scholar 

  • March JG, Shapira Z (1987) Managerial perspectives on risk and risk taking. Manag Sci 33(11):1404–1418

    Google Scholar 

  • Miller KD, Bromiley P (1990) Strategic risk and corporate performance: an analysis of alternative risk measures. Acad Manag J 33(4):756–779

    Google Scholar 

  • Mittal S, Deb K (2009) Optimal strategies of the iterated prisoner's dilemma problem for multiple conflicting objectives. IEEE Trans Evol Comput 13(3):554–565

    Google Scholar 

  • Moliterno TP, Beck N, Beckman CM, Meyer M (2014) Knowing your place: social performance feedback in good times and bad times. Organ Sci 25(6):1684–1702

    Google Scholar 

  • Muto N (2014) Strategic complexity in repeated extensive games. Games Econ Behav 83:45–52

    MathSciNet  MATH  Google Scholar 

  • Nowak MA, Sasaki A, Taylor C, Fudenberg D (2004) Emergence of cooperation and evolutionary stability in finite populations. Nature 428(6983):646–650

    Google Scholar 

  • Osório A (2018) Brownian signals: information quality, quantity and timing in repeated games. Comput Econ 52(2):387–404

    Google Scholar 

  • Park KM (2007) Antecedents of convergence and divergence in strategic positioning: the effects of performance and aspiration on the direction of strategic change. Organ Sci 18(3):386–402

    Google Scholar 

  • Płatkowski T (2015) Aspiration-based full cooperation in finite systems of players. Appl Math Comput 251:46–54

    MathSciNet  MATH  Google Scholar 

  • Rong Z-H, Zhao Q, Wu Z-X, Zhou T, Tse CK (2016) Proper aspiration level promotes generous behavior in the spatial prisoner’s dilemma game. Eur Phys J B 89(7):166

    MathSciNet  Google Scholar 

  • Safarzyńska K, van den Bergh JC (2010) Evolutionary models in economics: a survey of methods and building blocks. J Evol Econ 20(3):329–373

    Google Scholar 

  • Shutters ST (2012) Punishment leads to cooperative behavior in structured societies. Evol Comput 20(2):301–319

    Google Scholar 

  • Stahl DO (2011) Cooperation in the sporadically repeated prisoners’ dilemma via reputation mechanisms. J Evol Econ 21(4):687–702

    Google Scholar 

  • Stahl DO (2013) An experimental test of the efficacy of a simple reputation mechanism to solve social dilemmas. J Econ Behav Organ 94:116–124

    Google Scholar 

  • Sujil A, Verma J, Kumar R (2018) Multi agent system: concepts, platforms and applications in power systems. Artif Intell Rev 49(2):153–182

    Google Scholar 

  • Thaler RH (2005) Advances in behavioral finance. Princeton University Press, Princeton

    Google Scholar 

  • Tomasello M (2000) The cultural origins of human cognition. Harvard University Press, Cambridge, MA

    Google Scholar 

  • Uzzi B, Amaral LAN, Reed-Tsochas F (2007) Small-world networks and management science research: a review. Eur Manag Rev 4(2):77–91

    Google Scholar 

  • van Veelen M, García J, Rand DG, Nowak MA (2012) Direct reciprocity in structured populations. Proc Natl Acad Sci 109(25):9929–9934

    MATH  Google Scholar 

  • Wang H, Yan J, Yu J (2017a) Reference-dependent preferences and the risk–return trade-off. J Financ Econ 123(2):395–414

    MathSciNet  Google Scholar 

  • Wang IK, Qian L, Lehrer M (2017b) From technology race to technology marathon: a behavioral explanation of technology advancement. Eur Manag J 35(2):187–197

    Google Scholar 

  • Watts DJ, Strogatz SH (1998) Collective dynamics of 'small-world' networks. Nature 393(6684):440–442

    MATH  Google Scholar 

  • Wilson AJ, Wu H (2017) At-will relationships: how an option to walk away affects cooperation and efficiency. Games Econ Behav 102(Supplement C):487–507

    MathSciNet  MATH  Google Scholar 

  • Xian Y, Chen P (2011) Does social welfare preference always promote cooperation on Barabási and Albert networks? Comput Econ 37(3):249–266

    Google Scholar 

  • Xie N-G, Zhen K-X, Wang C, Ye Y, Wang L (2015) Evolution of cooperation driven by the diversity of emotions. Connect Sci 27(1):89–101

    Google Scholar 

  • Zeng W, Li M, Chen F (2016a) Cooperation in the evolutionary iterated prisoner’s dilemma game with risk attitude adaptation. Appl Soft Comput 44(Supplement C):238–254

    Google Scholar 

  • Zeng W, Li M, Chen F, Nan G (2016b) Risk consideration and cooperation in the iterated prisoner’s dilemma. Soft Comput 20(2):567–587

    Google Scholar 

  • Zeng W, Li M, Feng N (2017) The effects of heterogeneous interaction and risk attitude adaptation on the evolution of cooperation. J Evol Econ 27(3):435–459

    Google Scholar 

  • Zhang H (2018) Errors can increase cooperation in finite populations. Games Econ Behav 107(Supplement C):203–219

    MathSciNet  MATH  Google Scholar 

  • Zhang H, Gao M, Wang W, Liu Z (2014) Evolutionary prisoner׳s dilemma game on graphs and social networks with external constraint. J Theor Biol 358:122–131

    MathSciNet  MATH  Google Scholar 

  • Zschache J (2012) Producing public goods in networks: some effects of social comparison and endogenous network change. Soc Netw 34(4):539–548

    Google Scholar 

Download references

Acknowledgements

This study was supported by National Natural Science Foundation of China (Grant No. 71702040), and Natural Science Foundation of Hainan Province, China (Grant No. 20167244). The authors would also like to thank the High Performance Computing Centre (HPCC) of Tianjin University for providing the computing support.

Author information

Authors and Affiliations

  1. School of Management, Hainan University, Haikou, 570228, People’s Republic of China

    Weijun Zeng

  2. College of Management and Economics, Tianjin University, Tianjin, 300072, People’s Republic of China

    Minqiang Li

Authors
  1. Weijun Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weijun Zeng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, W., Li, M. Selective attention to historical comparison or social comparison in the evolutionary iterated prisoner’s dilemma game. Artif Intell Rev 53, 6043–6078 (2020). https://doi.org/10.1007/s10462-020-09842-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-020-09842-5

Keywords

JEL Classification

Search

Navigation