Skip to main content

Advertisement

SpringerLink
Log in

Strategic adaptation of humans playing computer algorithms in a repeated constant-sum game

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

This paper examines strategic adaptation in participants’ behavior conditional on the type of their opponent. Participants played a constant-sum game for 100 rounds against each of three pattern-detecting computer algorithms designed to exploit regularities in human behavior such as imperfections in randomizing and the use of simple heuristics. Significant evidence is presented that human participants not only change their marginal probabilities of choosing actions, but also their conditional probabilities dependent on the recent history of play. A cognitive model incorporating pattern recognition is proposed that capture the shifts in strategic behavior of the participants better than the standard non-pattern detecting model employed in the literature, the Experience Weighted Attraction model (and by extension its nested models, reinforcement learning and fictitious play belief learning).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

t :

Time measured by the number of rounds

i :

Indexes players

−i :

Indexes the matched opponent of player i

a i,t :

Action played at time t by player i

n :

Depth of pattern recognition

ω t :

A vector of the context (history of play) at time t, elements indexed by m

π i :

The realized payoff of player i at time t

c j :

A memory chunk, indexed by j

A j :

Total activation of chunk j

B j :

The base-level activation of chunk j

t q :

The time (or rounds) elapsed since a chunk was observed

γ :

The rate of chunk activation decay

w m :

Attention weight for element m in the context vector

Δ j :

Dissimilarity between the current context ω t and a chunk’s encoded context

θ :

The threshold value at which an action choice is made

δ :

Instantaneous strength of evidence, or drift of OUP process

τ :

Time during the decision rule (OUP process)

W(τ):

Wiener process

σ 2 :

The variance of the diffusion proce

X(τ):

The state, accumulated evidence, of the OUP process at time τ

References

  1. Abreu D., Rubinstein A. (1988) The structure of Nash equilibrium in repeated games with finite automata. Econometrica 56(6): 1259–1281

    Article  MathSciNet  MATH  Google Scholar 

  2. Anderson J. (2007) How can the human mind occur in the physical universe? (Vol. 3). Oxford University Press, Oxford

    Book  Google Scholar 

  3. Aoyagi M. (1996) Evolution of beliefs and the Nash equilibrium of normal form games. Journal of Economic Theory 70(2): 444–469

    Article  MathSciNet  MATH  Google Scholar 

  4. Banerjee D., Sen S. (2007) Reaching pareto-optimality in prisoner’s dilemma using conditional joint action learning. Autonomous Agents and Multi-Agent Systems 15(1): 91–108

    Article  MathSciNet  Google Scholar 

  5. Budescu D. V., Rapoport A. (1994) Subjective randomization in one- and two-person games. Journal of Behavioral Decision Making 7: 261–278

    Article  Google Scholar 

  6. Busemeyer J. (2002) Survey of decision field theory. Mathematical Social Sciences 43(3): 345–370

    Article  MathSciNet  MATH  Google Scholar 

  7. Cabrales, A., & Garcia-Fontes, W. (2000). Estimating learning models from experimental data. University of Pompeu Fabra working paper.

  8. Camerer C., Loewenstein G., Prelec D. (2005) Neuroeconomics: How neuroscience can inform economics. Journal of Economic Literature 43(1): 9–64

    Article  Google Scholar 

  9. Camerer C. F., Ho T. (1999) Experience-weighted attraction learning in normal-form games. Econometrica 67: 827–874

    Article  MATH  Google Scholar 

  10. Cheung Y. W., Friedman D. (1997) Individual learning in normal form games: Some laboratory results. Games and Economic Behavior 19: 46–76

    Article  MathSciNet  MATH  Google Scholar 

  11. Coricelli, G. (2005). Strategic interaction in iterated zero-sum games. Working paper, University of Arizona, Department of Economics.

  12. Cowan N. (2001) The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences 24(1): 87–114

    Article  Google Scholar 

  13. Diederich A., Busemeyer J. (2003) Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time. Journal of Mathematical Psychology 47(3): 304–322

    Article  MathSciNet  MATH  Google Scholar 

  14. Duersch P., Kolb A., Oechssler J., Schipper B. (2010) Rage against the machines: How subjects play against learning algorithms. Economic Theory 43(3): 407–430

    Article  MathSciNet  MATH  Google Scholar 

  15. Efron B. (1987) Better bootstrap confidence intervals. Journal of the American Statistical Association 82: 171–200

    Article  MathSciNet  MATH  Google Scholar 

  16. Efron B., Tibshirani R. (1994) An introduction to the bootstrap. Chapman and Hall, London

    Google Scholar 

  17. Engle-Warnick J., Slonim R. L. (2004) The evolution of strategies in a repeated trust game. Journal of Economic Behavior and Organization 55(4): 553–573

    Article  Google Scholar 

  18. Erev I., Haruvy E. (2005) On the potential uses and current limitations of data driven learning models. Journal of Mathematical Psychology 49(5): 357–371

    Article  MathSciNet  MATH  Google Scholar 

  19. Eurostat (2006, 13 July). Minimum wages in the eu25.

  20. Fox J. (1972) The learning of strategies in a simple, two-person zero-sum game without saddlepoint. Behavioral Science 17: 300–308

    Article  Google Scholar 

  21. Fudenberg D., Levine D. K. (1998) The theory of learning in games (Economics learning and social evolution). MIT Press, Cambridge

    Google Scholar 

  22. Gärdenfors P. (2004) Conceptual spaces. MIT Press, Cambridge

    Google Scholar 

  23. Gonzalez C. (2003) Instance-based learning in dynamic decision making. Cognitive Science 27(4): 591–635

    Article  Google Scholar 

  24. Gonzalez, C., Dutt, V., Healy, A., Young, M., & Bourne Jr, L. (2009). Comparison of instance and strategy models in ACT-R. Department of Social and Decision Sciences, 71.

  25. Gonzalez, C., & Lebiere, C. (2005). Instance-based cognitive models of decision making. In D. Zizzo & A. Courakis (Eds.), Transfer of knowledge in economic decision making (pp. 1–28). New York: Palgrave McMillan.

  26. Harrison G. W. (1989) Theory and misbehavior of first-price auctions. American Economic Review 79(4): 749–762

    Google Scholar 

  27. Ho T., Camerer C., Chong J. (2007) Self-tuning experience weighted attraction learning in games. Journal of Economic Theory 133(1): 177–198

    Article  MathSciNet  MATH  Google Scholar 

  28. Lebiere, C., & West R. L. (1999). A dynamic ACT-R model of simple games. Proceedings of the Twenty-first Conference of the Cognitive Science Society, pp. 296–301.

  29. Levitt S. D., List J. A., Reiley D. H. (2010) What happens in the field stays in the field: Exploring whether professionals play minimax in laboratory experiments. Econometrica 78(4): 1413–1434

    Article  MathSciNet  MATH  Google Scholar 

  30. Messick D. M. (1967) Interdependent decision strategies in zero-sum games: A computer-controlled study. Behavioral Science 12: 33–48

    Article  Google Scholar 

  31. Papke L., Wooldridge J. (2008) Panel data methods for fractional response variables with an application to test pass rates. Journal of Econometrics 145(1–2): 121–133

    Article  MathSciNet  Google Scholar 

  32. Rapoport A., Budescu D. (1997) Randomization in individual choice behavior. Psychological Review 104: 603–617

    Article  Google Scholar 

  33. Ratcliff R., Smith P. L. (2004) A comparison of sequential sampling models for two-choice reaction time. Psychological review 111(2): 333–367

    Article  Google Scholar 

  34. Roth A. E., Erev I. (1995) Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior 8(1): 164–212

    Article  MathSciNet  MATH  Google Scholar 

  35. Rubinstein A. (1986) Finite automata play the repeated prisoner’s dilemma. Journal of Economic Theory 39: 83–96

    Article  MathSciNet  MATH  Google Scholar 

  36. Rutström E. E., Wilcox N. T. (2009) Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test. Games and Economic Behavior 67(2): 616–632

    Article  MathSciNet  MATH  Google Scholar 

  37. Salmon T. C. (2001) An evaluation of econometric models of adaptive learning. Econometrica 69(6): 1597–1628

    Article  MathSciNet  MATH  Google Scholar 

  38. Schooler L., Anderson J. (1997) The role of process in the rational analysis of memory. Cognitive Psychology 32: 219–250

    Article  Google Scholar 

  39. Shachat, J., & Swarthout, J. (2011). Learning about learning in games through experimental control of strategic interdependence. Journal of Economic Dynamics and Control.

  40. Shachat J., Swarthout T. J. (2004) Do we detect and exploit mixed strategy play by opponents?. Mathematical Methods of Operations Research 59(3): 359–373

    Article  MathSciNet  MATH  Google Scholar 

  41. Sidak Z. (1967) Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association 62: 626–633

    MathSciNet  MATH  Google Scholar 

  42. Smith P. (2000) Stochastic dynamic models of response time and accuracy: A foundational primer. Journal of mathematical psychology 44(3): 408–463

    Article  MathSciNet  MATH  Google Scholar 

  43. Smith P. L., Ratcliff R. (2004) Psychology and neurobiology of simple decisions. Trends in neurosciences 27(3): 161–168

    Article  Google Scholar 

  44. Sonsino D. (1997) Learning to learn, pattern recognition, and Nash equilibrium. Games and Economic Behavior 18(2): 286–331

    Article  MathSciNet  MATH  Google Scholar 

  45. Sonsino D., Sirota J. (2003) Strategic pattern recognition: Experimental evidence. Games and Economic Behavior 44(2): 390–411

    Article  MATH  Google Scholar 

  46. Spiliopoulos, L. (2012). Pattern recognition and subjective belief learning in a repeated constant-sum game. Games and economic behavior, http://dx.doi.org/10.1016/j.geb.2012.01.005.

  47. Usher M., McClelland J. (2001) The time course of perceptual choice: The leaky, competing accumulator model. Psychological review 108(3): 550

    Article  Google Scholar 

  48. West R. L., Lebiere C. (2001) Simple games as dynamic, coupled systems: Randomness and other emergent properties. Cognitive Systems Research 1(4): 221–239

    Article  Google Scholar 

  49. Wilcox N. (2006) Theories of learning in games and heterogeneity bias. Econometrica 74(5): 1271–1292

    Article  MathSciNet  MATH  Google Scholar 

  50. Wong, T., Cokely E., & Schooler L. J. (2010). An online database of ACT-R parameters : Towards a transparent community-based approach to model development. Proceedings of ICCM (Vol. 1), pp. 282–286.

Download references

Author information

Authors and Affiliations

  1. The Australian School of Business, School of Economics, University of New South Wales, Sydney, NSW, 2052, Australia

    Leonidas Spiliopoulos

Authors
  1. Leonidas Spiliopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leonidas Spiliopoulos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Spiliopoulos, L. Strategic adaptation of humans playing computer algorithms in a repeated constant-sum game. Auton Agent Multi-Agent Syst 27, 131–160 (2013). https://doi.org/10.1007/s10458-012-9203-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-012-9203-z

Keywords

Search

Navigation