Skip to main content
SpringerLink
Log in

Coordinating actions in congestion games: impact of top–down and bottom–up utilities

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Congestion games offer a perfect environment in which to study the impact of local decisions on global utilities in multiagent systems. What is particularly interesting in such problems is that no individual action is intrinsically “good” or “bad” but that combinations of actions lead to desirable or undesirable outcomes. As a consequence, agents need to learn how to coordinate their actions with those of other agents, rather than learn a particular set of “good” actions. A congestion game can be studied from two different perspectives: (i) from the top down, where a global utility (e.g., a system-centric view of congestion) specifies the task to be achieved; or (ii) from the bottom up, where each agent has its own intrinsic utility it wants to maximize. In many cases, these two approaches are at odds with one another, where agents aiming to maximize their intrinsic utilities lead to poor values of a system level utility. In this paper we extend results on difference utilities, a form of shaped utility that enables multiagent learning in congested, noisy conditions, to study the global behavior that arises from the agents’ choices in two types of congestion games. Our key result is that agents that aim to maximize a modified version of their own intrinsic utilities not only perform well in terms of the global utility, but also, on average perform better with respect to their own original utilities. In addition, we show that difference utilities are robust to agents “defecting” and using their own intrinsic utilities, and that performance degrades gracefully with the number of defectors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Agogino, A., & Tumer, K. (2008). Regulating air traffic flow with coupled agents. In Proceedings of the seventh international joint conference on autonomous agents and multi-agent systems. Estoril, Portugal.

  2. Agogino A. K., Tumer K. (2008) Analyzing and visualizing multiagent rewards in dynamic and stochastic environments. Journal of Autonomous Agents and Multi Agent Systems 17(2): 320–338

    Article  Google Scholar 

  3. Agogino A. K., Tumer K. (2008) Efficient evaluation functions for evolving coordination. Evolutionary Computation 16(2): 257–288

    Article  Google Scholar 

  4. Arthur W. B. (1994) Complexity in economic theory: Inductive reasoning and bounded rationality. The American Economic Review 84(2): 406–411

    Google Scholar 

  5. Balmer, M., Cetin, N., Nagel, K., & Raney, B. (2004). Towards truly agent-based traffic and mobility simulations. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (pp. 60–67). New York, NY.

  6. Bando M., Hasebe K., Nakayama A., Shibata A., Sugiyama Y. (1995) Dynamical model of traffic congestion and numerical simulation. Physical Review E 51(2): 1035–1042

    Article  Google Scholar 

  7. Bazzan A., Kluegl F. (2009) Multiagent architectures for traffic and transportation engineering. Springer, Berlin

    Book  Google Scholar 

  8. Bazzan A. L., Klügl F. (2005) Case studies on the Braess paradox: Simulating route recommendation and learning in abstract and microscopic models. Transportation Research C 13(4): 299–319

    Article  Google Scholar 

  9. Bazzan, A. L., Wahle, J., & Klügl, F. (1999). Agents in traffic modelling—from reactive to social behaviour. In KI—Kunstliche Intelligenz (pp. 303–306).

  10. Boutilier, C. (1996). Planning, learning and coordination in multiagent decision processes. In Proceedings of the Sixth Conference on Theoretical Aspects of Rationality and Knowledge. Holland.

  11. Bowling M., Veloso M. (2002) Multiagent learning using a variable learning rate. Artificial Intelligence 136: 215–250

    Article  MathSciNet  MATH  Google Scholar 

  12. Brafman R. I., Tennenholtz M. (2004) Efficient learning equilibrium. Artificial Intelligence 159(1–2): 27–47

    Article  MathSciNet  MATH  Google Scholar 

  13. Burmeister B., Haddadi A., Matylis G. (1997) Application of multi-agent systems in traffic and transportation. IEEE Proceedings in Software Engineering 144(1): 51–60

    Article  Google Scholar 

  14. Carter C. R., Jennings N. R. (2002) Social responsibility and supply chain relationships. Transportation Research Part E 38: 37–52

    Article  Google Scholar 

  15. Challet D., Zhang Y. C. (1998) On the minority game: Analytical and numerical studies. Physica A 256: 514

    Article  Google Scholar 

  16. Cheng, J. (1997). The mixed strategy equilibria and adaptive dynamics in the bar problem. Tech. rep., Santa Fe Institute Computational Economics Workshop.

  17. Conitzer, V., & Sandholm, T. (2003). Complexity results about nash equilibria. In Proceedings of the 18th international joint conference on Artificial intelligence, IJCAI’03 (pp. 765–771).

  18. Dresner, K., & Stone, P. (2004). Multiagent traffic management: A reservation-based intersection control mechanism. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (pp. 530–537). New York, NY.

  19. Groves T. (1973) Incentives in teams. Econometrica: Journal of the Econometric Society 41: 617–631

    Article  MathSciNet  MATH  Google Scholar 

  20. Hall, S., & Draa, B. C. (2004). Collaborative driving system using teamwork for platoon formations. In The third workshop on Agents in Traffic and Transportation.

  21. Hardin G. (1968) The tragedy of the commons. Science 162: 1243–1248

    Article  Google Scholar 

  22. Helbing D. (1998) Structure and instability of high-density equations for traffic flow. Physical Review E 57(5): 6176–6179

    Article  Google Scholar 

  23. Helbing D. (2001) Traffic and related self-driven many-particle systems. Reviews of Modern Physics 73: 1067–1141

    Article  Google Scholar 

  24. Helbing D., Tilch B. (1998) Generalized force model traffic dynamics. Physical Review E 58(1): 133–138

    Article  Google Scholar 

  25. Hu, J., & Wellman, M. P. (1998). Multiagent reinforcement learning: Theoretical framework and an algorithm. In Proceedings of the fifteenth international conference on machine learning (pp. 242–250).

  26. Huberman, B. A., & Hogg, T. (1988). The behavior of computational ecologies. In The ecology of computation (pp. 77–115). New York: North-Holland.

  27. Ieong, S., McGrew, R., Nudelman, E., Shoham, Y., & Sun, Q. (2005). Fast and compact: A simple class of congestion games. In Proceedings of the 20th national conference on artificial intelligence—Volume 2, AAAI’05 (pp. 489–494).

  28. Jefferies, P., Hart, M. L., & Johnson, N. F. (2002) Deterministic dynamics in the minority game. Physical Review E, 65(016105).

  29. Jennings N. R., Sycara K., Wooldridge M. (1998) A roadmap of agent research and development. Autonomous Agents and Multi-Agent Systems 1: 7–38

    Article  Google Scholar 

  30. Kerner B. S., Rehborn H. (1996) Experimental properties of complexity in traffic flow. Physical Review E 53(5): R4275–4278

    Article  Google Scholar 

  31. Klügl, F., Bazzan, A., Ossowski, S. (Eds.) (2005) Applications of agent technology in traffic and transportation. Springer, Heidelberg

    MATH  Google Scholar 

  32. Lazar A. A., Orda A., Pendarakis D. E. (1997) Capacity allocation under noncooperative routing. IEEE Transactions on Networking 5(6): 861–871

    Article  Google Scholar 

  33. Littman, M. L. (1994). Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the 11th international conference on machine learning (pp. 157–163).

  34. Nagel, K. (2001). Multi-modal traffic in TRANSIMS. In Pedestrian and Evacuation Dynamics (pp. 161–172). Springer, Berlin.

  35. Oliveira, D., Bazzan, A. L. C., Silva, B. C., Basso, E. W., Nunes, L., Rossetti, R. J. F., Oliveira, E. C., Silva, R., & Lamb, L. C.: (2006). Reinforcement learning based control of traffic lights in non-stationary environments: a case study in a microscopic simulator. In B.D. Keplicz, A. Omicini, & J. Padget (Eds.), Proceedings of the 4th European workshop on multi-agent systems, EUMAS06 (pp. 31–42).

  36. Papadimitriou, C. (2001). Algorithms, games, and the internet. In Proceedings of the thirty-third annual ACM symposium on theory of computing, STOC ’01 (pp. 749–753).

  37. Parkes, D. (2004). On learnable mechanism design. In Collectives and the design of complex systems. New York: Springer.

  38. Parkes, D. C. (2001). Iterative combinatorial auctions: Theory and practice. Ph.D. thesis, University of Pennsylvania.

  39. Parkes, D. C., Shneidman, J. (2004). Distributed implementations of Vickrey-Clarke-Groves mechanisms. In Proceedings of the third international joint conference on autonomous agents and multiagent systems—volume 1, AAMAS ’04 (pp. 261–268). IEEE Computer Society.

  40. Porter R., Nudelman E., Shoham Y. (2006) Simple search methods for finding a nash equilibrium. Games and Economic Behavior, 63: 642–664

    Article  MathSciNet  Google Scholar 

  41. Rosenthal R. W. (1973) A class of games possessing pure-strategy nash equilibria. International Journal of Game Theory 2: 65–67

    Article  MathSciNet  MATH  Google Scholar 

  42. Sandholm T., Crites R. (1995) Multiagent reinforcement learning in the iterated prisoner’s dilemma. Biosystems 37: 147–166

    Article  Google Scholar 

  43. Stone P., Veloso M. (2000) Multiagent systems: A survey from a machine learning perspective. Autonomous Robots 8(3): 345–383

    Article  Google Scholar 

  44. Sutton R. S., Barto A. G. (1998) Reinforcement learning: An introduction. MIT Press, Cambridge, MA

    Google Scholar 

  45. Tennenholtz, M., & Zohar, A. (2009). Learning equilibria in repeated congestion games. In Proceedings of the 8th international conference on autonomous agents and multiagent systems—volume 1, AAMAS ’09 (pp. 233–240).

  46. Tesauro G., Kephart J. O. (2002) Pricing in agent economies using multi-agent q-learning. Autonomous Agents and Multi-Agent Systems 5: 289–304

    Article  Google Scholar 

  47. Tumer, K., & Agogino, A. (2007). Distributed agent-based air traffic flow management. In Proceedings of the sixth international joint conference on autonomous agents and multi-agent systems (pp. 330–337). Honolulu, HI.

  48. Tumer, K., Agogino, A. K., & Welch, Z. (2009) Traffic congestion management as a learning agent coordination problem. In: Bazzan, A., & Kluegl F. (Eds.) Multiagent architectures for traffic and transportation engineering. Berlin: Springer.

  49. Tumer, K., Welch, Z. T., & Agogino, A. (2008). Aligning social welfare and agent preferences to alleviate traffic congestion. In Proceedings of the seventh international joint conference on autonomous agents and multi-agent systems. Estoril, Portugal.

  50. Tumer, K., & Wolpert, D. (2004). A survey of collectives. In Collectives and the design of complex systems (pp. 1–42). Berlin: Springer.

  51. Tumer, K., & Wolpert, D. H. (2000). Collective intelligence and Braess’ paradox. In Proceedings of the seventeenth national conference on artificial intelligence (pp. 104–109). Austin, TX.

  52. Wiering, M. (2000) Multi-agent reinforcement leraning for traffic light control. In Proceedings of the seventeenth international conference on machine learning, ICML ’00 (pp. 1151–1158). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

  53. Wolpert D. H., Tumer K. (2001) Optimal payoff functions for members of collectives. Advances in Complex Systems 4(2/3): 265–279

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Oregon State University, Corvallis, OR, USA

    Kagan Tumer & Scott Proper

Authors
  1. Kagan Tumer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Scott Proper
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kagan Tumer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tumer, K., Proper, S. Coordinating actions in congestion games: impact of top–down and bottom–up utilities. Auton Agent Multi-Agent Syst 27, 419–443 (2013). https://doi.org/10.1007/s10458-012-9211-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-012-9211-z

Keywords

Search

Navigation