Aram Galstyan
ABSTRACT
In this paper we study a class of resource allocation games which are inspired by the El Farol Bar problem. We consider a system of competitive agents that have to choose between several resources characterized by their time dependent capacities. The agents using a particular resource are rewarded if their number does not exceed the resource capacity, and punished otherwise. Agents use a set of strategies to decide what resource to choose, and use a simple reinforcement learning scheme to update the accuracy of strategies. A strategy in our model is simply a lookup table that suggests to an agent what resource to choose based on the actions of its neighbors at the previous time step. In other words, the agents form a social network whose connectivity controls the average number of neighbors with whom each agent interacts. This statement of the adaptive resource allocation problem allows us to fully parameterize it by a small set of numbers. We study the behavior of the system via numeric simulations of 100 to 5000 agents using one to ten resources. Our results indicate that for a certain range of parameters the system as a whole adapts effectively to the changing capacity levels and results in very little under- or over-utilization of the resources.
- W. B. Arthur. Inductive reasoning and bounded rationality. American Economic Review, 84:406--411, 1994.Google Scholar
- R. Axelrod and W. D. Hamilton. The evolution of cooperation. Science, 211:1390--1396, 1981.Google ScholarCross Ref
- C. Boutilier, M. Goldszmidt, and B. Sabata. Sequential Auctions for the Allocation of Resources with Complimentarities. In International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, Aug. 1999. Google ScholarDigital Library
- A. Cavagna. Irrelevance of memory in the minority game. Phys. Rev. E, 59:R3783, 1999.Google ScholarCross Ref
- D. Challet and M. Marsili. Phase transition and symmetry breaking in the minority game. Phys. Rev. E, 60:R6271, 1999.Google ScholarCross Ref
- D. Challet and Y.-C. Zhang. Emergence of cooperation and organization in an evolutionary game. Physica A, page 407, 1997.Google Scholar
- D. Challet and Y.-C. Zhang. On the minority game: Analytical and numerical studies. Physica A, 256:514, 1998.Google ScholarCross Ref
- A. Chavez, A. Moukas, and P. Maes. Challenger: A Multi-agent System for Distributed Resource Allocation. In Proc. of Autonomous Agents, Marina del Rey, CA, Feb. 1997. Google ScholarDigital Library
- C. Claus and C. Boutilier. The dynamics of reinforcement learning in cooperative multiagent systems. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), pages 746--752, July 1998. Google ScholarDigital Library
- D. Fudenberg and D. K. Levine. The Theory of Learning in Games. MIT Press, Cambridge, MA, 1998.Google Scholar
- A. Galstyan and K. Lerman. Adaptive boolean networks and minority games with time-dependent capacities. Physical Review, E66:015103, 2002.Google Scholar
- N. Johnson, P. Hui, D. Zheng, and C. Tai. Minority game with arbitrary cuttoffs. Physica A, 269:493, 1999.Google ScholarCross Ref
- S. Kapetanakis and D. Kudenko. Reinforcement learning of coordination in cooperative multi agent systems. In Proceedings of the 17th National Conference on Artificial Intelligence (AAAI-02), Edmonton, Alberta, Canada, July 2002. Google ScholarDigital Library
- S. A. Kauffman. The Origins of Order. Oxford University Press, New York, 1993.Google Scholar
- J. F. Kurose and R. Simha. A microeconomic approach to optimal resource allocation in distributed computer systems. IEEE Transactions on Computers, 38(5):705--717, 1989. Google ScholarDigital Library
- See http://www.unifr.ch/econophysics/minority/ for an extensive collection of articles and references.Google Scholar
- T. Mullen and M. Wellman. Market-Based Negotiation for Digital Library Services. In Proc. of the 2nd USENIX Workshop on Electronic Commerce, Oakland, CA, Nov. 1996.Google Scholar
- R. W. Rosenthal. A class of games possessing pure-strategy nash equilibria. International Journal of Game Theory, 2:65--67, 1973.Google ScholarDigital Library
- T. Sandholm. Limitations of the Vickrey Auction in Computational Multiagent Systems. In International Conference on Multi-Agent Systems (ICMAS), pages 299--306, Kyoto, Japan, Dec. 1996.Google Scholar
- T. Sandholm. An Algorithm for Optimal Winner Determination in Combinatorial Auctions. In International Joint Conference on Artificial Intelligence (IJCAI), pages 542--547, Stockholm, Sweden, Aug. 1999. Google ScholarDigital Library
- T. Sandholm and S. Suri. Market Clearability. In International Joint Conference on Artificial Intelligence (IJCAI), pages 1145--1151, Seattle, WA, 2001. Google ScholarDigital Library
- R. Savit, R. Manuca, and R. Riolo. Adaptive competition, market efficiency, phase transition. Phys. Rev. Lett., 82(10):2203, 1999.Google ScholarCross Ref
- A. Schaerf, Y. Shoham, and M. Tennenholtz. Adaptive load balancing: A study in multi-agent learning. Journal of Artificial Intelligence Research, 2:475--500, 1995. Google ScholarDigital Library
- S. Sen, M. Sekaran, and J. Hale. Learning to coordinate without sharing information. In (American) National Conference on Artificial Intelligence, pages 426--431, Menlo Park, CA, 1994. AAAI Press. Google ScholarDigital Library
- R. G. Smith. The Contract Net Protocol. IEEE Tranactions on Computers, 29(12):1104--1113, Dec. 1980.Google ScholarDigital Library
- R. V. Sole, B. Luque, and S. A. Kauffman. Phase Transitions in Random Networks with Multiple States. SFI Working Papers, 00-02-011, 2000.Google Scholar
- M. Tan. Multi-Agents Reinforcement Learning: Independent vs Cooperative agents. In Proceeding of the $10^th$ International Conference on Machine Learning (ICML-93), 1993.Google Scholar
- J. M. Vidal and E. H. Durfee. The moving target function problem in multi-agent learning. In Proceedings of the 3rd International Conference on Multi-Agent Systems (ICMAS-98), 1998. Google ScholarDigital Library
- J. von Neumann and O. Morgenstern. Theory of Games and Economic Behavior. Princeton University Press, Princeton, NJ, 1944.Google Scholar
- W. E. Walsh and M. P. Wellman. A Market Protocol for Decentralized Task Allocation. In International Conference on Multi-Agent Systems (ICMAS), Paris, France, July 1998. Google ScholarDigital Library
- C. J. C. H. Watkins. Learning from Delayed Rewards. PhD thesis, Cambridge University, Cambridge, England, 1989.Google Scholar
- M. P. Wellman. Market-Oriented Programming: Some Early Lessons. In S. H. Clearwater, editor, Market-Based Control: A Paradigm for Distributed Resource Allocation, pages 74--95. World Scientific, Jan. 1996. Google ScholarDigital Library
- F. Ygge and H. Akkermans. Decentralized Markets versus Central Control: A Comparative Study. Journal of Artificial Intelligence Research, 11:301--333, 1999.Google ScholarDigital Library
Index Terms
- Resource allocation games with changing resource capacities
Recommendations
Resource reconstruction algorithms for on-demand allocation in virtual computing resource pool
Resource reconstruction algorithms are studied in this paper to solve the problem of resource on-demand allocation and improve the efficiency of resource utilization in virtual computing resource pool. Based on the idea of resource virtualization and ...
Reinforcement Learning-Based Resource Allocation in Edge Computing
Artificial Intelligence and SecurityAbstractThe problem of online resource allocation in edge computing has become a research hotspot. Meanwhile, reinforcement learning (RL) is suitable for solving online problems. In this paper, we combine edge computing online resource allocation with RL. ...
An Adaptive Strategy for Allocation of Resources with Gradually or Abruptly Changing Capacities
ICTAI '08: Proceedings of the 2008 20th IEEE International Conference on Tools with Artificial Intelligence - Volume 02In some resource allocation problems, the capacities of resources may change gradually or abruptly. We study a class of such resource allocation problems in this paper. The system consists of competitive agents that choose among several resources with ...
Comments