Dmitri A. Dolgov
ABSTRACT
Optimal resource scheduling in multiagent systems is a computationally challenging task, particularly when the values of resources are not additive. We consider the combinatorial problem of scheduling the usage of multiple resources among agents that operate in stochastic environments, modeled as Markov decision processes (MDPs). In recent years, efficient resource-allocation algorithms have been developed for agents with resource values induced by MDPs. However, this prior work has focused on static resource-allocation problems where resources are distributed once and then utilized in infinite-horizon MDPs. We extend those existing models to the problem of combinatorial resource scheduling, where agents persist only for finite periods between their (predefined) arrival and departure times, requiring resources only for those time periods. We provide a computationally efficient procedure for computing globally optimal resource assignments to agents over time. We illustrate and empirically analyze the method in the context of a stochastic job-scheduling domain.
- E. Altman and A. Shwartz. Adaptive control of constrained Markov chains: Criteria and policies. Annals of Operations Research, special issue on Markov Decision Processes, 28:101--134, 1991. Google Scholar
Digital Library
- R. Bellman. Dynamic Programming. Princeton University Press, 1957. Google ScholarDigital Library
- C. Boutilier. Solving concisely expressed combinatorial auction problems. In Proc. of AAAI-02, pages 359--366, 2002. Google ScholarDigital Library
- C. Boutilier and H. H. Hoos. Bidding languages for combinatorial auctions. In Proc. of IJCAI-01, pages 1211--1217, 2001. Google ScholarDigital Library
- D. Dolgov. Integrated Resource Allocation and Planning in Stochastic Multiagent Environments. PhD thesis, Computer Science Department, University of Michigan, February 2006. Google ScholarDigital Library
- D. A. Dolgov and E. H. Durfee. Optimal resource allocation and policy formulation in loosely-coupled Markov decision processes. In Proc. of ICAPS-04, pages 315--324, June 2004.Google Scholar
- D. A. Dolgov and E. H. Durfee. Computationally efficient combinatorial auctions for resource allocation in weakly-coupled MDPs. In Proc. of AAMAS-05, New York, NY, USA, 2005. ACM Press. Google ScholarDigital Library
- D. A. Dolgov and E. H. Durfee. Resource allocation among agents with preferences induced by factored MDPs. In Proc. of AAMAS-06, 2006. Google ScholarDigital Library
- K. Larson and T. Sandholm. Mechanism design and deliberative agents. In Proc. of AAMAS-05, pages 650--656, New York, NY, USA, 2005. ACM Press. Google ScholarDigital Library
- N. Nisan. Bidding and allocation in combinatorial auctions. In Electronic Commerce, 2000. Google ScholarDigital Library
- D. C. Parkes and S. Singh. An MDP-based approach to Online Mechanism Design. In Proc. of the Seventeenths Annual Conference on Neural Information Processing Systems (NIPS-03), 2003.Google Scholar
- D. C. Parkes, S. Singh, and D. Yanovsky. Approximately efficient online mechanism design. In Proc. of the Eighteenths Annual Conference on Neural Information Processing Systems (NIPS-04), 2004.Google Scholar
- M. L. Puterman. Markov Decision Processes. John Wiley & Sons, New York, 1994.Google ScholarDigital Library
- M. H. Rothkopf, A. Pekec, and R. M. Harstad. Computationally manageable combinational auctions. Management Science, 44(8):1131--1147, 1998. Google ScholarDigital Library
- T. Sandholm. An algorithm for optimal winner determination in combinatorial auctions. In Proc. of IJCAI-99, pages 542--547, San Francisco, CA, USA, 1999. Morgan Kaufmann Publishers Inc. Google ScholarDigital Library
Index Terms
- Combinatorial resource scheduling for multiagent MDPs
Recommendations
Resource allocation among agents with preferences induced by factored MDPs
AAMAS '06: Proceedings of the fifth international joint conference on Autonomous agents and multiagent systemsDistributing scarce resources among agents in a way that maximizes the social welfare of the group is a computationally hard problem when the value of a resource bundle is not linearly decomposable. Furthermore, the problem of determining the value of a ...
Computationally-efficient combinatorial auctions for resource allocation in weakly-coupled MDPs
AAMAS '05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systemsThe capabilities of an autonomous agent are often determined by the resources that are available to it. We examine the problem of allocating scarce resources among multiple self-interested agents, operating in complex, stochastic environments (modeled ...
Subjective approximate solutions for decentralized POMDPs
AAMAS '07: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systemsA problem of planning for cooperative teams under uncertainty is a crucial one in multiagent systems. Decentralized partially observable Markov decision processes (DEC-POMDPs) provide a convenient, but intractable model for specifying planning problems ...
Comments