Skip to main content
SpringerLink
Log in

Resource Allocation in the Grid with Learning Agents

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

One of the main challenges in Grid computing is efficient allocation of resources (CPU – hours, network bandwidth, etc.) to the tasks submitted by users. Due to the lack of centralized control and the dynamic/stochastic nature of resource availability, any successful allocation mechanism should be highly distributed and robust to the changes in the Grid environment. Moreover, it is desirable to have an allocation mechanism that does not rely on the availability of coherent global information. In this paper we examine a simple algorithm for distributed resource allocation in a simplified Grid-like environment that meets the above requirements. Our system consists of a large number of heterogenous reinforcement learning agents that share common resources for their computational needs. There is no explicit communication or interaction between the agents: the only information that agents receive is the expected response time of a job it submitted to a particular resource, which serves as a reinforcement signal for the agent. The results of our experiments suggest that even simple reinforcement learning can indeed be used to achieve load balanced resource allocation in large scale heterogenous system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. Schaerf, Y. Shoham and M. Tennenholtz, “Adaptive Load Balancing: A Studying Multi-Agent Learning”, Journal of Artificial Intelligence Research, Vol. 2, pp. 475–500, 1995.

    MATH  Google Scholar 

  2. A. Galstyan, S. Kolar and K. Lerman, “Resource Allocation Games with Changing Resource Capacities”, in Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-2003), Melbourne, Australia, 2003.

    Google Scholar 

  3. I. Foster, C. Kesselman and S. Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Interna-tional Journal of High Performance Computing Applications, Vol. 15, No. 3, pp. 200–222, 2001, http://www.globus.org/research/papers/anatomy.pdf.

    Article  Google Scholar 

  4. K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith and S. Tuecke, “A Resource Management Architecture for Metacomputing Systems”, in The 4th Workshop on Job Scheduling Strategies for Parallel Processing, pp.62–82, 1998.

  5. K. Czajkowski, I. Foster, C. Kesselman, V. Sander and S. Tuecke, “SNAP: A Protocol for Negotiating Service Level Agreements and Coordinating Resource Management in Distributed Systems”, Lecture Notes in Computer Science, Vol. 2537, pp. 153–183, 2002.

    Article  Google Scholar 

  6. R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction A Bradford Book, MIT, Cambridge, MA 1998.

    Google Scholar 

  7. C. Claus and C. Boutilier, “The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems”, in Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), pp.746–752, 1998.

  8. S. Kapetanakis and D. Kudenko, “Reinforcement Learning of Coordination in Cooperative Multi-agent Systems”, in Proceedings of the 17th National Conference on Artificial Intelligence (AAAI-02), Edmonton, Alberta, Canada, 2002.

    Google Scholar 

  9. Y. Shoham, T. Grenager and R. Powers, “Multi-agent Reinforcement Learning: A Critical Survey”, Web manuscript, 2003.

  10. C.J.C.H. Watkins, Learning from Delayed Rewards. PhD thesis, Cambridge University, Cambridge, England, 1989.

  11. B. Barish and R. Weiss, “LIGO and the Detection of Gravitational Waves”, Physics Today, Vol. 52, No. 10, pp. 44, 1999.

    Article  Google Scholar 

  12. K. Ranganathan and I. Foster, “Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applicationsn”, in 11th IEEE International Symposium on High Performance Distributed Computing (HPDC-11), Edinburgh, Scotland, 2002.

    Google Scholar 

  13. J.P. Crutchfield and Y. Sato, “Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems,” Physical Review E, Vol. 67, pp. 40–43, 2003.

    Google Scholar 

  14. R.W. Rosenthal, “A Class of Games Possessing Pure-strategy Nash Equilibria”, International Journal of Game Theory, Vol. 2, pp. 65–67, 1973.

    Article  MATH  MathSciNet  Google Scholar 

  15. W.B. Arthur, “Inductive Reasoning and Bounded Rationality”, American Economic Review, Vol. 84, pp. 406–411, 1994.

    Google Scholar 

  16. D. Challet and Y.-C. Zhang, “Emergence of Cooperation and Organization in an Evolutionary Game”, Physica A, Vol. 246, pp. 407, 1997.

    Article  Google Scholar 

  17. N. Johnson, P. Hui, D. Zheng and C. Tai, “Minority Game with Arbitrary Cuttoffs”, Physica A, Vol. 269, pp. 493, 1999.

    Article  Google Scholar 

  18. A. Galstyan and K. Lerman, “Adaptive Boolean Networks and Minority Games with Time-Dependent Capacities”, Physical Review, Vol. E66, 015103, 2002.

  19. K. Czajkowski, S. Fitzgerald, I. Foster and C. Kesselman, “Grid Information Services for Distributed Resource Sharing”. in Proc. 10th IEEE Symp. on High Performance Distributed Computing, IEEE Computer Society, 2001.

  20. UDDI, “Universal Description Discovery and Integration (UDDI)”, http://www.uddi.org, 2001.

  21. R. Wolski, “Forecasting Network Performance to Support Dynamic Scheduling Using the Network Weather Service”, in Proc. 6th IEEE Symp. on High Performance Distributed Computing Portland Oregon, IEEE, 1997.

    Google Scholar 

  22. P. Stelling, I. Foster, C. Kesselman, C. Lee and G. von Laszewski, “A Fault Detection Service for Wide Area Distributed Computations,” in Proc. 7th IEEE Symp. on High Performance Distributed Computing, pp. 268–278, 1998.

  23. Veridian Systems, “Portable Batch System”, Internet document. http://pbs.mrj.com, 2000.

  24. Platform Computing Corporation, “LSF Administrator's Guide”, Platform Computing Corporation, 1996.

  25. M. Livny, “Matchmaking: Distributed Resource Management for High Throughput Computing”, in Proc. 7th IEEE Symp. on High Performance Distributed Computing, 1998.

  26. R. Raman, M. Livny and M. Solomon, “Resource Management through Multilateral Matchmaking”, in Proc. 9th IEEE Symp. on High Performance Distributed Computing, 2000.

  27. E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, S. Patil, M. Su and K. Vahi, “Pegasus: Mappin Scientific Workflows onto the Grid”, in Proceedings of across Grid EU Conference, 2004.

  28. E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, K. Blackburn, A. Lazzarini, A. Arbree, R. Cavanaugh and S. Koranda, “Mapping Abstract Complex Workflows onto Grid Environments,” Journal of Grid Computing, Vol. 1, No. 1, pp. 25–39, 2003.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Marina del Rey, CA, 90292-6695, USA

    Aram Galstyan, Karl Czajkowski & Kristina Lerman

Authors
  1. Aram Galstyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karl Czajkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kristina Lerman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aram Galstyan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Galstyan, A., Czajkowski, K. & Lerman, K. Resource Allocation in the Grid with Learning Agents. J Grid Computing 3, 91–100 (2005). https://doi.org/10.1007/s10723-005-9003-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-005-9003-7

Key words

Search

Navigation