Abstract
Some multiagent learning methods simply extend reinforcement learning to multiple agents. In these methods, large state and action spaces are the most difficult problems. Moreover, previous proposals for using learning techniques to coordinate multiple agents have mostly relied on explicit or implicit information sharing, which makes cooperation affected by communication delays and the reliability of the information received. A Multiagent Cooperative Learning Algorithm (MCLA) is presented to solve these problems. In MCLA, an evaluating strategy based on long-time reward is proposed. Thus each agent acts independently and autonomously by perceiving and estimating each other. It also considers the learning process from the holistic point of view to obtain the optimum associated action strategy in order to reduce the state and action spaces. A series of simulations are provided to demonstrate the performance of the proposed algorithm.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Mikami, S.: Reinforcement learning for multi-agent systems. J. Japan Soc. Artif. Intell. 12, 845–849 (1997)
Arai, S.: Multiagent reinforcement learning: Uniting planning, theory, and various techniques for implementation. J. Japan Soc. Artif. Intell. 16, 476–481 (2001)
Wolpert, D.H., Sill, J., Tumer, K.: Reinforcement learning in distributed domains: Beyond team games. In: Proc. 17th National Conference on Artificial Intelligence, Seattle, WA, pp. 819–824 (2001)
Arai, S., Sycara, K.: Credit Assignment Method for Learning Effective Stochastic Policies in Uncertain Domains. In: Proc. Genetic and Evolutionary Computation Conf. (2001)
Arai, S., Sycara, K.: Effective Learning Approach for Planning and Scheduling in Multi-agent Domain. In: Proc. 6th ISAB - From Animals to Animats 6, pp. 507–516 (2000)
Arai, S., Sycara, K., Payne, T.: Multi-agent Reinforcement Learning for Planning and Scheduling Multiple Goals. In: Proc. 4th Int. Conf. on MAS (2000)
Tan, M.: Multi-agent reinforcement learning: Independent vs. cooperative agents. In: Proc. Tenth Int. Conf. Machine Learning, Amherst, MA, pp. 330–337 (1993)
Ahmadabadi, M.N., Asadpour, M.: Expertness Based Cooperative Q-learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B 32(1), 66–76 (2002)
Sousa, C., Custodio, L.: Cooperative Reinforcement Learning: exploring Communication and Cooperation Problems. In: Proc. 2005 IEEE International Symposium on Computational Intelligence in Robotics and Automation, Espoo, Finland, pp. 445–450 (2005)
Grefenstette, J.: Lamarckian learning in multi-agent environments. In: Proc. Fourth International Conference on Genetic Algorithms, San Mateo, CA, pp. 303–310 (1991)
Bull, L., Fogarty, T.C.: Evolving cooperative communicating classifier systems. In: Proc. Fourth Annual Conference on Evolutionary Programming (EP94), pp. 308–315 (1994)
Iba, H.: Evolutionary learning of communicating agents. Information Sciences 108 (1998)
Miconi, T.: When evolving populations is better than coevolving individuals: The blind mice problem. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03) (2003)
Berenji, H., Vengerov, D.: Learning, cooperation, and coordination in multi-agent systems. Technical Report IIS-00-10, Intelligent Inference Systems Corp., Sunnyvale, CA (2000)
Eshgh, S.M., Ahmadabadi, M.N.: An Extension of Weighted Strategy Sharing in Cooperative Q-learning for Specialized Agents. In: Proceedings of the 9th International Conference on Neural Information Processing, vol. 1, pp. 106–110 (2002)
Buchsbaum, D., et al.: A Simulation-Theory Inspired Social Learning System for Interactive Characters. In: Proc. 2005 IEEE International Workshop on Robots and Human Interactive Communication, pp. 85–90 (2005)
Bryson, J.J., Wood, M.A.: Learning discretely: Behaviour and organization in social learning. In: Proc. Third International Symposium on Imitation in Animals and Artifacts AISB, pp. 30–37 (2005)
Yanai, K., Iba, H.: Multi-agent Robot Learning by Means of Genetic Programming: Solving an Escape Problem. In: Liu, Y., et al. (eds.) ICES 2001. LNCS, vol. 2210, pp. 192–203. Springer, Heidelberg (2001)
Bowling, M.: Convergence problems of general-sum multi-agent reinforcement learning. In: Proceedings of the Seventeenth International Conference on Machine Learning, San Francisco, CA, pp. 89–94 (2000)
Matsuura, K.: A study on heterogeneous agents systems learning organized behavior. Doctoral Dissertation. Department of Engineering, Hokkaido University (1996)
Arai, S., Sycara, K.: Effective learning approach for planning and scheduling in multi-agent domain. In: Proceedings of the Sixth ISAB: From Animals to Animats, vol. 6, pp. 507–516 (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, F., Zeng, G. (2007). A Multiagent Cooperative Learning Algorithm. In: Shen, W., Luo, J., Lin, Z., Barthès, JP.A., Hao, Q. (eds) Computer Supported Cooperative Work in Design III. CSCWD 2006. Lecture Notes in Computer Science, vol 4402. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72863-4_75
Download citation
DOI: https://doi.org/10.1007/978-3-540-72863-4_75
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72862-7
Online ISBN: 978-3-540-72863-4
eBook Packages: Computer ScienceComputer Science (R0)