Abstract
The application of reinforcement learning to multi-agent systems has attracted recent attention. In multi-agent systems, the state space to be handled constitutes a major problem efficiently in learning of agents. In order to cooperate agents in the same environment, it is needed to observe and evaluate the action of other agents in the multi-agent system. This case increases the dimension of state space proportional to the number of agents, exponentially. This paper presents a novel approach to overcome this problem. The approach uses together the advantages of the modular architecture, internal model and fuzzy logic in multi-agent systems. In our cooperation method, one agent estimates its action according to the internal model of the other agent. The internal model is acquired by the observation and evaluation of the other agent’s actions. Fuzzy logic maps from input fuzzy sets, representing state space of each learning module to the output fuzzy sets representing the action space. The fuzzy rule base of each learning module is built through the Qlearning. Experimental results handled on pursuit domain show the effectiveness and applicability of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
H. Berenji and D. Vengerov, “Cooperation and coordination between fuzzy reinforcement learning agents in continuous state partially observable markov decision processes”, In Proceedings of the 8 th IEEE International Conference on Fuzzy Systems (FUZZ-IEEE’99) 1999.
H. Berenji and D. Vengerov, “Advantage of cooperation between reinforcement learning agents in difficult stochastic problems”, In Proceedings of the 9th IEEE International Conference on Fuzzy Systems (FUZZ-IEEE’00) 2000.
R. Howard. “Dynamic Programming and Markov Processes” The MIT Press, 1960.
J. Hu and M. P. Wellman, “Multiagent reinforcement learning: theoretical framework and an algorithm” In Proceedings of the Fifteenth International Conference on Machine Learning (ICML-98), pp:242–250, 1998.
M. Kaya and A. Kilič. “Fuzzy-Reinforcement Learning in Cooperative Multi-Agent Systems”, International Symposium on Computer and Information Sciences (ISCIS 2001), Turkey, November 5–7, 2001.
T. Kohri, K. Matsubayashi, and M. Tokoro. “An adaptive architecture for modular Qlearning”, In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97), pages 820–825, 1997.
L. J. Lin. “Self-improving reactive agents based on reinforcement learning, planning and teaching”, Machine Learning, Vol: 8, pp: 293–321, 1992.
M. L. Littman. “Markov games as a framework for multi agent reinforcement learning”, Proceedings of the Eleventh International Conference on Machine Learning, pp. 157–163. San Francisco, CA 1994.
Y. Nagayuki, S. Ishii and K. Doya. “Multi-Agent Reinforcement Learning: An Approach Based on the Other Agent’s Internal Model” Fourth International Conference on MultiAgent Systems (ICMAS), 215–221, Los Alamitos: IEEE Computer Society (2000).
N. Ono and K. Fukomoto, “Multi-agent reinforcement learning: A modular approach”. In Proceedings of the Second International Conference on Multi-Agent Systems (ICMAS96), pp: 252–258, 1996.
K. H. Park, Y. J. Kim and J. H. Kim. “Modular Q-learning based multi-agent cooperation for robot soccer”, Robotics and Autonomous Systems Vol:35, pp:109–122, 2001.
T.W. Sandholm and R. H. Crites. “Multi agent reinforcement learning in the Iterated Prisoner’s Dilemma”, Biosystems, 37:147–166, 1995.
M. Tan, “Multi-agent reinforcement learning: independent vs. cooperative agents” In Proceedings of the Tenth International Conference on Machine Learning (ICML-93), pp:330–337, 1993.
C. J. C. H. Watkins and P. Dayan. “Technical Note: Q-Learning” Machine Learning, 8:279–92, 1992.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gültekin, I., Arslan, A. (2002). Modular-Fuzzy Cooperation Algorithm for Multi-agent Systems. In: Yakhno, T. (eds) Advances in Information Systems. ADVIS 2002. Lecture Notes in Computer Science, vol 2457. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36077-8_26
Download citation
DOI: https://doi.org/10.1007/3-540-36077-8_26
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00009-9
Online ISBN: 978-3-540-36077-3
eBook Packages: Springer Book Archive