Abstract
We recently proposed swarm reinforcement learning methods in which multiple agents are prepared and they learn not only by individual learning but also by learning through exchanging information among the agents. The methods have been applied to a problem in discrete state-action space so far, and Q-learning method has been used as the individual learning. Although many studies in reinforcement learning have been done for problems in the discrete state-action space, continuous state-action space is required for coping with most real-world tasks. This paper proposes a swarm reinforcement learning method based on an actor-critic method in order to acquire optimal policies rapidly for problems in the continuous state-action space. The proposed method is applied to an inverted pendulum control problem, and its performance is examined through numerical experiments.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning. MIT Press, Cambridge (1998)
Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco (2001)
Iima, H., Kuroe, Y.: Reinforcement Learning through Interaction among Multiple Agents. In: SICE-ICASE International Joint Conference, pp. 2457–2462 (2006)
Iima, H., Kuroe, Y.: Swarm Reinforcement Learning Algorithms Based on Particle Swarm Optimization. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 1110–1115 (2008)
Watkins, C.J.C.H., Dayan, P.: Q-Learning. Machine Learning 8, 279–292 (1992)
Busoniu, L., Babuska, R., Schutter, B.D.: A Comprehensive Survey of Multiagent Reinforcement Learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C 38, 156–172 (2008)
Kimura, H., Kobayashi, S.: An Analysis of Actor/Critic Algorithms using Eligibility Traces: Reinforcement Learning with Imperfect Value Function. In: 15th International Conference on Machine Learning, pp. 278–286 (1998)
Doya, K.: Reinforcement Learning in Continuous Time and Space. Neural Computation 12, 219–245 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Iima, H., Kuroe, Y. (2010). Swarm Reinforcement Learning Method Based on an Actor-Critic Method. In: Deb, K., et al. Simulated Evolution and Learning. SEAL 2010. Lecture Notes in Computer Science, vol 6457. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17298-4_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-17298-4_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17297-7
Online ISBN: 978-3-642-17298-4
eBook Packages: Computer ScienceComputer Science (R0)