Skip to main content
SpringerLink
Log in

A multi-agent reinforcement learning approach to robot soccer

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

In this paper, a multi-agent reinforcement learning method based on action prediction of other agent is proposed. In a multi-agent system, action selection of the learning agent is unavoidably impacted by other agents’ actions. Therefore, joint-state and joint-action are involved in the multi-agent reinforcement learning system. A novel agent action prediction method based on the probabilistic neural network (PNN) is proposed. PNN is used to predict the actions of other agents. Furthermore, the sharing policy mechanism is used to exchange the learning policy of multiple agents, the aim of which is to speed up the learning. Finally, the application of presented method to robot soccer is studied. Through learning, robot players can master the mapping policy from the state information to the action space. Moreover, multiple robots coordination and cooperation are well realized.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baghaei KR, Agah A (2007) Multi-agent task allocation for robot soccer. J Intell Syst 16(3): 207–240

    Article  Google Scholar 

  • Jouffe L (1998) Inference system learning by reinforcement methods. IEEE Trans Syst Man Cybern 28(3): 338–355

    Article  Google Scholar 

  • Kim JH, Vadakepat P (2000) Multi-agent systems: a survey from the robot-soccer perspective. Int J Intell Autom Soft Comput 6(1): 3–17

    Google Scholar 

  • Littman ML (1994) Markov games as a framework for multiagent learning. In: Proceedings of the 11th international conference on machine learning. Morgan Kaufmann, San Francisco, pp 157–163

  • Liu F, Zeng GZ (2006) Multi-agent cooperative learning research based on reinforcement learning. In: 10th international conference on computer supported cooperative work in design. Nanjing, pp 1408–1413

  • Lu Y, Lin X (2006) Applications of neural network to short-term electric load forecasting. J Shenyang Univ Technol 28(1): 41–44

    MathSciNet  Google Scholar 

  • Moore AW, Atkeson CG (1995) The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces. Mach Learn 21(3): 199–233

    Google Scholar 

  • Murao H, Kitamura S (1997) Q-learning with adaptive state segmentation (QLASS). In: Proceedings of IEEE international symposium on computational intelligence in robotics and automation. pp 179–184

  • Reis LP, Lau N, Oliveira EC (2001) Situation based strategic positioning for coordinating a team of homogeneous agents. In: Lecture notes in artificial intelligence, vol 2103. pp 175–197

  • Specht DF (1990) Probabilistic neural networks. Neural Netw 3(1): 109–118

    Article  Google Scholar 

  • Specht DF (1990) Probabilistic neural networks and the polynomial adaline as complementary techniques for classification. IEEE Trans Neural Netw 1(1): 111–121

    Article  Google Scholar 

  • Stone P, Veloso M (2000) Multiagent systems: a survey from a machine learning perspective. Auton Robots 8(3): 345–383

    Article  Google Scholar 

  • Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press Cambridge, Massachusetts

    Google Scholar 

  • Taniguchi Y, Mori T, Ishii S (2007) Reinforcement learning for cooperative actions in a partially observable multi-agent system. In: Lecture notes in computer science, vol 4668. pp 229–238

  • Touzet CF (1997) Neural reinforcement learning for behaviour synthesis. Robotics Auton Syst 22(3): 251–281

    Article  Google Scholar 

  • Veloso M, Stone P, Han K (1998) CMUnited-97: RoboCup-97 small-robot world champion team. AI Magazine 19(3): 61–69

    Google Scholar 

  • Weinberg M, Rosenschein JS (2004) Best-response multiagent learning in non-stationary environments. In: Proceedings of the thrid international joint conference on autonomous agents and multiagent systems, vol 2. pp 506–513

  • Wu CJ, Lee TL (2004) A Fuzzy Mechanism for action selection of soccer robots. J Intell Robotic Syst 39(1): 57–70

    Article  MathSciNet  Google Scholar 

  • Xie SM, Chen C, Ding XY (2007) Endpoint prediction of basic-oxygen furnace based on BP neural network. J Shenyang Univ Technol 29(6): 707–710

    Google Scholar 

  • Yang E, Gu D (2004) Multiagent reinforcement learning for multi-robot systems: a survey. Technical Report CSM-404, Department of Computer Science, University of Essex

  • Zhong Y (2003) Research on distributed reinforcement learning theory and its application in multi-robot systems. Dissertation for the Degree, Harbin Engineering University

Download references

Author information

Authors and Affiliations

  1. Shenyang University of Technology, 110870, Shenyang, China

    Yong Duan & Bao Xia Cui

  2. Northeastern University, 11004, Shenyang, China

    Xin He Xu

Authors
  1. Yong Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bao Xia Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin He Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Duan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Duan, Y., Cui, B.X. & Xu, X.H. A multi-agent reinforcement learning approach to robot soccer. Artif Intell Rev 38, 193–211 (2012). https://doi.org/10.1007/s10462-011-9244-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-011-9244-8

Keywords

Search

Navigation